AI

Local AI services running on the kontti server.

Ollama and Qdrant are reachable only from inside the container network — see Keeping the backends off the LAN for why.

Ollama

Ollama runs local language models and exposes them via a REST API on port 11434. Inference currently runs on the CPU, not the GPU — see below for how that turned out to be the case.

[Container]
Image=docker.io/ollama/ollama:latest

# GPU device access — passed through, but currently inert (see below)
AddDevice=/dev/kfd
AddDevice=/dev/dri
Environment=OLLAMA_VULKAN=1
Environment=HSA_OVERRIDE_GFX_VERSION=11.0.2

# One model resident at a time — the CPU default is three
Environment=OLLAMA_MAX_LOADED_MODELS=1

# Deliberately no PublishPort — see below
Network=ai.network

[Service]
# Largest model on disk (8.4 GiB) + KV cache + runtime overhead
MemoryMax=12G

The GPU lines that do nothing

This page used to state that Ollama ran on the AMD GPU via the Vulkan backend, with HSA_OVERRIDE_GFX_VERSION needed to stop it falling back to CPU inference. That was written from the configuration, and the configuration was wrong about itself.

Ollama 0.32 drops an integrated GPU by default unless OLLAMA_IGPU_ENABLE=1 is set. It says so in its own log — dropping integrated GPU — and reports total_vram="0 B". So both device nodes are passed through, OLLAMA_VULKAN=1 is set, the GFX override is set, and none of it takes effect: every model is loaded into host RAM and every token is generated on the CPU. The lines stay in the unit with a comment recording that they are currently inert, because enabling the iGPU is a separate change — and one that would also move the memory sizing below.

This surfaced from measuring memory, not from reading the config, which is the point worth keeping. Nothing in the quadlet, in systemctl status, or on this page contradicted the GPU story; only the running process did. It's the same rule as trusting the process over the management layer, applied to a performance claim rather than a security one.

Sizing a limit for a workload that hadn't run

MemoryMax=12G comes from the measured-not-guessed sizing applied across kontti, and it's the case where the measurement was most misleading. Ollama's 14-day peak RSS was 349 MiB — a number that says nothing useful, because no large model was run inside the measurement window. The real driver is the largest model blob on disk, 8.4 GiB, and since inference runs on the CPU that whole model lands in host RAM. The ceiling is that plus KV cache and runtime overhead.

OLLAMA_MAX_LOADED_MODELS=1 exists for the same reason. On CPU inference Ollama's default keeps three models resident at once — up to roughly 25 GiB on a 29 GiB host. MemoryMax would stop that, but by OOM-killing the service; capping the loaded-model count stops it cleanly, with Ollama evicting the previous model instead of being killed. The two consumers (Open WebUI and plex-sync) never run concurrently, so nothing here needs a second model in memory.

Open WebUI

Open WebUI provides a ChatGPT-like web interface for Ollama. It connects to Ollama's API and supports model selection, conversation history, and document uploads.

Authentication is enabled (WEBUI_AUTH=True). It was previously left off, with security resting entirely on network-level access control — the hostname has no public DNS record and only resolves on the LAN. That held, but it meant any device on the network could open a web UI that holds external AI provider API keys and conversation history, so a login is now enforced as a second layer rather than relying on the network boundary alone.

Turning it on was not a flag flip. Open WebUI has no environment variable for the admin password — unlike Grafana — and running with WEBUI_AUTH=False had already auto-created an admin@localhost account whose password nobody knew. Deleting that account wasn't an option either: the existing conversation history is bound to its user id, so removing the user would have orphaned it.

The resolution was to write a bcrypt hash directly into the auth table of Open WebUI's own database, keeping the plaintext in the Ansible vault. It works, but it leaves an edge worth recording: updating the vault value alone does nothing. The vault holds the password, the database holds the hash, and nothing reconciles the two — so a rotation means changing both, or the new value is simply fiction. That is the same two-writers problem that shows up wherever a service owns a file Ansible also wants to own.

What that login does not do is protect Ollama or Qdrant. Those are separate services on separate ports, and a login in front of one of their clients is not a control over them — see below.

Keeping the backends off the LAN

Open WebUI's login secures port 3001. Ollama (11434) and Qdrant (6333) were still published to the LAN on their own ports, and neither supports authentication of any kind: no API key was set on Qdrant, and Ollama has no auth mechanism at all. Any device on the network could run models on the GPU, or read and write the entire vector database, without passing through Open WebUI at all.

Framing that as "fixed by enabling WEBUI_AUTH" was the actual mistake — it treated a login on a client as a control over the backends. For a service that cannot authenticate callers itself, the only control that works is removing its network exposure. Both were therefore moved onto a dedicated ai.network with no PublishPort line, and both consumers (Open WebUI and plex-sync) now reach them by container name inside that network:

[Network]
NetworkName=ai

Two details worth recording:

• Removing the port from the firewall list would not have closed it. Podman's PublishPort installs a DNAT rule, and firewalld's filter_INPUT chain accepts DNAT'd connections (ct status dnat accept) before it ever reaches the zone's port rules. Deleting the publish line is what actually closes the port; taking it out of the firewall config would have looked like a fix and changed nothing. This was measured rather than reasoned about — a container published a port that was deliberately absent from the firewall list, and it answered from the other host anyway.
• Closing the ports blinded the monitoring. The health probes for both services ran from the Blackbox Exporter on the monitoring host, in a different network on a different machine, and started firing on services that were perfectly healthy. The fix was a second, minimal Blackbox Exporter running on kontti inside ai.network, scraped by Prometheus as the blackbox_ai job. It publishes only its own probe_* metrics to the LAN, so the probe comes back without either backend going back with it.

That first finding is narrower than "the firewall doesn't work", and the distinction is worth keeping straight. Containers using Network=host bind directly with no DNAT involved, so their ports — Node Exporter's 9100, cAdvisor's 8081 — genuinely are enforced by the declarative port list, as is anything a host process listens on. What the list does not do is gate ports published by a container. So the working rule is: the firewall list is the source of truth for host-network services and a statement of intent everywhere else, but to reason about what is actually exposed, read the quadlets' PublishPort lines. The cost of not knowing that was concrete — Qdrant's unused gRPC port 6334 had been described as "behind the firewall anyway", and was reachable from the LAN the entire time.

SearXNG

SearXNG is a self-hosted meta search engine. It aggregates results from multiple sources without tracking searches or sending data to third parties. It uses Valkey (a Redis-compatible store) for caching.

Qdrant

Qdrant is a vector database. Here it stores embeddings of the Plex media library so the catalogue can be queried in natural language rather than by exact title — see plex-sync below for how the data gets in.

plex-sync

plex-sync is a small custom Python service that turns the Plex library into a searchable vector index. It's the piece that connects three otherwise-separate local services — Ollama, Qdrant, and Plex — into a semantic search pipeline that runs entirely on-prem, with no cloud API calls and no library metadata leaving the network.

How it works

graph LR
    plex[Plex API<br/>movies & shows] -->|descriptive text| sync[plex-sync]
    sync -->|embed| ollama[Ollama<br/>nomic-embed-text]
    ollama -->|768-dim vector| qdrant[(Qdrant<br/>plex_movies / plex_shows)]
    qdrant -->|search| mcp[Media Server MCP]
    mcp -->|tools| owui[Open WebUI]

1. Fetch — the script reads every movie and TV library from the Plex API.
2. Describe — each item is flattened into a compact text block (title, original title, year, genres, countries, ratings, studio, director, top cast, summary). Embedding the description rather than just the title is what makes fuzzy, plot-based queries work.
3. Embed — the text is sent to Ollama's nomic-embed-text model, producing a 768-dimensional vector.
4. Store — the vector and a rich payload (ratings, runtime, genres, view stats…) are upserted into Qdrant, into separate plex_movies and plex_shows collections.

Incremental by design

Re-embedding the whole library every night would be wasteful. Each item's description is hashed (MD5), and that hash is stored alongside the vector. On the next run the script checks Qdrant for an existing point with the same plex_id and the same content hash — if both match, nothing has changed and the item is skipped. Only new or edited titles reach Ollama, so a daily sync over a large library costs a handful of embeddings rather than thousands.

One limitation is worth naming rather than leaving to be discovered: the hash covers the input text, not the model that embedded it. Swapping nomic-embed-text for a different model would leave every hash matching, so nothing would re-embed — and the collection would quietly end up holding vectors produced by two different models, degrading search quality without raising an error anywhere. Folding the model name into the hashed input is the one-line fix. It hasn't been needed yet because the model hasn't changed, which is exactly the condition under which this kind of assumption stays invisible.

Packaging and scheduling

plex-sync builds its own minimal container (python:3.14-slim, the current stable Python release, plus qdrant-client and requests) from a Quadlet .build file, so the image is produced on the host rather than pulled from a registry. A systemd timer runs it once a day at 04:00 with a randomised delay and Persistent=true, so a missed run (for example, the server was down) is caught up on the next boot. The HTTP client uses exponential-backoff retries, since Plex and Ollama can both be briefly busy.

The result feeds the Media Server MCP and Open WebUI: a query like "bleak Nordic crime dramas" resolves against vectors instead of a literal title match.

The two MCP servers

Model Context Protocol servers give an AI assistant typed tools to call instead of an API to be told about. Two of them run here:

• Home Assistant MCP — the official add-on, running on the Home Assistant host. It exposes the smart home as tools: reading sensor states, calling services, triggering automations.
• Media Server MCP — a third-party self-hosted server in a Deno container on port 8085, exposing the Plex library. Its tools search the library, look up metadata and generate recommendations, and the search reaches the Qdrant collections rather than matching titles literally.

Neither is in use

Both were set up to see what the protocol could do, and both have stayed installed since. Nothing else on this page depends on either of them.

That is worth stating plainly, because it is easy to read an architecture diagram as evidence of daily use. The half of this stack that genuinely runs is the embedding pipeline: plex-sync refreshes the Qdrant collections every night through Ollama whether or not anything queries them. The query side — Open WebUI, the MCP tools — is still where the experiment stopped.

My experience

Ollama's main job so far has been generating embeddings for the plex-sync pipeline — feeding Plex metadata into Qdrant via the nomic-embed-text model. I tested larger language models locally, but the GPU doesn't have enough power for a good experience.

I've tested the semantic Plex search through Open WebUI using Gemini models, and it works reasonably well, but it has remained more of an experiment. Open WebUI in general is still at the experimentation stage. I'm waiting for local language models and affordable server hardware to mature before I expect to use them seriously.

The Open WebUI login is a small case of recording a fix against the wrong risk. I'd turned it on and written down that it stopped any device on the network reaching the API keys and the vector database — but it only guards Open WebUI's own port, while Ollama and Qdrant stayed published with no authentication of their own. It came up while I was going through the container hardening with an LLM. On a closed home network I wasn't especially worried, and having the LLM there made it quick to close properly.