Ollama on a headless Linux server — disk fills, API unreachable

/ is full and the API only answers on localhost

作者 Jakub Rusinowski

错误信息

Error: write /usr/share/ollama/.ollama/models/blobs/sha256-...: no space left on device

$ curl http://server.local:11434/api/tags
curl: (7) Failed to connect to server.local port 11434: Connection refused

1. 1

   Find out what is actually filling the disk

   Before moving anything, confirm the model store is the culprit rather than logs or a runaway container, and see what you have accumulated. ollama list almost always reveals a couple of large models pulled once during evaluation and never used since. Removing those is instant and free.

   bash复制

   # bash
   df -h /
   sudo du -sh /usr/share/ollama/.ollama/models
   sudo du -xh / --max-depth=2 2>/dev/null | sort -rh | head -15
   
   ollama list
   ollama rm llama3.1:70b

   生效了吗？The freed space shows up immediately in df -h /, and ollama list no longer includes the model.

   bash复制

   df -h / && ollama list

2. 2

   Move the store to a data volume, and hand it to the ollama user

   Stop the service first — a running Ollama holds file handles inside the store. Move the directory, chown it to ollama:ollama (the step people skip, and the reason the service then cannot write), set OLLAMA_MODELS through a systemd drop-in rather than your shell, and restart.

   注意：Verify the destination is complete before deleting anything from the source. The store is content-addressed, so a partially moved copy looks structurally fine and fails only on the models whose blobs did not arrive.

   bash复制

   # bash
   sudo systemctl stop ollama
   sudo mkdir -p /data/ollama-models
   sudo rsync -a --info=progress2 /usr/share/ollama/.ollama/models/ /data/ollama-models/
   sudo chown -R ollama:ollama /data/ollama-models
   
   sudo systemctl edit ollama.service
   # [Service]
   # Environment="OLLAMA_MODELS=/data/ollama-models"
   
   sudo systemctl daemon-reload && sudo systemctl start ollama
   ollama list

   生效了吗？Every model is still listed and a fresh pull lands on the data volume. Only then remove the old directory.

   bash复制

   ollama list
   ollama pull gemma3:1b
   sudo du -sh /data/ollama-models
   sudo ls /usr/share/ollama/.ollama/models/blobs | wc -l

3. 3

   Handle the hardened-server details: noexec /tmp and proxies

   Two things bite specifically on servers built to a hardening baseline. If /tmp is mounted noexec, Ollama cannot execute from its temporary directory and fails in ways that look like corruption; OLLAMA_TMPDIR points it somewhere it is allowed to. And behind a corporate proxy the variable Ollama reads is HTTPS_PROXY, not HTTP_PROXY — the registry is HTTPS, and the lowercase HTTP form is simply not consulted.

   bash复制

   # bash — is /tmp noexec?
   mount | grep -E ' /tmp '
   
   sudo mkdir -p /var/tmp/ollama && sudo chown ollama:ollama /var/tmp/ollama
   sudo systemctl edit ollama.service
   # [Service]
   # Environment="OLLAMA_TMPDIR=/var/tmp/ollama"
   # Environment="HTTPS_PROXY=http://proxy.internal:3128"
   
   sudo systemctl daemon-reload && sudo systemctl restart ollama

   生效了吗？A pull completes through the proxy, and the log shows no execution-permission errors.

   bash复制

   ollama pull gemma3:1b && journalctl -u ollama --no-pager -n 20

4. 4

   Choose how to expose the API — in this order最常见的修复方法

   OLLAMA_HOST=0.0.0.0:11434 exposes an API with no authentication. Anyone who reaches the port can run inference on your GPU and pull models onto your disk, and an open endpoint on the public internet is worse than it sounds: unauthenticated multi-gigabyte writes, unmetered compute, and no log of who did it. In order of how much we recommend them: (1) an SSH tunnel — nothing is exposed, it works today, and the client still talks to localhost; (2) a reverse proxy with authentication and TLS terminated at the proxy, with Ollama still bound to loopback behind it; (3) binding to a VPN or Tailscale interface address rather than 0.0.0.0, so only the private network can reach it; (4) 0.0.0.0 with a firewall restricted to specific source addresses, on a LAN you administer — and "trusted" means you know every device on it, not that it is your house.

   bash复制

   # bash — (1) SSH tunnel, from the CLIENT machine
   ssh -N -L 11434:127.0.0.1:11434 you@server.example.com
   curl http://localhost:11434/api/tags
   
   # (3) Bind to a Tailscale address only, never 0.0.0.0
   # [Service]
   # Environment="OLLAMA_HOST=100.101.102.103:11434"

   生效了吗？With a tunnel, the client reaches the API on its own localhost and ss on the server still shows Ollama listening only on 127.0.0.1.

   bash复制

   ss -ltnp | grep 11434

   检查哪些模型适配你的硬件size the models on a shared server against the VRAM you actually have打开显存检查器 →
5. 5

   Put a reverse proxy in front rather than opening the port

   For anything more than one person, a reverse proxy is the right shape: Ollama stays on loopback, the proxy holds the certificate and the credentials, and you get an access log — which an open port does not give you. Caddy is two lines. Keep the OLLAMA_HOST default of 127.0.0.1 while doing this: the proxy connects locally, so there is no reason to expose the port at all.

   bash复制

   # Caddyfile — TLS and basic auth in front of a loopback-bound Ollama
   # ollama.example.com {
   #     basic_auth {
   #         alice $2a$14$...   # caddy hash-password
   #     }
   #     reverse_proxy 127.0.0.1:11434
   # }
   
   # bash
   sudo systemctl reload caddy
   curl -u alice:secret https://ollama.example.com/api/tags

   生效了吗？The request succeeds with credentials and is rejected without them, and Ollama is still bound to 127.0.0.1.

   bash复制

   curl -s -o /dev/null -w "%{http_code}\n" https://ollama.example.com/api/tags
   ss -ltnp | grep 11434

6. 6

   Tune keep-alive, parallelism and loaded models for a shared box

   Three defaults are laptop defaults. OLLAMA_KEEP_ALIVE is 5m, so the first request after a quiet spell pays the full load time — raising it keeps a model warm, at the cost of holding its VRAM the whole time, which is a real trade on a shared card. OLLAMA_NUM_PARALLEL is 1 request per model, so concurrent users queue. OLLAMA_MAX_LOADED_MODELS is 3× your GPU count (3 on CPU), and every simultaneously loaded model holds its own weights and KV cache. Raise these against measured free VRAM, not optimism: each parallel slot needs its own KV cache, so doubling parallelism can cost more memory than loading a second model.

   bash复制

   # bash
   sudo systemctl edit ollama.service
   # [Service]
   # Environment="OLLAMA_KEEP_ALIVE=30m"
   # Environment="OLLAMA_NUM_PARALLEL=4"
   # Environment="OLLAMA_MAX_LOADED_MODELS=2"
   
   sudo systemctl daemon-reload && sudo systemctl restart ollama
   journalctl -u ollama -f

   生效了吗？Under load, ollama ps still shows GPU placement rather than a split, and VRAM has headroom.

   bash复制

   ollama ps
   nvidia-smi --query-gpu=memory.used,memory.total --format=csv