OpenHands with a local model: setup for Ollama or LM Studio

OpenHands is an open-source (MIT) coding agent with a web GUI that runs in Docker, plus a CLI. Its local-model guide connects it to LM Studio, Ollama, vLLM or SGLang through an OpenAI-compatible base URL. It suits users with at least a 24 GB GPU or a 64 GB Apple Silicon Mac, which is what the docs ask for to run their recommended Qwen3.6-35B-A3B.

Source code Official docs Compare all agents

At a glance

Type: web-ui, cli
Licence: MIT
Latest release: 1.11.0 (2026-07-09)
Status: Maintained
Local backends: ollama, lmstudio, vllm, sglang, openai-compatible
Needs native tool calling: Not verified
MCP support: Not verified
Recommended minimum context: 22,000 tokens
Start-up prompt: Not measured
GitHub stars: Not verified

How to set it up with a local model

Pick your backend. Each command has its own Copy button. Where a step uses an example model, swap in the one you run; the table below lists the ones that fit your machine.

Install it (Web GUI (Docker, as in the local-LLM guide)):

docker run -it --rm --pull=always \
    -e AGENT_SERVER_IMAGE_REPOSITORY=ghcr.io/openhands/agent-server \
    -e AGENT_SERVER_IMAGE_TAG=1.26.0-python \
    -e LOG_ALL_EVENTS=true \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v ~/.openhands:/.openhands \
    -p 3000:3000 \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.openhands.dev/openhands/openhands:1.8

Start Ollama with at least 22K context (32K here), listening on all interfaces so the Docker container can reach it, then pull the model (replace qwen3.6:35b-a3b if you want another).
```
OLLAMA_CONTEXT_LENGTH=32768 OLLAMA_HOST=0.0.0.0:11434 OLLAMA_KEEP_ALIVE=-1 nohup ollama serve &
ollama pull qwen3.6:35b-a3b
```
Start the OpenHands web GUI with the Docker command above and open http://localhost:3000.
In Settings, open the LLM tab, click "see advanced settings" and turn on the Advanced toggle.
Set the custom model (openai/ plus the Ollama tag), the base URL and any placeholder API key, then Save Settings.
```
Custom Model: openai/qwen3.6:35b-a3b
Base URL:     http://host.docker.internal:11434/v1
API Key:      local-llm
```

Other ways to install

CLI (uv)

uv tool install openhands --python 3.12

CLI (binary)

curl -fsSL https://install.openhands.dev/install.sh | sh

Check that it works

Check that the OpenHands container can reach the model server (use port 11434 for Ollama), then start a conversation and ask it to create a file.

docker exec -it openhands-app curl -s http://host.docker.internal:1234/v1/models

Which local models fit, by machine

Checked by this site's engine at the context OpenHands needs, with 32 GB of system RAM for MoE offload. A Mac counts three quarters of unified memory as usable.

Machine	Model	Quant	Context	Fit	Ollama
12 GB GPU	Qwen 3.5 9B	Q4_K_M	32,768	Comfortable (7.4 GB of 12 GB)	`qwen3.5:9b`
	Qwen 3.6 35B-A3B	Q4_K_M	32,768	With CPU offload (MoE) (22.7 GB of 12 GB)	`qwen3.6:35b-a3b`
16 GB GPU	Qwen 3.5 9B	Q4_K_M	32,768	Comfortable (7.4 GB of 16 GB)	`qwen3.5:9b`
	Qwen 3.6 35B-A3B	Q4_K_M	32,768	With CPU offload (MoE) (22.7 GB of 16 GB)	`qwen3.6:35b-a3b`
24 GB GPU	Qwen3.8 27B	Q4_K_M	32,768	Comfortable (19.9 GB of 24 GB)	`qwen3.8:27b`
	Qwen 3.6 27B	Q4_K_M	32,768	Comfortable (19.9 GB of 24 GB)	`qwen3.6:27b`
	Devstral Small 2 24B	Q4_K_M	32,768	Comfortable (20.7 GB of 24 GB)	`devstral-small-2:24b`
	Qwen 3.6 35B-A3B	Q4_K_M	65,536	Tight (23.3 GB of 24 GB)	`qwen3.6:35b-a3b`
32 GB GPU	Qwen3.8 27B	Q4_K_M	32,768	Comfortable (19.9 GB of 32 GB)	`qwen3.8:27b`
	Qwen 3.6 27B	Q4_K_M	32,768	Comfortable (19.9 GB of 32 GB)	`qwen3.6:27b`
	Devstral Small 2 24B	Q4_K_M	32,768	Comfortable (20.7 GB of 32 GB)	`devstral-small-2:24b`
	Qwen 3.6 35B-A3B	Q4_K_M	65,536	Comfortable (23.3 GB of 32 GB)	`qwen3.6:35b-a3b`
	Qwen3-Coder 30B-A3B (MoE)	Q4_K_M	65,536	Comfortable (25.7 GB of 32 GB)	`qwen3-coder:30b`
Mac, 64 GB	Qwen 3.6 35B-A3B	Q8_0	131,072	Comfortable (40.7 GB of 48 GB)	—

Your exact machine: filter the matrix or check any model in the analyzer.

Common problems with local models

OpenHands needs at least 22,000 tokens of context (32,768 recommended). Ollama's small default context cannot even fit the system prompt, and the agent misbehaves.
From inside Docker, localhost is the container: use host.docker.internal and keep --add-host host.docker.internal:host-gateway on the docker run line.
On Linux, LM Studio binds to 127.0.0.1 by default and Docker cannot reach it until "Serve on Local Network" is enabled.
If the agent acts like a plain chatbot or keeps failing tool calls, the docs say the model is the usual cause; try a stronger tool-calling model.

Other coding agents

How they rank for local models: the best agent harness for local LLMs.

Sources (4), checked 2026-10-20