Terminal Coding Agents on Local Models: OpenCode, Qwen Code, Aider & Friends

Two years ago the good coding agents were closed products tied to one provider. In 2026 the strongest ones in the terminal are open source and provider-agnostic by design — you configure an endpoint, and the agent does not care whether it is a frontier API or llama-server on the box under your desk. OpenCode in particular grew into the de-facto open answer to the closed terminal agents, MIT-licensed and able to reach a long list of providers including local ones.

For local-first developers this is the important shift. You are no longer choosing between "good agent, someone else's model" and "your model, mediocre tooling." You can run a well-engineered harness on weights you own. The remaining constraint is the one this whole hub keeps returning to: the heavier the harness, the more window it needs before it starts working.

The pattern is consistent: the agents that do the most assume the most context. On 16 GB, Aider's compact repo map and search/replace edit formats mean far more of the window is your code and far less is scaffolding. On 24 GB and up, the fuller agents pay off — their planning, todo tracking and multi-file editing genuinely reduce the number of turns.

The mechanics are the same everywhere: an OpenAI-compatible base URL, a model name, and a dummy API key for clients that insist on one.

# 1. Serve a model with a context window that fits an agent's prompt
cat > Modelfile <<'EOF'
FROM qwen3.6:27b
PARAMETER num_ctx 32768
EOF
ollama create qwen3.6:27b-32k -f Modelfile
ollama serve            # OpenAI-compatible at http://localhost:11434/v1

# 2. The environment most CLI agents read
export OPENAI_BASE_URL=http://localhost:11434/v1
export OPENAI_API_KEY=local            # ignored, but often required to be set

# 3. Aider, explicitly
aider --model ollama_chat/qwen3.6:27b-32k

# llama.cpp instead of Ollama — same interface, finer control
llama-server -m qwen3-coder-8b-q4_k_m.gguf --port 8080 --ctx-size 16384 \
  --jinja        # use the GGUF's chat template, needed for tool calls

Three things to get right, in order of how often they are wrong:

1. Context window. The default is too small for any agent's system prompt plus a file. This is the single most common cause of "the agent ignores instructions" — see context engineering. 2. Tool-call support. If the agent uses function calling, the served model needs a chat template that supports it (--jinja on llama-server; an agentic tune rather than a base chat model). Otherwise pick an agent that uses diff formats instead (tool calling). 3. VRAM headroom for the cache. Weights plus 32K of KV cache, not weights alone. Check it with the calculator before you discover it mid-task.

Not a rivalry — most people end up with both, and the split follows the task.

Terminal agents own work that is naturally command-shaped: repo-wide refactors, migrations, "make the suite green," anything you want to run in a worktree and review as a diff. They compose with everything else you have — pipe output in, run them over SSH, wrap them in a shell loop. That last property is what makes them the natural host for loop engineering. Editor agents own work where seeing the file matters: exploratory changes, UI work, anything where you want inline diffs and the language server's opinion beside you. Autocomplete is a separate job with a separate model — a small FIM-tuned model answering in milliseconds, running alongside whichever agent you use. Full breakdown in the tool matrix.

What you should expect, so you configure around it rather than concluding the tooling is broken:

• Long autonomous runs are wall-clock expensive. A 27B at 30–60 tokens/sec re-reading a growing context is not a frontier API. Scope tasks tighter and cap iterations; the answer is fewer, better turns.
• Some agents assume a large window. An agent designed against a 200K context may compact late and plan expansively. Where it is configurable, compact earlier; where it is not, prefer the lighter agent.
• Model swapping is not free locally. Switching models means loading weights — seconds to a minute. The per-task model switching that costs nothing on a hosted provider is a real pause on your own GPU; keep one model resident for a session.
• The hardest 5% still favours frontier models. The hybrid pattern is legitimate: local by default, and one API call for the task that keeps derailing. The cost comparison does that math; privacy-critical repos simply take the smaller model and the tighter scope.