Terminal Coding Agents on Local Models: OpenCode, Qwen Code, Aider & Friends

Written by Jakub Rusinowski · Last updated 2026-08-04 · Hardware figures computed by our VRAM engine

The terminal is where open-source coding agents caught up fastest. OpenCode, Qwen Code and Aider are all model-agnostic and all speak the OpenAI-compatible API that Ollama, LM Studio, llama.cpp and vLLM expose — so pointing them at your own GPU is a base-URL and a model name. What differs is harness weight: Aider is the lightest and survives a 16K window; the newer agents do more and want 24 GB and up.

Two years ago the good coding agents were closed products tied to one provider. In 2026 the strongest ones in the terminal are open source and provider-agnostic by design — you configure an endpoint, and the agent does not care whether it is a frontier API or llama-server on the box under your desk. OpenCode in particular grew into the de-facto open answer to the closed terminal agents, MIT-licensed and able to reach a long list of providers including local ones.

For local-first developers this is the important shift. You are no longer choosing between "good agent, someone else's model" and "your model, mediocre tooling." You can run a well-engineered harness on weights you own. The remaining constraint is the one this whole hub keeps returning to: the heavier the harness, the more window it needs before it starts working.

The four that matter locally

The pattern is consistent: the agents that do the most assume the most context. On 16 GB, Aider's compact repo map and search/replace edit formats mean far more of the window is your code and far less is scaffolding. On 24 GB and up, the fuller agents pay off — their planning, todo tracking and multi-file editing genuinely reduce the number of turns.

Pointing an agent at your own GPU

The mechanics are the same everywhere: an OpenAI-compatible base URL, a model name, and a dummy API key for clients that insist on one.

# 1. Serve a model with a context window that fits an agent's prompt
cat > Modelfile <<'EOF'
FROM qwen3.6:27b
PARAMETER num_ctx 32768
EOF
ollama create qwen3.6:27b-32k -f Modelfile
ollama serve            # OpenAI-compatible at http://localhost:11434/v1

# 2. The environment most CLI agents read
export OPENAI_BASE_URL=http://localhost:11434/v1
export OPENAI_API_KEY=local            # ignored, but often required to be set

# 3. Aider, explicitly
aider --model ollama_chat/qwen3.6:27b-32k

# llama.cpp instead of Ollama — same interface, finer control
llama-server -m qwen3-coder-8b-q4_k_m.gguf --port 8080 --ctx-size 16384 \
  --jinja        # use the GGUF's chat template, needed for tool calls

Three things to get right, in order of how often they are wrong:

1. Context window. The default is too small for any agent's system prompt plus a file. This is the single most common cause of "the agent ignores instructions" — see context engineering. 2. Tool-call support. If the agent uses function calling, the served model needs a chat template that supports it (--jinja on llama-server; an agentic tune rather than a base chat model). Otherwise pick an agent that uses diff formats instead (tool calling). 3. VRAM headroom for the cache. Weights plus 32K of KV cache, not weights alone. Check it with the calculator before you discover it mid-task.

Which model to serve a terminal agent

Q4_K_M weights from the site’s compute engine — add KV cache for the context you configured. Agent prompts are large; do not size on weights alone.

Terminal or editor?

Not a rivalry — most people end up with both, and the split follows the task.

Terminal agents own work that is naturally command-shaped: repo-wide refactors, migrations, "make the suite green," anything you want to run in a worktree and review as a diff. They compose with everything else you have — pipe output in, run them over SSH, wrap them in a shell loop. That last property is what makes them the natural host for loop engineering.

Editor agents own work where seeing the file matters: exploratory changes, UI work, anything where you want inline diffs and the language server's opinion beside you.

Autocomplete is a separate job with a separate model — a small FIM-tuned model answering in milliseconds, running alongside whichever agent you use. Full breakdown in the tool matrix.

Honest limits on local hardware

What you should expect, so you configure around it rather than concluding the tooling is broken:

• Long autonomous runs are wall-clock expensive. A 27B at 30–60 tokens/sec re-reading a growing context is not a frontier API. Scope tasks tighter and cap iterations; the answer is fewer, better turns.
• Some agents assume a large window. An agent designed against a 200K context may compact late and plan expansively. Where it is configurable, compact earlier; where it is not, prefer the lighter agent.
• Model swapping is not free locally. Switching models means loading weights — seconds to a minute. The per-task model switching that costs nothing on a hosted provider is a real pause on your own GPU; keep one model resident for a session.
• The hardest 5% still favours frontier models. The hybrid pattern is legitimate: local by default, and one API call for the task that keeps derailing. The cost comparison does that math; privacy-critical repos simply take the smaller model and the tighter scope.

No hardware? Rent the GPU first

Want to know whether a heavier terminal agent is worth a hardware upgrade? Rent a 48 GB GPU for an evening, serve the bigger model, and run your own repo through it before spending anything.

• RunPod — RTX 4090 ≈ $0.3–0.7/hr, A100/H100 by the hour; serverless per-second billing available
• Vast.ai — Marketplace of hosted GPUs — often the cheapest per hour (interruptible options)
• Lambda — On-demand A100/H100/B200 instances and clusters, per-hour billing

Full list on the cloud AI directory.

Frequently asked questions

Keep going

• Local OpenAI-compatible API servers
• Installing Ollama
• Report #2 — the open coding-model landscape
• Aider with Local Models: Setup and the Models That Actually Work
• Cline with Local Models: What Actually Works (and What Doesn’t)
• The Agent Harness: What Actually Turns a Local Model Into a Coding Agent
• Continue vs Cline vs Aider vs Tabby: the Local Coding Tool Matrix
• Local AI coding assistant — the full hub