Run a Local Coding Agent End-to-End: Ollama + Qwen3-Coder + VS Code

Written by Jakub Rusinowski · Last updated 2026-07-12 · Hardware figures computed by our VRAM engine

A complete local coding agent is three pieces: Ollama serving a coding model (Qwen3-Coder 8B on 8 GB VRAM, Qwen 3.6 27B on 24 GB), an editor extension that speaks to it (Continue for chat + autocomplete, Cline for autonomous multi-file tasks), and a small autocomplete model running alongside. Setup takes 30–45 minutes including downloads; nothing leaves your machine.

This guide wires up the whole stack, end to end, on Windows, macOS, or Linux. When you're done, VS Code will have Copilot-style ghost-text completions, highlight-and-ask chat, and an agent that can plan and execute multi-file changes — all served from your own GPU, working on an airplane, and costing $0/month.

If you haven't picked a model yet, skim the best local coding models by VRAM tier first, or just take the defaults below — they're chosen per hardware tier and you can swap models later with one config line.

1. Confirm what your hardware can run

Check your GPU or Mac against the compatibility checker — it uses the same VRAM math as this guide. The tiers this guide targets: 8 GB VRAM → Qwen3-Coder 8B; 16 GB → Devstral-2 22B; 24 GB → Qwen 3.6 27B or Qwen 2.5 Coder 32B; Mac 96 GB+ / 2×24 GB → Qwen3-Coder 80B-A3B. Add StarCoder2 3B for autocomplete on any tier with ~3 GB spare.

2. Install Ollama

Download from ollama.com (macOS/Windows installers, or the Linux one-liner) — full walkthrough in our Ollama install guide. Verify it's serving:

ollama --version
curl http://localhost:11434

The second command should answer "Ollama is running". That local HTTP endpoint (port 11434) is what your editor extensions will talk to.

3. Pull a chat/agent model for your tier

Pull the model matching your hardware tier from step 1:

# 8 GB VRAM
ollama pull qwen3-coder:8b
# 16 GB VRAM
ollama pull devstral:22b
# 24 GB VRAM
ollama pull qwen3.6:27b
# Mac 96GB+ unified / 2x24 GB
ollama pull qwen3-coder:80b-a3b-q4

Test it before touching the editor: ollama run qwen3-coder:8b "write a python function that merges two sorted lists". If tokens stream at a comfortable reading pace or faster, you're good.

4. Pull a small autocomplete model

Inline completion is a latency game, not a quality game — a 3B model that answers in 150 ms beats a 27B model that answers in two seconds. Run a dedicated fill-in-the-middle model alongside your chat model:

ollama pull starcoder2:3b

Skip this only if your GPU is already at its VRAM limit — Continue will fall back to using your chat model for completions (slower, still works).

5. Install Continue in VS Code and point it at Ollama

Install Continue from the VS Code marketplace (also available for JetBrains). Open its config (gear icon → config.yaml) and register both models:

models:
  - name: Qwen3-Coder (local)
    provider: ollama
    model: qwen3-coder:8b        # swap for your tier's model
    roles: [chat, edit, apply]
  - name: StarCoder2 autocomplete
    provider: ollama
    model: starcoder2:3b
    roles: [autocomplete]

That's the whole integration: ghost-text completions as you type, Cmd/Ctrl+L to chat about selected code, Cmd/Ctrl+I for inline edits.

6. Add Cline for autonomous multi-file tasks (optional)

Continue is assist-style; Cline is an agent — give it a task and it plans, edits across files, runs commands, and shows you diffs to approve. Install Cline from the marketplace, set the API provider to Ollama, and select your chat model. One honest caveat: agentic loops burn context fast and lean hard on model capability. On 8 GB hardware keep tasks small and single-file; the bigger your tier's model, the more autonomy actually works. See Continue vs Cline vs Aider for which tool fits which workflow.

7. Raise the context window

Ollama defaults to a small context window, and coding agents eat context (system prompt + file contents + diffs). Raise it per-model:

OLLAMA_CONTEXT_LENGTH=16384 ollama serve

or set num_ctx in a Modelfile. 16K is a good floor for agent work; go higher only if your VRAM allows — KV cache grows with context, which is why the compatibility checker lets you model context length explicitly.

8. Verify the loop and set expectations

Real test: open a project, ask Continue to explain a function, accept a couple of completions, then give Cline a small task ("add input validation to this endpoint and update its test"). If completions feel sluggish, drop to a smaller autocomplete model or lower quant. If agent edits go off the rails, that's usually the model being too small for the task, not the tooling — check the model tier list for what your hardware realistically supports, or rent a bigger GPU for an afternoon via the cloud directory to feel the difference before upgrading.

Frequently asked questions

Do I need a GPU to run a local coding assistant?
No, but it changes the experience. CPU-only inference runs Qwen3-Coder 8B at a few tokens/sec — usable for chat, too slow for autocomplete. Any 8 GB GPU (or an Apple Silicon Mac with 16 GB unified memory) makes the full stack — completions plus chat — feel instant.
Which VS Code extension is best for local models: Continue or Cline?
They solve different problems and run happily together. Continue replaces Copilot (autocomplete + chat + inline edits) and is the one to install first. Cline is an autonomous agent for multi-file tasks — better with bigger models, overkill for simple completion. Aider covers the same agent ground from the terminal.
Can I use LM Studio instead of Ollama?
Yes — LM Studio exposes the same style of OpenAI-compatible local server (default port 1234), and both Continue and Cline support it as a provider. Ollama is more convenient as an always-on background service; LM Studio is friendlier for browsing and testing models. The models themselves are identical GGUF files.
How much does a local coding agent cost to run?
After hardware, effectively just electricity — a 24 GB GPU coding all day costs a few dollars per month in power, versus $10/month for Copilot Pro or $20/month for Cursor Pro forever. If you already own the GPU, the payback is immediate; if not, see our cost comparison for the break-even math.
Does any of my code leave my machine with this setup?
No. Ollama serves on localhost, and with Continue/Cline pointed at it, prompts, code context, and completions all stay on your hardware. That is the core argument for local coding agents in privacy-sensitive codebases — client work under NDA, unreleased products, regulated industries.

Keep going