Loop Engineering on a Local LLM: Agent Loops That Check Their Own Work

Three practices have stacked up in three years. Prompt engineering was about the wording of one request. Context engineering was about what the model can see when it answers. Loop engineering — the term went mainstream in mid-2026 — is about what happens after the answer: the model acts, something objective observes the result, and the system decides whether to accept it, retry it, or stop. It's the layer underneath every serious coding agent shipped this year, and it's the layer where local models gain the most ground.

The reason is unglamorous. A frontier model can often carry a vague instruction to a decent result on the first try, so a sloppy loop is survivable. A 22–32B model running on your own GPU frequently cannot — but it is very good at making a failing test pass when you hand it the failure output. Loop engineering converts the model's weakness (judgement, long-horizon planning) into something the loop supplies for free, and leans on its strength (fast, local, unmetered iterations). A well-designed loop on a 24 GB card routinely beats a poorly-designed loop on an API model that costs money per attempt.

Every coding agent — Aider, Cline, OpenCode, Claude Code, Codex — is the same five-step cycle underneath. The differences are in what each step is allowed to do.

Step 4 is what separates loop engineering from "running the agent again." A verifier has one job: return pass or fail identically every time, with no model in the decision. pytest -q, tsc --noEmit, cargo test, go build ./..., ruff check, npm run lint — all valid. "Ask a second model whether the code looks right" is not, at least not as the gate; LLM-as-judge belongs outside the objective check, never in place of it.

This is why the first question of any local loop project is not "which model?" but "what is my pass/fail command, and how long does it take to run?" A repo without a fast, trustworthy test command cannot be looped on. Fixing that comes first.

Loops nest. Each level wraps the one beneath it and adds a different kind of correction. Knowing which level you are operating at tells you what your next improvement should be.

Most people who say "the agent doesn't work" are stuck at level 1 with a model that isn't strong enough to self-assess. Moving to level 2 — same model, same hardware, one added pytest gate — is usually the largest single quality jump available, and it costs nothing.

The concept is provider-neutral; the engineering constraints are not. Five differences drive every design decision on your own hardware.

• Iterations are free but not fast. No per-token bill, so a 12-iteration loop is fine financially. But a 27B at Q4 on a 24 GB card produces perhaps 30–60 tokens/sec, and each iteration re-reads a growing context. Loops are wall-clock-bound, not budget-bound — which flips the usual advice: prefer fewer, better-informed iterations over brute force.
• Context is a hardware cost. Every retry appends failure output. On an API the window is someone else's problem; locally, KV cache is VRAM you have to own. A 32K context on a 27B adds several GB on top of the weights — model the exact number with the VRAM checker before you design a loop that assumes 64K.
• Tool-call reliability is the failure mode. Frontier models emit well-formed tool calls essentially always. Local models drift — wrong argument names, prose wrapped around JSON, a second call hallucinated after the first. A loop that can't parse an action can't act. See tool calling with local models for the grammar-constrained fix.
• Degradation is steeper. Local models lose coherence faster as context fills. The practical consequence: reset context between attempts rather than accumulating ten failed attempts in one window. Carry forward the goal, the current diff, and the latest failure — not the transcript.
• You own the whole stack. Nothing rate-limits you, nothing deprecates your model, and no vendor sees the repo. For a loop that runs unattended over a private codebase, that is the entire argument.

Before adopting a framework, build the twenty-line version — it makes the moving parts obvious and it is genuinely useful. This drives any OpenAI-compatible local endpoint (Ollama on :11434, LM Studio on :1234, llama-server, vLLM) and gates on your real test command.

# loop.py — goal loop (level 2) against a local endpoint
import subprocess, json, urllib.request

ENDPOINT = "http://localhost:11434/v1/chat/completions"
MODEL    = "devstral-small-2:24b"    # whatever your VRAM supports
VERIFY   = ["pytest", "-q"]          # THE gate. Must be deterministic.
MAX_ITERS = 6                        # always cap. always.

def verify():
    p = subprocess.run(VERIFY, capture_output=True, text=True)
    return p.returncode == 0, (p.stdout + p.stderr)[-4000:]   # tail only

def ask(messages):
    body = json.dumps({"model": MODEL, "messages": messages,
                       "temperature": 0.1, "stream": False}).encode()
    req = urllib.request.Request(ENDPOINT, body, {"Content-Type": "application/json"})
    with urllib.request.urlopen(req) as r:
        return json.load(r)["choices"][0]["message"]["content"]

goal = "Make the failing tests in tests/ pass. Change only src/."
for i in range(1, MAX_ITERS + 1):
    ok, output = verify()
    if ok:
        print(f"PASS after {i-1} iteration(s)"); break
    # Fresh context every attempt: goal + current failure only.
    reply = ask([
        {"role": "system", "content": "You are editing a repo. Reply ONLY with a unified diff."},
        {"role": "user", "content": f"{goal}\n\nTest output:\n{output}"},
    ])
    subprocess.run(["git", "apply", "-"], input=reply, text=True)
    print(f"--- iteration {i} applied ---")
else:
    print("Cap reached without a green suite — reverting.")
    subprocess.run(["git", "checkout", "--", "."])

Four design decisions in that file are the whole lesson: the verifier is a subprocess, not a prompt; the failure output is truncated to its tail so context doesn't explode; context is rebuilt each iteration instead of accumulated; and the loop reverts on cap rather than leaving half-finished edits behind. Run it inside a git worktree or a container so a bad diff costs you nothing — see sandboxing a local coding agent.

A loop inherits the quality of its check exactly. Grade yours against these, in order of value per minute spent:

1. Speed before coverage. A 4-second subset that runs every iteration is worth more than a 6-minute full suite that runs once. Point the loop at the fast subset; run the full suite once at the end. On local hardware the test run often costs less wall-clock than one model turn — that asymmetry is the reason to gate aggressively. 2. Fail loudly and specifically. The failure text is the next prompt. Assertion messages that say what was expected versus received teach the model in one turn; AssertionError alone burns three. 3. Stack cheap gates first. Type check (tsc --noEmit, mypy) and lint catch a large share of local-model errors — wrong import paths, invented method names, argument-count slips — in under a second, before a single test runs. 4. Close the reward-hacking holes. A model told "make tests pass" will, given the chance, edit the test, delete the assertion, or add @pytest.mark.skip. Constrain the writable path (Change only src/), and add a gate that fails if the test files changed: git diff --exit-code -- tests/. This is not paranoia; it is the single most common way loop output looks green and is worthless. 5. Know what the check does not cover. Green tests mean "no known regression," not "correct." Every loop needs a human diff review at the end. The loop's job is to hand you a small, plausible, already-passing diff — not to merge it.

Two habits prevent most of these: cap everything (iterations, per-iteration wall clock, tokens per turn), and log every iteration to disk — prompt, tool call, verifier output. The log is how you tell "the model is too small" apart from "my prompt was ambiguous," and they demand opposite responses.

A good loop lowers the bar, but it doesn't erase it. The floor is set by two abilities: emitting a valid tool call or diff, and correctly interpreting a stack trace.

• 8B (8 GB GPU) — viable at level 2 for narrow, single-file goals with a crisp failing test. Expect to babysit; keep the cap at 3–4. Best paired with strict diff formats and constrained decoding.
• 16 GB tier — the first tier where an unattended loop is reasonable. Devstral Small 2 24B is explicitly agent-tuned and follows edit formats well.
• 24 GB tier — Qwen 3.6 27B or Qwen 2.5 Coder 32B: multi-file goals, tolerable failure-interpretation, loops that finish while you make coffee. This is the tier most people should target.
• Workstation / 96 GB+ Mac — Qwen3-Coder 80B-A3B. Fast MoE decode makes long loops practical, which matters more than raw quality once a verifier is in place.

Per-tier reasoning and exact VRAM figures live in best local coding models; the honest capability ladder for autonomy is in VRAM for coding models.

Not every task deserves a loop. The ones that do share a shape: a machine-checkable definition of done, and enough repetition to justify writing the check.

• Test-driven repair — the canonical case. Failing test in, green diff out.
• Mechanical migrations — framework version bumps, API renames, type annotations across hundreds of files. The verifier is the build; the model does the tedium.
• Lint and type debt — point the loop at tsc --noEmit and let it grind the error count down overnight on hardware you already own.
• Test generation with a coverage gate — generate tests, gate on coverage delta and on the suite still passing.
• Doc/code drift — regenerate docs from source, verify by re-running the doc examples.

Tasks that resist loops: anything whose success is a matter of taste (API design, naming), anything where the check costs more to write than doing the work by hand, and anything touching a system you cannot safely re-run. For those, stay in the interactive assistant mode covered in the tool matrix.