Written by Jakub Rusinowski · Last updated July 10, 2026
Founder, LLM Configurator — AI educator & workshop leader on local LLM deployment
<|im_start|>assistant<|im_end|> <|im_start|>user I I I the the of of<|endoftext|>assistantassistantThe model loads fine, VRAM usage looks normal — but the output is wrong: leaked template tags like <|im_start|> or [INST] in the reply, answers that ignore your question, the model talking to itself in fake user/assistant turns, refusing to stop generating, or fluent-looking text degenerating into word salad. Most common right after downloading a community GGUF from Hugging Face, importing a model into a hand-written Modelfile, or converting your own fine-tune.
Chat models don't see your message alone — the runtime wraps it in the model's chat template, the exact scaffolding of special tokens it was trained on. Llama 3 expects <|start_header_id|> headers, Qwen/ChatML models expect <|im_start|> markers, Gemma expects <start_of_turn>, and older Alpaca-style tunes expect "### Instruction:" blocks. Feed a model the wrong scaffolding and it's reading a foreign language: it leaks tags, misses its stop token, and produces nonsense. The weights are fine; the wrapper is wrong. (True VRAM problems announce themselves differently — crashes and OOM errors, not fluent gibberish. Genuinely corrupted downloads are the rarer look-alike, covered below.)
Official Ollama library models (ollama pull llama3.1, qwen2.5, gemma3, …) ship with the correct template and stop tokens baked in — template gibberish essentially only happens with manually imported GGUFs and hand-written Modelfiles. If a hand-wired model misbehaves and the same model exists in the library, pulling the official version is the 30-second fix. Keep the manual route for models that genuinely aren't in the library (like your own fine-tunes).
ollama rm my-broken-import
ollama pull qwen2.5:7b # template + stop tokens includedConfirm the mismatch instead of guessing. Every model family's correct template is visible on any properly configured model of that family, and on the model's Hugging Face card (tokenizer_config.json → chat_template). Thirty seconds of comparison tells you exactly which scaffolding your Modelfile should carry.
# What template is my model actually using?
ollama show my-model --template
# What SHOULD a model of this family use?
ollama show llama3.1 --template
ollama show qwen2.5 --templateWhen you must hand-write it (imported GGUF, converted fine-tune), copy the template from a same-family library model verbatim — never type it from memory; one missing newline or token matters. The stop parameter must match the family too: <|eot_id|> for Llama 3, <|im_end|> for ChatML/Qwen, <end_of_turn> for Gemma. Then recreate and retest.
# Modelfile for a ChatML/Qwen-family GGUF
FROM ./my-model.gguf
TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
PARAMETER stop <|im_end|>
# apply:
ollama create my-model-fixed -f ModelfileLM Studio reads the template embedded in GGUF metadata, and modern GGUFs carry the right one — but older or hand-converted files carry none, and LM Studio falls back to a guess. If output shows leaked tags, open the model's settings (My Models → gear icon → Prompt tab) and pick the correct format (ChatML for Qwen-family, Llama 3 for Llama-family, Gemma for Gemma) instead of the wrong autodetect. The moment tags stop appearing in output, you've got the right one.
A converted fine-tune must be served with the template it was trained on — which follows the base model family. This is the single most common reason a fine-tune that evaluated well 'comes out broken' in Ollama: the behavior is in there, the wrapper is hiding it. The GGUF conversion guide's Modelfile section walks the whole flow.
If the scaffolding checks out and output is still word salad, you're in the rarer failure class: a corrupted or truncated download, a broken experimental quant, or a GGUF too new/old for your runtime. Compare the file's size against the source repo listing (a partial download is smaller), re-download, prefer Q4_K_M-or-higher quants from reputable uploaders, and update Ollama/LM Studio — brand-new architectures need current runtime versions.
# Size sanity check vs the Hugging Face repo listing
ls -lh my-model.gguf
# Update the runtime (new model support lands fast)
ollama --versionBecause nothing is broken mechanically — the model is answering a malformed conversation. A wrong chat template feeds the model scaffolding tokens it wasn't trained on, so it produces leaked tags, ignored questions, and nonsense. Memory problems crash with errors; template problems generate fluent garbage.
Special chat-template tokens leaking into text. <|im_start|>/<|im_end|> belong to ChatML-family models (Qwen and many fine-tunes); <|eot_id|> and <|start_header_id|> belong to Llama 3. Seeing them in a reply is the definitive symptom of a template/stop-token mismatch.
Two authoritative sources: ollama show <same-family-library-model> --template for a known-good copy, or the chat_template field in tokenizer_config.json on the model's Hugging Face repo. Copy exactly — whitespace and newlines are part of the format.
Sometimes — that's the second suspect. A truncated/corrupted GGUF or an extreme low-bit quant also produces nonsense. Distinguish them: template problems show structured symptoms (leaked tags, fake turns, no stopping), file problems show unstructured word salad from the first token. Verify file size against the source and re-download to rule it out.
Each runtime resolves the template independently: LM Studio read the GGUF's embedded template while your hand-written Ollama Modelfile overrode it with the wrong one (or vice versa). The weights are provably fine — align both tools on the same, correct template and the difference disappears.