Written by Jakub Rusinowski · Last updated July 10, 2026
A system prompt is a standing instruction your local LLM reads before every conversation — and it is the single cheapest upgrade you can make. The same 8B model that gives generic, hedge-filled answer
A system prompt is a standing instruction your local LLM reads before every conversation — and it is the single cheapest upgrade you can make. The same 8B model that gives generic, hedge-filled answers by default becomes sharp and consistent once you tell it who it is, how to format output, and what to refuse to invent.
Last Updated: July 2026
Cloud chatbots ship with thousands of words of hidden system prompt tuning their tone. The model you just pulled with ollama run has almost none — you're talking to the raw instruction-tuned model. That's why fresh local models tend to:
None of that is a model-quality problem. It's a missing-instructions problem, and how to write a system prompt is the fix.
Prompt: "Should I use PostgreSQL or SQLite for my side project?"
Default Llama 3.1 8B (no system prompt):
> Great question! The choice between PostgreSQL and SQLite depends on many factors, including your use case, scale requirements, and personal preference. PostgreSQL is a powerful, full-featured database… *(300 more words of both-sides filler)*
Same model, with this system prompt: *"You are a senior engineer. Give a direct recommendation first, then at most 3 bullet points of reasoning. State assumptions instead of hedging."*
> Use SQLite. — It's a single file, zero setup, and handles side-project traffic easily. — Switch to Postgres only when you need concurrent writers or a hosted deployment. — Assumption: this runs on one machine; if it's multi-user from day one, start with Postgres.
Same weights, same VRAM, same speed. The entire difference is 30 words of instruction.
You have three options, from quick to permanent.
Option 1 — one session only, inside the REPL:
>>> /set system You are a senior engineer. Answer directly, no hedging.Option 2 — in an API call (for scripts and apps):
curl http://localhost:11434/api/chat -d '{
"model": "llama3.1",
"messages": [
{"role": "system", "content": "You are a senior engineer. Answer directly."},
{"role": "user", "content": "Postgres or SQLite for a side project?"}
]
}'Option 3 — bake it into a custom model with a Modelfile SYSTEM directive. This is the right choice for a persona you use daily:
FROM llama3.1
SYSTEM You are a senior engineer. Give the direct recommendation first, then at most 3 bullets of reasoning. If you are not confident, say exactly what you would check.
PARAMETER temperature 0.3ollama create eng -f Modelfile
ollama run engNote the temperature 0.3 — factual/technical personas benefit from lower temperature, creative ones from higher (0.8–1.0). Full Modelfile reference: Ollama docs.
In LM Studio, open a chat and look at the right-hand panel: the System Prompt field sits at the top of the chat settings. Paste your instructions there — it applies to that chat. To make it permanent, save it as a Preset (the dropdown above the field), and every new chat with that preset starts configured. The same field is exposed per-request via the system role on LM Studio's local server API.
Copy these as-is; each is deliberately short. Long system prompts dilute themselves — with 7–13B models, 50–120 words of instruction outperforms a 600-word constitution.
> You are a senior expert in whatever field the question concerns. Lead with the answer in the first sentence. Then give only the reasoning that would change my decision. No hedging, no "it depends" without stating what it depends on. If you don't know, say so plainly.
> You are a strict but constructive code reviewer. For any code I paste: 1) list bugs or correctness risks first, 2) then performance issues, 3) then style nits, clearly separated. Show fixed code only for the bugs. Never rewrite the whole file unasked. Assume the language's latest stable version.
> You are my writing editor. When I paste text, return: 1) the edited version, tightened for clarity, 2) a 2–3 bullet list of what you changed and why. Preserve my voice — make it sharper, not different. Never add facts I didn't write. Flag any sentence you weren't sure how to interpret.
> You are a patient tutor. Never give the full answer immediately. Ask me one question at a time to guide me toward it, correct my mistakes gently, and only reveal the solution when I ask for it explicitly or after three failed attempts. Keep each message under 80 words.
It's a special message, marked with the system role, that the model receives before the conversation starts. Chat models are specifically trained to weight it more heavily than user messages, which is why it's the right place for persona, tone, and format rules.
Three usual causes: the prompt is too long (trim to ~100 words), it contains contradictory instructions, or the conversation has grown long enough that early tokens lost influence. Very small models (under 3B) also follow system prompts far more loosely than 7B+ models — if compliance matters, use at least a 7–8B model.
Always try the system prompt first: it's free and instant. Fine-tuning is only worth it when you need behavior a prompt can't reach — a very specific output schema, domain vocabulary, or a style applied over thousands of generations. See Fine-Tuning vs Prompting for the full decision framework.
It occupies context-window tokens (a 100-word system prompt is roughly 130 tokens — negligible), and it's processed once at prompt evaluation. The practical cost is effectively zero; the accuracy cost of a bloated 1,000-word system prompt is real, though — small models start ignoring parts of it.