Local LLM not working on Windows

Windows breaks local AI in ways the other two platforms mostly don't, and almost none of them announce themselves as an error. The driver installs, the model loads, generation starts — and it is running on your CPU at four tokens a second while a perfectly good RTX sits at zero percent. That is the shape of most Windows problems here: something silently chose the slow path and nothing told you.

There are four culprits behind nearly all of it. The NVIDIA driver on Windows falls back to system RAM instead of returning an out-of-memory error, so an oversized model degrades instead of failing honestly. Ollama installs per-user without Administrator rights, so a terminal that was already open — or one opened as Administrator — cannot find it. Hybrid laptops hand a command-line process the integrated GPU by default. And WSL2 has its own driver plumbing that a well-meant Linux driver install inside the distro will quietly destroy.

Start with the diagnosis table on whichever page matches your symptom. If you do not have a symptom yet, run ollama ps and look at the Processor column — 100% GPU, 100% CPU, or a split tells you which half of this list you are in.