Windows
作者： Jakub Rusinowski · 最后更新： 2026年9月1日
AI 教育者、本地 LLM 部署工作坊讲师
Task Manager → Performance → GPU
Dedicated GPU memory: 7.9/8.0 GB
Shared GPU memory: 6.2/15.9 GB <-- this should be 0.0 GB| 如果你看到 | 原因是 | 前往 |
|---|---|---|
| "Shared GPU memory" climbs above 0 GB while the model generates | The driver spilled the allocation into system RAM over PCIe | Fix 2: turn off the sysmem fallback for this app |
| Dedicated GPU memory is pinned at ~100% of the card | The model plus KV cache is larger than your VRAM | Fix 3: make the model fit |
| A Linux machine with the same card gives a CUDA OOM instead | Linux has no sysmem fallback, so the same allocation fails honestly there | Fix 2, then Fix 3 — you want the honest failure too |
| You are running inside WSL2 | WSL2 does not honour the Control Panel fallback setting | This page does not apply — see the WSL2 guide |
| Shared memory is 0 and it is still slow | Something else is the bottleneck — probably layers on the CPU | Fix 4: check where the layers actually are |
There is no error here either, and this is the most confusing Windows failure mode precisely because everything reports success. The model loads. The GPU is busy. Nothing is red. It just generates at two or three tokens a second on a card that should manage sixty.
Since driver branch 536 the NVIDIA Windows driver does not fail a CUDA allocation that would exceed available VRAM. It satisfies it out of system RAM instead, and Windows surfaces that pool as Shared GPU memory in Task Manager. The allocation succeeds, so your inference tool has nothing to report and reports nothing. What has actually changed is that every weight in the spilled region now crosses the PCIe bus on every single token. PCIe gives you roughly an order of magnitude less bandwidth than the card's own memory, and decoding is bandwidth-bound, so the slowdown is brutal and out of proportion to how far over budget you are. On Linux the same allocation simply fails with CUDA out of memory — which is why your Linux friend gets a clean error and you get a mystery.
Open Task Manager, go to Performance → GPU, and start a generation. Two numbers matter. Dedicated GPU memory is your real VRAM; if it is pinned near the top, you are at the ceiling. Shared GPU memory should sit at 0. Anything above zero while a model is loaded means you are streaming weights over PCIe, and that is your answer. Compare Dedicated GPU memory against the model's actual on-disk size plus a couple of gigabytes for the KV cache and overhead.
# PowerShell — the same two numbers without opening Task Manager
Get-Counter "\GPU Process Memory(*)\Dedicated Usage" -MaxSamples 1
Get-Counter "\GPU Process Memory(*)\Shared Usage" -MaxSamples 1Open NVIDIA Control Panel → Manage 3D Settings → Program Settings, add the executable that actually runs inference — ollama.exe, LM Studio, or the python.exe in your virtual environment — and set CUDA - Sysmem Fallback Policy to Prefer No Sysmem Fallback. From then on an oversized allocation returns an out-of-memory error rather than quietly crossing the bus. Set it per-program, not globally, so you do not change behaviour for games and creative apps that legitimately benefit from the fallback.
⚠️ This converts silent slowness into hard out-of-memory crashes. That is the point — an honest error tells you what to fix — but the app will now stop rather than struggle on, so expect the first run after the change to fail if you were relying on the spill.
ollama run llama3.1:8b "hi"
ollama psThe fallback policy only changes how you find out. The underlying fact is that the weights plus the KV cache plus a little overhead exceed your VRAM, and there are exactly three levers: a lower quant (Q6 to Q4_K_M roughly halves the weights for a quality difference most people cannot pick out in chat), a shorter context (the KV cache grows linearly with it, and a 32K window on an 8B model is gigabytes on its own), or a smaller model. Work out which combination fits before you download anything else.
# PowerShell — cap the context for one run and watch what changes
$env:OLLAMA_CONTEXT_LENGTH="4096"
ollama run llama3.1:8b "hi"ollama ps reads 100% GPU.ollama psShared GPU memory and CPU offload are two different slow paths and they need different fixes. If Shared GPU memory is 0 and it is still slow, the layers were never on the card to begin with: your tool chose CPU placement rather than the driver spilling VRAM. ollama ps distinguishes them in one line — a 100% GPU row with a shared-memory spill is this page's problem; a split or 100% CPU row is a placement problem.
# PowerShell
ollama ps
# llama.cpp: force all layers onto the GPU and see whether it now fails
.\llama-cli.exe -m model.gguf -ngl 999Windows hands VRAM to whatever asks first, and a browser with hardware acceleration on, a second monitor, Discord, or an open game can be holding one to two gigabytes before your model starts. On an 8 GB card that is the difference between fitting and spilling. Close them and reload the model — not just the model, because placement is decided once at load time and does not revisit the decision when memory frees up later.
# PowerShell — what is holding VRAM right now
nvidia-smi --query-compute-apps=pid,process_name,used_memory --format=csvnvidia-smi --query-gpu=memory.used,memory.total --format=csvIf you are running inside WSL2, none of the Control Panel guidance above applies — WSL2 does not honour the sysmem fallback setting, and that is an upstream limitation rather than something you have configured wrongly. If you switched to No Sysmem Fallback and are now getting hard out-of-memory errors, that is the expected outcome and the next page is about reading them.
上报问题时请附上这些信息
nvidia-smi --query-gpu=driver_version --format=csv)It is system RAM the graphics driver is allowed to use as an overflow for VRAM. For rendering it is a useful safety net. For LLM inference it is a trap: the weights held there are read over PCIe on every token, which is roughly an order of magnitude slower than the card's own memory.
Because the Windows driver has spilled oversized CUDA allocations into system RAM since driver branch 536, while Linux has no equivalent fallback. The same too-big model therefore fails loudly on Linux and degrades quietly on Windows. Same cause, two very different symptoms.
No. Set it per-program for your inference executable. Games and creative applications sometimes rely on the fallback to avoid crashing, and turning it off system-wide trades a mystery in one app for hard failures in several others.
The specific NVIDIA Control Panel setting does not exist for them, and the driver behaviour differs. The underlying arithmetic is identical though: a model larger than your VRAM will either fail or fall back to something much slower, and the fix is still to make it fit.
No — you removed the thing that was hiding the problem. The model never fitted; it was running partly out of system RAM the whole time. Drop a quant level or shorten the context until it loads cleanly, and it will be far faster than it was.