Qwen3.8-27B: which quant to run on 12, 16, 24 and 32 GB of VRAM

On a 24 GB card (RTX 3090, 4090, 7900 XTX), run Qwen3.8-27B at a 4-bit quant such as Unsloth UD-Q4_K_XL (17.56 GB) or bartowski Q4_K_M (17.44 GB); with a q8_0 KV cache, 128K tokens of context fits by this site's analyzer. On 16 GB, UD-IQ4_XS (14.25 GB) fits with 16K context at a q8_0 cache or 32K at q4_0, or UD-Q3_K_XL (13.15 GB) with 48K. On 32 GB, step up to Q6_K with 128K. The full 262K window at Q8_0 needs a 64 GB Mac. On 12 GB the model only fits fully on the GPU at 2-bit, which costs a lot of quality, so I would offload part of it to system RAM or pick a smaller Qwen instead.

Check your exact GPU and context in the analyzer, or size the cache alone with the KV cache calculator.

This page contains affiliate links.

What Qwen3.8-27B is, from the model card

Qwen3.8-27B is a dense 27B vision-language model released under Apache 2.0 on 14 August 2026. Thinking mode is on by default.

The layout matters for memory. Only 16 of the 64 layers keep a classic KV cache. The other 48 hold a fixed-size state, however long the conversation gets.

The KV cache maths

Each token costs 16 layers × 4 KV heads × 256 dims × 2 (K and V) × 2 bytes = 64 KiB at f16. With a q8_0 cache it is about 34 KiB, and with q4_0 about 18 KiB. A community member measured 64.69 KiB per token at f16 in llama.cpp, which matches. The same person measured 38.26 KiB at q8_0, about 12% above theory, so the theory understates a q8_0 cache by roughly 10%. The figures below come from this site's engine, which counts the cache layer by layer as llama.cpp allocates it.

The DeltaNet state adds a fixed ~150 MiB per sequence (48 layers × 48 heads × 128 × 128 × 4 bytes, plus a small conv state). It does not grow with context. The engine includes it in each cache figure below, in decimal GB.

Totals are the 17.56 GB file, the cache and the engine's fixed runtime overhead (see the methodology). Add about 0.9 GB if the vision projector sits on the GPU, and about 0.7 GB for the built-in MTP draft head.

Pick your quant by VRAM

This is the table most people want. Every verdict is the analyzer's: the file, the KV cache at the context shown and runtime overhead, against the memory named, with the vision projector off the GPU. The engine counts a 24 GB card as 24 billion bytes, so its ceilings run a little shorter than some community reports: one user runs 150K context on a single RTX 3090 with a q8_0 cache (HF discussion #68).

Single-source sizes, read from the Hugging Face file listings on 20 Oct 2026: UD-IQ2_XXS 7.27 GB, bartowski Q4_K_M 17.44 GB, Q5_K_M 20.92 GB, UD-Q6_K 21.98 GB, bartowski Q6_K 23.86 GB, MLX 8-bit 29.50 GB, MLX 6-bit 22.78 GB. MLX sizes are the weight files only. UD-Q4_K_XL, UD-IQ4_XS, UD-Q3_K_XL and Q8_0 are confirmed by two sources.

On the Mac row: macOS only lets the GPU use part of unified memory by default. Apple does not publish the limit. This site assumes roughly three quarters is usable, so about 48 GB on a 64 GB machine. Q8_0 with a full f16 cache comes close to that line, so a q8_0 cache is the safer choice there.

For card-level detail, see the 16 GB, 24 GB and 32 GB pages.

The full file list, with sizes

Unsloth repacked every 27B GGUF to "Dynamic v3.0" on 19 August 2026, and sizes changed. If a guide quotes 17.9 GB for UD-Q4_K_XL, it predates or misreads the current files. bartowski also re-released his quants, built with llama.cpp b10896.

Sizes in decimal GB, read from the Hugging Face file listings on 20 Oct 2026. Both repos also ship a 0.93 GB vision projector (mmproj).

Ollama tags (all list 256K context and text + image input): MLX for Mac: lmstudio-community publishes 4-bit (16.05 GB), 5-bit (19.42 GB), 6-bit (22.78 GB) and 8-bit (29.50 GB) of weights. mlx-community has 4-bit, 8-bit, bf16, mxfp4, mxfp8, nvfp4 and MTP variants.

How much quality each quant costs

bartowski is the only quantiser I found who publishes a full text table for this model. He measured perplexity and KL divergence (KLD) against bf16 on wiki.test.raw over 100 chunks. Lower KLD is better. "Same top token" is how often the quant picks the same next token as bf16.

Source: bartowski model card, measured by the quantiser. Everything from IQ4_XS up stays close to the original. 3-bit roughly doubles KLD. 2-bit multiplies it by about ten.

Unsloth claims its Dynamic v3.0 quants beat other providers by over 10% on top-1 agreement at equal size. Its 27B KLD chart is an image only, so I can't quote numbers.

One caution: a single user reported UD-Q4_K_XL and UD-Q6_K_XL sometimes never finishing very long xhigh thinking runs, while an NVFP4 build did. It is unreplicated, and KLD would not catch it. If long agent runs loop, try a lower reasoning_effort or another quantiser.

Speed: what the community reports

I have not run these tests. Every number below is community-reported, and most are single runs.

I found no RTX 4090 figure I could cite.

The biggest lever is MTP, the model's built-in draft head. llama.cpp uses it with --spec-type draft-mtp, and reports show +40% to +60% decode speed.

Setup by runtime

llama.cpp

The GGUFs use the qwen35 architecture tag, the same family as Qwen3.5. So an "unknown model architecture" error means a build from before Qwen3.5 support. Qwen3.8 adds an extra MTP block (block 64), and older builds fail on that instead. Use b10896 or newer, the build bartowski quantised with. One user on WSL with an RTX 3090 also got garbage output from an older CUDA DeltaNet kernel, fixed around b10450.

A 24 GB starting point, built from community commands:

llama-server -hf unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL \
  -ngl 999 -c 131072 -fa on -ctk q8_0 -ctv q8_0 \
  --spec-type draft-mtp --spec-draft-n-max 2 \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --jinja

To lower reasoning depth, add --chat-template-kwargs '{"reasoning_effort":"medium"}' (Unsloth docs). To turn thinking off, pass {"enable_thinking": false} in the same way, and switch to the non-thinking sampling below. On 16 GB, --n-cpu-ffn N moves N feed-forward blocks to the CPU, the dense-model cousin of --n-cpu-moe.

LM Studio

The common failure is missing tensor 'blk.64.ssm_conv1d.weight'. It means the bundled llama.cpp runtime is too old for the MTP block. One user fixed it in the headless llmster by switching to llama.cpp runtime 2.28.2 with lms runtime get --allow-incompatible. In the desktop app, update the runtime from the runtime settings. My LM Studio "failed to load" guide covers the other causes. LM Studio has also been seen trimming the context window on its own, so check the context slider after loading.

Ollama

ollama pull qwen3.8:27b gets the 18 GB Q4_K_M build with vision. On a 16 GB card it will not fit fully, so Ollama splits it between GPU and CPU and it slows down. The -mlx, -nvfp4 and -mxfp8 tags use Ollama's MLX engine, which needs macOS. Update Ollama before pulling. Support for Qwen3.8 27B arrived in Ollama v0.32.12, so use that or newer.

MLX on Mac

Use lmstudio-community/Qwen3.8-27B-MLX-4bit (16.05 GB) on 24–32 GB Macs and the 6-bit or 8-bit build on 48 GB+.

Recommended sampling (model card)

Qwen keeps reasoning from earlier turns by default (preserve_thinking), which fills context faster. Users report 64K as a practical floor for agent work.

12 GB: squeeze it in, or pick a smaller model?

On 12 GB, Qwen3.8-27B does not fit well. Fully on the GPU, only the 2-bit files leave room for context: UD-Q2_K_XL at 32K, or UD-IQ2_XXS at 96K (see the table above). bartowski's numbers put 2-bit at roughly ten times the KLD of Q4_K_M. A 3-bit file such as UD-IQ3_XXS (10.93 GB) leaves almost no room for the cache.

Three realistic options:

• Offload part of it. If you have 32 GB+ of system RAM, run UD-IQ4_XS or UD-Q3_K_XL with --n-cpu-ffn and accept slower decode. A 16 GB RTX 5060 Ti doing this reported 11–13 tok/s; a 12 GB card will offload more and run slower. UD-IQ4_XS at 32K with a q8_0 cache: 16.3 GB, needs CPU offload on 12 GB + 32 GB RAM.
• Pick a smaller Qwen. There is no smaller Qwen3.8 dense model. The Qwen org's options are older: Qwen3.6-35B-A3B (April 2026, Apache 2.0, a mixture-of-experts that offloads well) and Qwen3.5-9B (February 2026, older than six months, so possibly stale). Check both in the model list before choosing.
• Rent a bigger GPU for the heavy jobs. A 24 GB or 32 GB cloud card runs the 4-bit or 6-bit build with long context. RunPod and Vast.ai bill by the hour; see the cloud GPU guide.

Rent GPUs on RunPod → Rent GPUs on Vast.ai →

My take

Qwen3.8-27B is the first 27B dense model I would size for 24 GB without hesitation. The hybrid layout makes its KV cache about a quarter the size you would expect, so 4-bit weights and 150K context fit on one RTX 3090 or 7900 XTX. That is the sweet spot.

On 16 GB it works, but you are trading. IQ4_XS keeps quality and gives you 32K with a q4_0 cache. Q3_K_XL gives you more room but loses quality. For a thinking model that burns context, I lean towards Q3_K_XL with 48K, unless your tasks are short.

On 12 GB I would not run it at 2-bit for anything that matters. Partial offload or a smaller model is the better call.

And turn on MTP. It is a free 40–60% on decode.

Get started

• Check your exact GPU and context in the analyzer
• Size the KV cache for any context length
• What runs on 16 GB of VRAM
• What runs on 24 GB of VRAM
• Fix "failed to load" errors in LM Studio

Frequently asked questions