Local AI Report #3 — Best Small LLMs for 8 GB and 16 GB RAM Laptops

Your laptop does not hand the whole of its memory to a model. The operating system, your browser and everything else already resident take a share, and committing the remainder to a model is what turns a working machine into a swapping one.

We size the budget as a fraction of installed memory rather than by subtracting an "OS footprint", because the footprint question does not have one answer. On Windows, the figure in Task Manager labelled In use and the figure a vendor quotes as "used" are different quantities: the larger one usually includes the standby cache, which holds cached file data and is reclaimable on demand. A model can take that memory back. It cannot take back what is genuinely committed.

*Read Available, not In use. Available is the number that tells you what a model can have.

Installed RAM	Platform	Budget for the model	Basis
8 GB	Windows / Linux	6.40 GB	80% of installed
16 GB	Windows / Linux	12.80 GB	80% of installed
8 GB	macOS	6.00 GB	75% of unified memory
16 GB	macOS	12.00 GB	75% of unified memory

The macOS figure is lower because the GPU's share of unified memory is capped by the system. That cap is adjustable, so 75% is a sensible default rather than a hard wall.

This is system RAM*, not VRAM. If you have a discrete graphics card, its dedicated memory is the number that matters instead and this page does not apply — see Best LLMs for 8 GB VRAM or Best LLMs for 16 GB VRAM. A discrete 8 GB card runs models this page rules out, because its bandwidth is five to ten times a laptop's.

A model's parameter count tells you how much memory it occupies. It does not tell you how fast it runs. Those are two different quantities and conflating them is the most common mistake in this category.

Residency is every parameter, because all of them must be in memory. Per-token traffic is only the parameters actually read to produce one token. For an ordinary dense model these are the same. For a Mixture-of-Experts model they are not: only a fraction of the experts activate per token, so a large MoE can occupy a great deal of memory and still generate quickly.

That is why the tables below carry two different columns, and why a 12B and a 26B-A4B are not comparable on size alone. The full arithmetic is at our methodology page, and what quantisation actually costs you covers the other half of the trade.

Every row computed at Q4_K_M with an f16 KV cache, against the 6.40 GB budget. "Max context" is the largest standard context length whose total footprint still fits.

The practical pick is a 4B-class model at 16K context. Llama 3.2 3B and Phi-4 Mini leave the most headroom; Gemma 3 4B is the largest that still holds 16K comfortably.

Do not pull these on 8 GB

The widely-repeated claim that "8 GB runs a 7B" is where most articles go wrong. A 7B–8B at Q4 is roughly 4.6–5.4 GB of weights alone; add a KV cache for anything past a couple of thousand tokens and you are over. It loads, then it swaps.

Same method, against the 12.80 GB budget.

16 GB is where a laptop becomes genuinely useful. A 9B at 32K handles summarising and drafting without feeling cramped. The 14B models fit, but read the speed section before choosing one — they are three to four times slower than a 4B on the same machine, and on a laptop that is the difference between usable and abandoned.

Still nothing here for GPT-OSS 20B (12.07 GB), Gemma 4 26B-A4B (15.70 GB) or Qwen 3.5 27B (16.30 GB). Those need more memory than a 16 GB machine can give them.

For workload-specific picks once you know your budget, see general assistant, summarisation, translation and RAG over your own files.

Generating a token means reading the model's active weights out of memory. Every token. So the ceiling is arithmetic:

> tokens/sec = memory bandwidth × efficiency ÷ bytes read per token

Your CPU's core count barely enters into it. This is why a faster processor with the same memory does not help much, and why a second memory stick does.

What your laptop's memory actually delivers

For comparison, a mid-range discrete card is 300–500 GB/s and a high-end one exceeds 1,000. That gap, not the CPU, is why laptop inference is slower.

Estimated generation speed

Every figure below is ESTIMATED. We publish a range, not a point, and the range is wide on purpose — see the methodology note at the foot of this page.

Roughly: anything under about 5 tok/s is painful for long output and fine for short answers. Above 15 feels conversational.

We are not publishing time-to-first-token. Our model for it is wrong by around an order of magnitude on hardware that is not an NVIDIA card, and a wrong TTFT is worse than none.

Single-channel memory is the most common unforced error on a laptop, and the arithmetic above shows exactly what it costs: DDR5-5600 delivers 44.8 GB/s on one stick and 89.6 GB/s on two. Exactly double. Since generation is bandwidth-bound, so is your token rate.

Many machines ship with one stick in a two-slot chassis. Adding a second matched stick is usually the cheapest performance change available, and it is a bigger win than dropping from a 4B to a 2B.

Check what you have:

• Windows — Task Manager → Performance → Memory, look at "Slots used"
• macOS — unified memory, not applicable; the bandwidth is fixed by the chip
• Linux — sudo dmidecode -t memory | grep -i "locator\|size"

Note the trade: soldered LPDDR5X in a thin-and-light is fast (120 GB/s) but cannot be upgraded. A socketed machine is slower out of the box and fixable.

Platform setup details are on Windows, macOS and Linux.

Model cards advertise context windows of 128K or 256K tokens. On a laptop those numbers are unreachable, because the KV cache — the model's memory of the conversation so far — grows linearly with context and comes out of the same budget as the weights.

The cache cost per 1,024 tokens for the models above runs from about 0.067 GB (Qwen 3.5 0.8B) to 0.210 GB (Phi-4 14B). At 64K tokens that is between 4 GB and 13 GB — on its own, before the weights.

That is what the "max context that fits" column is doing: it is the largest standard length where weights plus cache plus runtime overhead still clear the budget.

The one-line fix: quantise the cache. Setting OLLAMA_KV_CACHE_TYPE=q8_0 roughly halves cache memory and buys back context. It costs some quality; we have not measured how much, so we are not going to put a number on it.

One caveat we would rather state than hide: the "KV basis" column marks whether the cache figure is computed from a model's published attention configuration (measured) or inferred from its parameter count (estimated). The inference is not reliable — across models where we can check it against the real configuration, its average error is about 51%. Treat every estimated row as approximate, and prefer a measured row when choosing between two similar models.

A local model on a laptop is genuinely useful for a narrow band of work. It is not a substitute for a frontier model, and the gap is largest exactly where people most want it to be small.

1. Long documents are slow at the start, not just during output. Reading a 4,000-word document is several thousand tokens of processing before the first word appears. On a laptop that wait is measured in minutes, not seconds. 2. A 4B model is not a 400B model. Expect competent summarising, rewriting and short answers. Do not expect reliable multi-step reasoning, current facts, or code you can ship unread. 3. Quality drops with quantisation and we are not quantifying it here. Q4 is the standard trade-off; below Q4 the loss becomes obvious. See quantisation explained. 4. Battery and heat are real. Sustained generation runs the machine hard. Fanless laptops throttle within minutes. 5. We hold zero CPU benchmarks. Every speed figure on this page is a model output, not a measurement — see the note below. 6. Context is far shorter than advertised, per the section above. 7. We are not publishing download sizes. The figure a model registry reports is the only authority, and we could not read it from our build environment. We would rather leave a gap than print a number we did not check. 8. An NPU will probably not help. Current local runtimes largely target CPU and GPU; a Copilot+ NPU is excellent at small always-on models and a poor general LLM engine. 9. A cheap discrete GPU beats all of this. Even an entry-level card with 8 GB of VRAM outruns any CPU configuration here, because its bandwidth is several times a laptop's — see our 4 GB VRAM guide and what 32 GB of VRAM buys if you are weighing an upgrade against running in the cloud.

Apple Silicon runs the same arithmetic with two structural advantages. Memory is unified, so there is no separate VRAM pool to fill and no copying between them. And bandwidth is high for the class: 68 GB/s on an M1 and 120 GB/s on an M4, against 38–90 GB/s for typical socketed laptop memory.

The offsetting constraint is the system's cap on how much unified memory the GPU may address, which is why we budget 75% rather than 80%. On a 16 GB Mac that is 12.00 GB.

An 8 GB Apple laptop is still an 8 GB machine. It gets 6.00 GB and the same 4B-class ceiling as any other 8 GB laptop — the bandwidth advantage makes it faster, not roomier.

Setup specifics are on Local AI on macOS.

Memory figures are exact arithmetic and we stand behind them. Weights are the parameter count times the quantisation's bits-per-weight. The KV cache is computed from each model's attention configuration where we have it, and inferred from parameter count where we do not — the "KV basis" column says which. Total footprint adds a fixed runtime overhead. The same functions produce these numbers as produce every other memory figure on this site, so a page here cannot disagree with the VRAM calculator or with our methodology page. Speed figures are estimates, and here is precisely why they are ranges. Throughput is bandwidth × efficiency ÷ bytes-per-token. Bandwidth and bytes-per-token are known. Efficiency — what share of theoretical peak the hardware actually sustains — is not, for CPUs, because we hold zero CPU or integrated-GPU benchmarks. Our entire measured set is 14 rows, all discrete-GPU or Apple Metal.

So the ranges above are bounded by two named endpoints rather than a made-up margin:

• 0.35 — the site's uncalibrated seed value for CPU decode, which we have reason to think is pessimistic.
• 0.85 — what the published third-party measurements we could actually verify imply.

The true figure for your machine is somewhere in between. We would rather show you that honestly than pick a midpoint and present it as knowledge.

On third-party benchmarks. We checked six published laptop measurements against physics before using any of them. Three required reading memory faster than the memory bus allows — one implied 99.6 GB/s on an 89.6 GB/s bus. Those were discarded. Of the three that survived, our estimate brackets each: for a 12B on single-channel DDR5 the observed figure was 4.5–5.3 tok/s, inside the range this model produces. This is the check that most published comparisons skip. What is deliberately missing. No download sizes, no model release dates, no time-to-first-token. The first two need a model registry we could not reach from our build environment; the third needs a constant that is wrong by roughly an order of magnitude off NVIDIA hardware. Every number that is here, we checked. Full workings: our methodology and the benchmark data.