Local AI Report #1 — The Mid-2026 Local LLM & Hardware Landscape

第 #1 · 2026年7月7日 · 最后更新: 2026年7月7日 · Jakub Rusinowski

The first biweekly digest on the state of local LLMs: the mid-2026 model generation (Gemma 4, Qwen 3.6, DeepSeek V4), where GPU prices actually stand, and what the current best-value setup looks like.

要点速览

• The mid-2026 open-model generation has settled: Gemma 4, Qwen 3.6 and DeepSeek V4 are the families to evaluate first if you last checked in early 2026.
• MoE architectures keep pushing capability per GB of VRAM down-market — [VERIFY] active-parameter counts before quoting any specific ratio.
• 24 GB cards remain the sweet-spot ceiling for single-GPU setups; used NVIDIA GeForce RTX 3090 pricing is the number to watch — [VERIFY] current street price.
• Tooling churn is real: check your Ollama and LM Studio versions before pulling any of the new model families — [VERIFY] minimum versions per family.

新模型与值得关注的模型

模型	参数	显存 (Q4)	备注
Gemma 4 12B Unified	[VERIFY] 12B	[VERIFY] ~8 GB	Google’s June follow-up to the Gemma 4 line; the single-model answer for 16 GB machines.
Qwen 3.6 27B	[VERIFY] 27B	[VERIFY] ~16 GB	Efficiency refresh of the 3.5 dense line; the default general model for 24 GB cards.
Qwen 3.6 35B-A3B MoE	[VERIFY] 35B total / 3B active	[VERIFY] ~20 GB	MoE variant — fast token rates for its quality class thanks to the small active set.
DeepSeek V4-Flash	[VERIFY] 284B total / 13B active	[VERIFY] multi-GPU / 128 GB unified	The reasoning flagship you can actually self-host — if you have workstation-class memory.
Mistral Small 4	[VERIFY] 119B total / ~6.5B active	[VERIFY] ~24 GB	Long-context (256K) MoE; borderline on 24 GB — check quant fit before committing.
IBM Granite 4.1 8B	[VERIFY] 8B	[VERIFY] ~6 GB	Hybrid Mamba2+Attention — the enterprise-friendly pick for 8 GB cards.

硬件观察

The single-GPU picture hasn't changed structurally this fortnight: 24 GB of VRAM is still the ceiling that separates "runs everything sensible at Q4" from "picks its battles."

• New flagship tier: NVIDIA GeForce RTX 5090 street prices — [VERIFY] current listing vs. its launch MSRP. The premium over a used NVIDIA GeForce RTX 4090 ([VERIFY] typical used price) still buys bandwidth more than capability for most local-LLM workloads.
• Value watch: used NVIDIA GeForce RTX 3090 remains the cheapest 24 GB entry — [VERIFY] current eBay median. If it drifts under [VERIFY] threshold, it is the default recommendation again.
• 16 GB bracket: NVIDIA GeForce RTX 4060 Ti 16GB — [VERIFY] current price. Still the budget pick for 14B-class dense models and the smaller MoEs.
• Apple side: Apple M4 Pro machines handle the new mid-size MoE families comfortably on unified memory; no price movement worth acting on this issue — [VERIFY].

*All prices above are pending two-source verification and dated 2026-07-07; GPU prices in 2026 are volatile — treat anything older than a month as stale.*

工具更新

• Ollama — [VERIFY] current version and which of the new families (Gemma 4 Unified, Qwen 3.6, Granite 4.1) ship official tags. Hybrid-architecture models (Mamba2+Attention) have historically lagged a release behind.
• LM Studio — [VERIFY] current version; check MoE offload behavior on 16 GB cards before recommending the 35B-A3B class to workshop attendees.
• llama.cpp — [VERIFY] whether Granite 4.1's hybrid blocks are upstream yet; if not, the Ollama tag is the only practical path.

If you're on a setup from before June, the practical advice is unchanged: update the runtime first, then pull new models — not the other way around.

最佳性价比

The current best-value setup (unchanged from last month, pending the price checks above):

• GPU: used NVIDIA GeForce RTX 3090 24 GB at [VERIFY] price, or NVIDIA GeForce RTX 4060 Ti 16GB at [VERIFY] price if buying new on a budget.
• Daily driver model: Qwen 3.6 27B at Q4 on 24 GB; Gemma 4 12B Unified on 16 GB.
• Reasoning fallback: DeepSeek V4-Flash via a cloud API for the rare tasks that exceed local capability — the hybrid pattern beats over-buying hardware.

Check your own card against these models with the GPU & VRAM checker — the fit verdicts there use the same VRAM math as our model pages.

工作坊笔记

From recent workshop sessions: the most common mistake is still buying hardware before profiling the actual workload. Two attendees this month planned multi-GPU rigs for tasks a 27B dense model handles on a single 24 GB card. Start from the model that solves your problem, then buy the minimum hardware that runs it at Q4 with your typical context length — the cost calculator makes the break-even explicit.

方法说明. 估算值均标注为估算；经核实的数据附有来源链接。速度数据基于基准测试页面公开的内存带宽估算公式。

← 所有报告