Best Local LLMs for Document analysis
作者： Jakub Rusinowski · 最后更新： 2026年9月6日
Pulling structured facts, tables and clauses out of long documents such as contracts and reports.
Top pick: Gemma 4 31B
Scores 95.2/100 for document analysis and extraction. 31B parameters, needing about 19.5 GB at Q4_K_M, 250K context, Apache-2.0.
Ranked for document analysis and extraction
| Model | Score | Params | Context | Licence | Quality index |
|---|---|---|---|---|---|
| 1. Gemma 4 31B | 95.2 | 31B | 250K | Apache-2.0 | — (estimated) |
| 2. Qwen 3.6 27B | 94.6 | 28B | 256K | Apache-2.0 | — (estimated) |
| 3. Qwen3.8 27B | 94 | 28B | 256K | Apache-2.0 | — (estimated) |
| 4. Qwen 3.7 35B-A3B | 93.2 | 35B | 256K | Apache-2.0 | — (estimated) |
| 5. MiniMax M2.5 230B | 90.9 | 230B | 977K | Modified MIT (attribution required) | — (estimated) |
| 6. Ternary Bonsai 27B | 90.6 | 27B | 256K | Apache 2.0 | — (estimated) |
Frontier open-weight models (multi-GPU / private cloud)
These also score for document analysis and extraction, but none runs on a single consumer GPU or workstation, so they are listed apart rather than ranked against the local picks above.
- MiMo-V2.5-Pro 1T — Private cloud / datacenter cluster
- Inkling (NVFP4) — Private cloud / datacenter cluster
- DeepSeek V4-Pro — Private cloud / datacenter cluster
- DeepSeek V4.1 Flash — Multi-GPU server
- Mistral Large 3 675B-A41B — Multi-GPU server
Best pick for your memory budget
The strongest model overall is rarely the right answer — what matters is the strongest model that fits the memory you have. These picks are re-ranked per tier, so each one uses its budget rather than simply being small.
| Memory | Typical hardware | Recommended models |
|---|---|---|
| 8 GB | RTX 4060, RTX 3070, base MacBook Air | Ternary Bonsai 27B (90.6) Ternary Bonsai 2 27B (86.1) IBM Granite 4.1 Granite 4.1 8B (82.2) |
| 12 GB | RTX 3060 12 GB, RTX 5070 | Ternary Bonsai 27B (90.1) Ternary Bonsai 2 27B (85.7) Qwen 3 14B (83.4) |
| 16 GB | RTX 5080, RTX 4080, RX 9070 XT | Devstral Small 2 24B (89.6) Ternary Bonsai 27B (88.8) Ternary Bonsai 2 27B (84.4) |
| 24 GB | RTX 4090, RTX 3090, RX 7900 XTX | Gemma 4 31B (97.4) Qwen 3.6 27B (96.8) Qwen3.8 27B (96.2) |
| 48 GB | RTX 6000 Ada, MacBook Pro M4 Max 48 GB | Gemma 4 31B (95.6) Qwen 3.6 27B (94.6) Qwen3.8 27B (94) |
| 128 GB+ | Mac Studio, DGX Spark, multi-GPU | Qwen3.8-Flash-Next (93.4) Gemma 4 31B (92.8) Llama 4.5 Scout (92.4) |
How this ranking works
Same context profile as RAG but with a higher coding weight (20%), since extraction almost always ends in structured output — JSON, a table, or a schema the model must not break. Licence importance is 0.6: the documents are usually commercial.
Worked example — Gemma 4 31B: capability 92.5 × 0.418, quality 91.7 × 0.246, context 97.3 × 0.16, license 100 × 0.066, accessibility 80 × 0.111 + 3 tag bonus (long-context).
Requirements applied: context floor 32,768 tokens (ideal 262,144), quality floor 55, licence weight 0.6, latency weight 0.4.
Running document analysis and extraction locally
- Constrained/grammar-based decoding is more reliable than prompting for valid JSON, and is supported by llama.cpp and Ollama.
FAQ
What is the best local LLM for document analysis and extraction?
Gemma 4 31B, scoring 95.2/100 against this workload's published requirements. 159 models qualified.
What hardware do I need for document analysis and extraction?
A credible answer starts at 8 GB of memory. Larger budgets unlock materially stronger models — the table above lists the best pick at each tier.
How were these models ranked?
Same context profile as RAG but with a higher coding weight (20%), since extraction almost always ends in structured output — JSON, a table, or a schema the model must not break. Licence importance is 0.6: the documents are usually commercial.
Hardware for This Workload
- Best GPU for document analysis
- Best models for the NVIDIA GeForce RTX 4060 Ti 8GB
- Best models for the NVIDIA GeForce RTX 3080 Ti
- Best models for the NVIDIA GeForce RTX 4090 Laptop GPU
Related Workloads
- Best local LLMs for agents
- Best local LLMs for cybersecurity
- Best local LLMs for fine-tuning
- Best local LLMs for enterprise assistant