Best Local LLMs for Agents
作者： Jakub Rusinowski · 最后更新： 2026年10月15日
Multi-step autonomous loops where the model calls tools, reads results and decides what to do next.
Top pick: Ternary Bonsai 27B
Scores 94/100 for autonomous agents and tool use. 27B parameters, needing about 6.6 GB at Ternary (1.58-bit, ~1.71 bpw), 256K context, Apache 2.0.
Ranked for autonomous agents and tool use
| Model | Score | Params | Context | Licence | Quality index |
|---|---|---|---|---|---|
| 1. Ternary Bonsai 27B | 94 | 27B | 256K | Apache 2.0 | — (estimated) |
| 2. Devstral Small 2 24B | 91.2 | 24B | 256K | Apache-2.0 | — (estimated) |
| 3. Nemotron 3.5 Bolt 74B | 90.8 | 74B | 256K | OpenMDW-1.1 | — (estimated) |
| 4. Nemotron-Cascade 2 30B-A3B | 90.6 | 32B | 1M | NVIDIA Open Model License | — (estimated) |
| 5. Qwen3-Coder 30B-A3B (MoE) | 89.9 | 31B | 256K | Apache-2.0 | — (estimated) |
| 6. Seed-OSS-36B-Instruct | 89.5 | 36B | 512K | Apache-2.0 | — (estimated) |
Frontier open-weight models (multi-GPU / private cloud)
These also score for autonomous agents and tool use, but none runs on a single consumer GPU or workstation, so they are listed apart rather than ranked against the local picks above.
- DeepSeek V4.1 Flash — Multi-GPU server
- Kimi K2.5 — Private cloud / datacenter cluster
- Inkling (NVFP4) — Private cloud / datacenter cluster
- MiMo-V2.5-Pro 1T — Private cloud / datacenter cluster
- Qwen3.8-Max — Private cloud / datacenter cluster
Best pick for your memory budget
The strongest model overall is rarely the right answer — what matters is the strongest model that fits the memory you have. These picks are re-ranked per tier, so each one uses its budget rather than simply being small.
| Memory | Typical hardware | Recommended models |
|---|---|---|
| 8 GB | RTX 4060, RTX 3070, base MacBook Air | Ternary Bonsai 27B (94) Ternary Bonsai 2 27B (88.8) GLM-6 9B (86.8) |
| 12 GB | RTX 3060 12 GB, RTX 5070 | Ternary Bonsai 27B (93.5) Ternary Bonsai 2 27B (88.4) GLM-6 9B (86) |
| 16 GB | RTX 5080, RTX 4080, RX 9070 XT | Devstral Small 2 24B (92.6) Ternary Bonsai 27B (92.2) Ternary Bonsai 2 27B (87) |
| 24 GB | RTX 4090, RTX 3090, RX 7900 XTX | Nemotron 3.5 Bolt 74B (93.1) Nemotron-Cascade 2 30B-A3B (92.8) Devstral Small 2 24B (92.6) |
| 48 GB | RTX 6000 Ada, MacBook Pro M4 Max 48 GB | Nemotron 3.5 Bolt 74B (91.9) Nemotron-Cascade 2 30B-A3B (91) Seed-OSS-36B-Instruct (90.5) |
| 128 GB+ | Mac Studio, DGX Spark, multi-GPU | Devstral-2 123B (95.2) Qwen3.8-Flash-Next (93.9) Qwen3-Coder-Next (80B-A3B MoE) (93.1) |
How this ranking works
Reliable structured tool calls are a hard gate here, not a bonus: a model with no tool-use tag is excluded regardless of score. Coding weight is high (40%) because tool arguments are effectively code. Context needs match RAG — an agent loop accumulates every observation it has ever seen.
Worked example — Ternary Bonsai 27B: capability 89.3 × 0.427, quality 87.7 × 0.251, context 100 × 0.163, license 70 × 0.045, accessibility 100 × 0.113 + 3 tag bonus (agents).
Requirements applied: context floor 32,768 tokens (ideal 262,144), quality floor 60, licence weight 0.4, latency weight 0.6. Tool-calling capability is required.
Running autonomous agents and tool use locally
- Agent context grows monotonically across a run; a 32K window that looks generous at step 1 is exhausted by step 20.
- A single malformed tool call can end a run, so consistency matters more than peak capability.
FAQ
What is the best local LLM for autonomous agents and tool use?
Ternary Bonsai 27B, scoring 94/100 against this workload's published requirements. 54 models qualified.
What hardware do I need for autonomous agents and tool use?
A credible answer starts at 8 GB of memory. Larger budgets unlock materially stronger models — the table above lists the best pick at each tier.
How were these models ranked?
Reliable structured tool calls are a hard gate here, not a bonus: a model with no tool-use tag is excluded regardless of score. Coding weight is high (40%) because tool arguments are effectively code. Context needs match RAG — an agent loop accumulates every observation it has ever seen.
Hardware for This Workload
- Best GPU for agents
- Best models for the NVIDIA GeForce RTX 4060 Ti 8GB
- Best models for the NVIDIA GeForce RTX 3080 Ti
- Best models for the NVIDIA GeForce RTX 4090 Laptop GPU
Related Workloads
- Best local LLMs for cybersecurity
- Best local LLMs for document analysis
- Best local LLMs for fine-tuning
- Best local LLMs for ocr