Written by Jakub Rusinowski · Last updated March 1, 2024
Model library → BitNet b1.58 → BitNet b1.58 3B
A 3B parameter model that outperforms Llama 3B while using a fraction of the energy. Optimized for CPU inference via bitnet.cpp.
BitNet b1.58 3B needs about 3 GB of VRAM at 1.58-bit — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
| Parameters | 3 Billion |
| Context window | 2,048 |
| Architecture | BitNet (Ternary) |
| Provider | Microsoft |
| Licence | MIT |
| Specified at | 1.58-bit |
| System RAM | 4 GB |
| Record updated | 2024-03-01 |
MIT — commercial use permitted. Commercial use permitted. No usage restrictions beyond attribution.
Modelled on a reference NVIDIA RTX 4090 (24 GB), with no KV cache (this record has no published architecture). Speed figures are ESTIMATES from the memory-bandwidth roofline described on the methodology page, not benchmarks we ran — rows marked measured come from published or reader-submitted runs. VRAM here includes the KV cache, so it reads higher than the headline figure above, which does not.
| Quant | Weights | VRAM needed | Est. speed | Fit on 24 GB |
|---|---|---|---|---|
| Q2_K | 1.1 GB | 1.9 GB | ~234 tok/s (est.) | Fits comfortably |
| Q3_K_M | 1.4 GB | 2.2 GB | ~213 tok/s (est.) | Fits comfortably |
| Q4_K_M | 2.0 GB | 2.8 GB | ~183 tok/s (est.) | Fits comfortably |
| Q5_K_M | 2.4 GB | 3.2 GB | ~168 tok/s (est.) | Fits comfortably |
| Q6_K | 2.7 GB | 3.5 GB | ~156 tok/s (est.) | Fits comfortably |
| Q8_0 | 3.5 GB | 4.3 GB | ~134 tok/s (est.) | Fits comfortably |
| F16 | 6.6 GB | 7.4 GB | ~86 tok/s (est.) | Fits comfortably |
Want the memory numbers alone, at every quantization level and your own context length? Use the BitNet b1.58 3B VRAM calculator.
or compare on Vast.ai from $0.35/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
The cheapest catalogued GPU that runs BitNet b1.58 3B is the Intel Arc B570 (10 GB).
Install Ollama, then run:
ollama run hf.co/1bitLLM/bitnet_b1_58-3BWeights on Hugging Face: 1bitLLM/bitnet_b1_58-3B.
Best for: research, edge devices, cpu inference.
← All BitNet b1.58 models | VRAM calculator | Check your own hardware