How to Use the Hardware Analyzer

The GPU & VRAM Compatibility Checker is a free, browser-based tool that tells you exactly which local LLMs your computer can run. Select your GPU or use Auto-Detect, enter your VRAM and system RAM, and it instantly lists every compatible model with VRAM usage, tokens-per-second estimates, electricity-cost figures, and one-click Ollama install commands — no sign-up, and nothing leaves your device.

How to use it

  1. Pick your GPU — choose from the dropdown or click Auto-Detect to read your hardware from the browser.
  2. Set your VRAM and system RAM — Apple Silicon users enter total unified memory.
  3. Choose your usage — daily hours and workload adjust the speed and electricity-cost estimates.
  4. Review your matches — models are grouped into runs great, runs, and won't fit, each with a copy-paste Ollama command.

Quick VRAM Guide

4 GB VRAMLlama 3.2 3B, Phi-3.5 Mini, Gemma 3 4B (Q4), SmolLM2
6–8 GB VRAMLlama 3.1 8B, DeepSeek R1 8B, Qwen 3 8B, Phi-4 Mini
12–16 GB VRAMLlama 4 Scout (Q4), Qwen 3 14B, Mistral NeMo 12B
24 GB VRAMLlama 3.3 70B (Q4), DeepSeek R1 32B, Mistral Small 3.1

Frequently Asked Questions

What hardware do I need to run a local LLM?
At a minimum you need a GPU with 4–6 GB of VRAM, or an Apple Silicon Mac with 8 GB+ of unified memory, to run small 3B–8B models smoothly. Mid-size 13B–34B models want 16–24 GB of VRAM, and 70B models need roughly 24 GB+ on a dedicated GPU (or 48–64 GB of unified memory on a Mac). You also need enough system RAM — 16 GB minimum, 32 GB recommended — because the OS and any layers offloaded from the GPU live there. The checker above matches your exact GPU and RAM against ${MODEL_VARIANT_COUNT} model variants so you don't have to guess.
How much VRAM do I need for a 7B, 13B, or 70B model?
As a rule of thumb at Q4_K_M quantization, a 7B model needs about 5–6 GB of VRAM, a 13B model about 9–10 GB, and a 70B model roughly 40–48 GB. Moving up to Q8 roughly doubles those figures, and full FP16 roughly quadruples them (a 70B at FP16 needs ~140 GB and multiple GPUs). Budget an extra 1–2 GB on top for the context/KV cache, more if you use long context windows. On Apple Silicon the model draws from unified memory instead of dedicated VRAM, so a 64 GB Mac can run a 70B model at Q4.
What's the difference between quantized and full-precision models?
Full-precision models keep every weight in 16-bit floating point (FP16/BF16) — the format they were trained in and the highest quality. Quantized models compress those weights to 8-bit, 4-bit, or lower, cutting VRAM use by 2–4× for a small and usually unnoticeable drop in quality. A 7B model, for example, shrinks from ~14 GB at FP16 to ~4.5 GB at Q4_K_M. Unless you are fine-tuning or running research evaluations, a 4-bit or 5-bit quant is the right choice for local inference.
What is the best temperature setting for an LLM?
Temperature controls randomness: lower values make output focused and repeatable, higher values make it more varied and creative. Use 0.0–0.3 for factual Q&A, coding, and data extraction; 0.6–0.8 for general chat and assistants; and 0.9–1.2 for brainstorming or creative writing. Most tools default to around 0.7–0.8, which is a sensible starting point. If you want more diversity, it is usually better to adjust top_p (nucleus sampling) than to push temperature very high.
Is running a local LLM free?
The software is free and open source — Ollama, LM Studio, llama.cpp, and the open-weight models themselves cost nothing to download or run. Your only real costs are the hardware (a GPU or a capable Mac) and the electricity to power it, which typically works out to a fraction of a cent per 1,000 tokens. Unlike cloud APIs there are no per-token fees, no rate limits, and your data never leaves your machine. For anyone with steady daily usage, a local setup usually pays for itself within a few months — the cost calculator on this site estimates your exact break-even point.
How do I choose between Ollama, LM Studio, and llama.cpp?
llama.cpp is the underlying inference engine most local tools are built on — pick it if you want maximum control, scripting, and the newest features from the command line. Ollama wraps llama.cpp in a simple CLI plus a background server with an OpenAI-compatible API, making it the easiest option for developers who want to call models from code. LM Studio is a polished desktop GUI with a model browser and chat interface, ideal for beginners or anyone who prefers clicking to typing. All three are free, run the same GGUF model files, and can happily coexist on one machine.
Can I run a local LLM without a GPU (CPU-only)?
Yes — Ollama and llama.cpp both support CPU-only inference, and it works well for small models. Expect roughly 1–8 tokens per second on a modern multi-core CPU versus 30–100+ on a dedicated GPU, so it is fine for short prompts but slow for long generations. Because the whole model loads into system RAM instead of VRAM, plan on at least 16 GB of RAM for 7B models and 32 GB for 13B models. For the best CPU-only experience, stick to 3B–8B models at Q4 quantization.
Which quantization should I choose — Q4, Q8, or FP16?
Q4_K_M is the best all-round default: it cuts VRAM roughly 4× versus FP16 with quality loss that is hard to notice in day-to-day use. Step up to Q5_K_M or Q6_K if you have spare VRAM and want a little more accuracy, or Q8_0 for near-lossless output at about double the Q4 size. Reserve full FP16/BF16 for fine-tuning, evaluation, or research where every bit of precision counts. Below Q4 (Q3, Q2) the file keeps shrinking but the model starts making noticeably more mistakes, so only go there to squeeze a model that otherwise will not fit.