The one rule you need
Almost everyone runs models quantized — compressed to about 4 bits per weight (the format called Q4_K_M), which keeps ~98% of the quality at roughly a quarter of the size. The rule is simple:
In practice, a model's "billions of parameters" (B) maps to VRAM roughly like this: a 7–8B model needs ~6–8 GB, a 14B needs ~10–12 GB, a 32B needs ~20–24 GB, and a 70B needs ~40–48 GB — all at Q4.
8 GB VRAM — entry level
Plenty for everyday chat, summaries and simple coding. Run 7–9B models at Q4: Llama 3.1 8B, Qwen3 8B, or Gemma 3 4B at higher quality (Q8, ~4.5 GB). For images, Sana and SDXL also fit here.
12 GB VRAM — the sweet spot for most
The best value tier. Run 12–14B models, which feel noticeably smarter: Gemma 3 12B, Qwen3 14B, Phi-4 14B, or DeepSeek-R1-Distill 14B for step-by-step reasoning.
16 GB VRAM — comfortable mid-range
Runs 14B models with room for long context, plus purpose-built 20B models like gpt-oss-20B, and even Gemma 3 27B at Q4 if you keep the context modest. A great all-rounder.
24 GB VRAM — the local power tier
Where local AI really shines. Run 27–35B models at Q4 with long context: Qwen3 32B, Gemma 3 27B, or DeepSeek-R1 32B. These rival cloud assistants for most tasks and are excellent for coding.
48 GB+ and beyond
With 48 GB you reach 70B models like Llama 3.3 70B at Q4 — near-frontier quality. Beyond that, giant Mixture-of-Experts models (DeepSeek V3, Qwen3 235B) need multiple cards or server GPUs.
No GPU? Apple Silicon & CPU
Apple Silicon (M-series) is a hidden gem: its unified memory is shared with the GPU, so a Mac with 32 GB can run models that would need a 24 GB+ NVIDIA card. A 64 GB Mac comfortably runs 70B models. On a CPU with no GPU, you can still run small 1–4B models (slowly) using system RAM — fine for testing, not for daily use.
VRAM cheat-sheet
| VRAM | Model size (Q4) | Example models | Good for |
|---|---|---|---|
| 8 GB | 7–9B | Llama 3.1 8B, Qwen3 8B, Gemma 3 4B | Chat, summaries, light coding |
| 12 GB | 12–14B | Gemma 3 12B, Qwen3 14B, Phi-4 14B | Smarter chat, reasoning, coding |
| 16 GB | 14–27B | gpt-oss-20B, Gemma 3 27B (Q4) | Long context, stronger coding |
| 24 GB | 27–35B | Qwen3 32B, DeepSeek-R1 32B | Near-cloud quality, serious coding |
| 48 GB+ | 70B+ | Llama 3.3 70B, MoE models | Frontier-class, local |
Frequently Asked Questions
How much VRAM do I need to run a local LLM?
8 GB is the practical minimum for a useful model (7–8B). 12–16 GB is the sweet spot for most people, and 24 GB lets you run 32B models that rival cloud assistants.
What is quantization (Q4)?
Quantization compresses a model to fewer bits per weight. Q4_K_M (~4 bits) is the standard: it keeps almost all the quality while cutting the size — and VRAM need — to about a quarter.
Can I run a 70B model at home?
Yes, with ~48 GB of VRAM — two RTX 3090/4090 cards, an RTX 6000, or a 64 GB Apple Silicon Mac. On a single 24 GB card, a 32B model is the practical ceiling.
Do I need an NVIDIA GPU?
NVIDIA is easiest and fastest, but not required. Apple Silicon Macs are excellent thanks to unified memory, and AMD GPUs work with tools like Ollama and LM Studio.
Does the context window affect VRAM?
Yes. Longer context (the amount of text the model reads at once) uses extra memory for the "KV cache". A big 128K context can add several GB, so leave headroom above the base model size.
What if I only have a CPU?
You can still run small 1–4B models using system RAM, but it is slow. For a real experience you want a GPU or an Apple Silicon Mac.