Only Have 8GB VRAM? These Are the Only Models Worth Running

Tech-focused summary (8GB VRAM models that “actually work” in 2026)

The video argues that the key limiter on 8GB GPUs isn’t just model size—it’s whether the model fits alongside KV cache and system overhead. It claims many guides overestimate what “loads” but underestimate whether the model can maintain speed at usable context lengths.

Core constraint / performance rule

• 6 GB is the realistic usable VRAM budget for running an 8GB-card model setup.
• If you exceed that, the GPU starts using system RAM over PCIe, causing a major speed drop (example cited: ~50 tokens/sec down to <2 tokens/sec).
• Selection criterion: only models that fit into ~6GB with quality quantization and still leave “reserve” for context/KV cache.
• Important distinction: “running” vs “actually working,” especially at larger context windows.

Bottom line: successful loading isn’t the same as keeping decoding speed at practical context sizes.

---

Top 5 models worth running on 8GB (with stated reasoning)

5) Gemma 3 (4B)

• Chosen not for maximum strength, but for multilingual performance.
• Claims strong switching across English/Spanish/Japanese for its size.
• With 4B quantized, expected memory use is ~2–3GB, leaving room for longer context.

Use case: document reading across multiple languages.

---

4) “54 mini” (likely “DeepSeek/54B mini” / small ~3.8B model)

• Underrated and preferred for tasks larger models struggle with.
• Best at structured output and function calls (agent loops).
• Claimed more predictable JSON/function-call behavior than a compressed ~7B model.
• Supports large context windows on 8GB.

Nuance: not ideal for “free-flow creative writing” vibes—better for structure.

---

3) Mistral 7B in Q4 (GGUF/M format)

• Presented as the most reliable general assistant.
• Good writing, knowledge, and conversational ability.
• In Q4, fits on 8GB with comfortable margin.
• Rule: keep context window within reasonable limits because KV cache grows quickly.

Use case: daily general work when you need dependability.

---

2) GLM 4.6 V-Flash (multimodal / vision)

• Highlighted for a key advantage: image understanding (multimodal, not text-only).
• Works up to about 16,000 context tokens.
• Warning: if context is pushed to ~32K, decoding speed drops dramatically (example: ~55 → ~17 tokens/sec) due to RAM limitations.

Verdict: excellent vision model on 8GB only if you keep context ≤ ~16K.

---

1) Qwen 3.5-9B in Q4 GGUF quantization (text)

• Named “winner” for 8GB because it’s both fast and fits.
• Claims on real hardware (RTX 3070 8GB):
  • ~54–58 tokens/sec consistently
  • Fits in VRAM even at 32K context, using < ~7GB
• Also claimed to rank highly on intelligent benchmarks up to 10B params.
• If you don’t need images, use the text-only version; skipping the visual encoder saves >1GB of storage/overhead.

Use case: best all-round choice for chat, documents, and programming without needing vision.

---

Caveats / guidance on quantization

• Quantization always reduces quality (“price to pay”), and Q4 GGUF-M is described as a “golden mean” for 8GB.
• Warning against builds that barely fit before KV cache/context: they may silently spill over to CPU/system RAM and destroy speed.
• The list is meant to preserve real working headroom, not just successful model loading.

---

Practical “choose this if…” recap

• Best overall intelligence (text): Qwen 3.5-9B Q4 GGUF
• Need image recognition: GLM 4.6 V-Flash (cap context near 16K)
• Reliable daily assistant: Mistral 7B Q4
• Clean JSON / function calls: “54 mini” model
• Many languages: Gemma 3 (4B)

---

Main speakers / sources

• Primary source: the video creator/host narrating personal testing (“my favorite,” “my results,” “real hardware” mentions).
• Referenced hardware: RTX 3070 with 8GB VRAM (used in their tests).
• Model authors/brands (as sources): Gemma (Google), Mistral, Qwen, GLM (vision), and the “54 mini” small model (exact provider not clearly identified in the subtitles).