Z.ai
GLM-5.3
The big GLM. 2-bit still wants ~245 GB combined memory. Homelab / Mac Studio Ultra territory.
- params
- 753B MoE
- license
- GLM-5.3
- Q4-ish
- 420 GB
- context
- 1024k
- tasks
- chat · code · reason · agent
catalog
every model we match against
Curated open-weight catalog with GGUF sizes. Live Hugging Face noise lives on drops. Filter by VRAM ceiling to see what a card can hold at Q4.
38 models
Z.ai
GLM-5.3 Flash
Frontier-class multimodal MoE. 1-bit Unsloth GGUF ~93 GB; 3-bit wants ~120 GB unified. Not a 24 GB card model.
- params
- 321B MoE
- license
- MIT
- Q4-ish
- 200 GB
- context
- 1024k
- tasks
- chat · code · reason · vision · agent
Qwen
Qwen3.8-Flash-Next
180B hybrid (6B active + n-gram memory), 256k multimodal with thinking + tools. UD-Q4 ≈ 100 GB — 128 GB unified or 2×24 GB + RAM.
- params
- MoE 180B · 6B active
- license
- Qwen-Community
- Q4-ish
- 100 GB
- context
- 256k
- tasks
- chat · code · reason · vision · agent
DeepSeek
DeepSeek-V4 Flash
284B MoE with 13B active and 1M context. Even aggressive UD-IQ2 needs ~75 GB combined — 128 GB unified or multi-GPU + fat RAM.
- params
- MoE 284B · 13B active
- license
- MIT
- Q4-ish
- 160 GB
- context
- 1024k
- tasks
- chat · code · reason · agent
Qwen
Qwen3.8 27B
Best all-rounder you can actually run. Vision + reasoning + 256k context, Apache-2.0, Unsloth Q4 around 17 GB.
- params
- 27.8B dense
- license
- Apache-2.0
- Q4-ish
- 17 GB
- context
- 256k
- tasks
- chat · code · reason · vision · agent
- params
- 12B
- license
- Non-commercial
- Q4-ish
- 10 GB
- context
- n/a
- tasks
- image
Meta
Llama 4 Scout
Meta’s natively multimodal MoE. Q4 is ~55 GB — 64 GB unified or 2× 24 GB + RAM.
- params
- 109B MoE
- license
- Llama 4
- Q4-ish
- 55 GB
- context
- 10240k
- tasks
- chat · code · vision
Black Forest Labs
FLUX.1 schnell
Apache image model. 8–12 GB VRAM at fp8/nf4, 24 GB comfortable at fp16.
- params
- 12B
- license
- Apache-2.0
- Q4-ish
- 8 GB
- context
- n/a
- tasks
- image
DeepSeek
DeepSeek-R1 Distill 32B
The distill that still feels like R1. Q4 on a 24 GB card is the intended home.
- params
- 32B
- license
- MIT
- Q4-ish
- 20 GB
- context
- 128k
- tasks
- reason · code · chat
Mistral
Devstral Small
Mistral’s coding specialist. Same 24B footprint, tuned for repos and tools.
- params
- 24B
- license
- Apache-2.0
- Q4-ish
- 14 GB
- context
- 128k
- tasks
- code · agent
Qwen
Qwen3 32B
Dense 32B for 24 GB cards at Q4. Serious coding and long-horizon agent work.
- params
- 32B
- license
- Apache-2.0
- Q4-ish
- 20 GB
- context
- 128k
- tasks
- chat · code · reason · agent
Mistral
Mistral Small 3.2
Apache-2.0 24B with vision. Q4 ~14 GB — sweet spot for 16 GB cards.
- params
- 24B
- license
- Apache-2.0
- Q4-ish
- 14 GB
- context
- 128k
- tasks
- chat · code · vision · agent
OpenAI
Whisper large-v3
Speech-to-text. Q8 ~3 GB VRAM, CPU-ok. Pair with any chat model.
- params
- 1.5B
- license
- Apache-2.0
- Q4-ish
- 1.5 GB
- context
- n/a
- tasks
- audio
Google
Gemma 3 27B
Google’s dense 27B with vision. Q4 ~17 GB — same class as Qwen3.8-27B.
- params
- 27B
- license
- Gemma
- Q4-ish
- 17 GB
- context
- 128k
- tasks
- chat · vision · code · reason
Qwen
Qwen2.5-Coder 32B
Still one of the best local coding models. Q4 ~20 GB.
- params
- 32B
- license
- Apache-2.0
- Q4-ish
- 20 GB
- context
- 32k
- tasks
- code · agent
Meta
Llama 3.3 70B
Still the 70B workhorse. Q4 wants ~40 GB (2× 24 GB or 48 GB unified).
- params
- 70B
- license
- Llama 3.3
- Q4-ish
- 40 GB
- context
- 128k
- tasks
- chat · code · reason
NVIDIA
Nemotron 3 Nano 30B-A3B
3B active MoE. Fast on consumer GPUs, official GGUF from ggml-org.
- params
- 30B-A3B
- license
- NVIDIA
- Q4-ish
- 18 GB
- context
- 128k
- tasks
- chat · code · agent
Qwen
Qwen3.5 9B
Unified vision-language 9B. The 8 GB card champion — multilingual, sharp at code, Q4 sits around 5.7 GB with cache room.
- params
- 9B
- license
- Apache-2.0
- Q4-ish
- 5.7 GB
- context
- 128k
- tasks
- chat · code · reason · vision
Google
Gemma 3 12B
Multimodal 12B. Q4 ~8 GB, Q5 ~10 GB. Excellent on 12 GB cards.
- params
- 12B
- license
- Gemma
- Q4-ish
- 8.1 GB
- context
- 128k
- tasks
- chat · vision · code
OpenGVLab
InternVL3 8B
Strong open vision-language at 8B. Q4 ~6 GB. Screenshot / document QA.
- params
- 8B
- license
- Apache-2.0
- Q4-ish
- 6 GB
- context
- 32k
- tasks
- vision · chat
DeepSeek
DeepSeek-R1 Distill 8B
Distilled reasoner. Long traces, strong math, fits any 8 GB card at Q4.
- params
- 8B
- license
- MIT
- Q4-ish
- 5.2 GB
- context
- 32k
- tasks
- reason · code · chat
Nomic
Nomic Embed Text v1.5
Local RAG embeddings. CPU is enough. Matryoshka dims.
huggingface
ollama pull nomic-embed-text- params
- 137M
- license
- Apache-2.0
- Q4-ish
- 0.3 GB
- context
- 8k
- tasks
- embed
OpenBMB
MiniCPM-V 4.5
OCR and screenshot specialist. Fits 8 GB at Q4.
- params
- 8B
- license
- Apache-2.0
- Q4-ish
- 5.8 GB
- context
- 32k
- tasks
- vision · chat
Microsoft
Phi-4 14B
Small model, dense knowledge. Q4 ~9 GB, Q5 ~11 GB. Punches above 14B on STEM.
- params
- 14B
- license
- MIT
- Q4-ish
- 9 GB
- context
- 16k
- tasks
- chat · code · reason
Qwen
Qwen3 8B
Thinking-mode 8B. Fast, cheap, still beats most 7B-class models at reasoning.
- params
- 8B
- license
- Apache-2.0
- Q4-ish
- 5.2 GB
- context
- 128k
- tasks
- chat · code · reason
- params
- 82M
- license
- Apache-2.0
- Q4-ish
- 0.5 GB
- context
- n/a
- tasks
- audio
Stability
Stable Diffusion 3.5 Medium
Runs on 8 GB. Open-ish weights, good default if FLUX is too heavy.
- params
- 2.5B
- license
- Stability Community
- Q4-ish
- 6 GB
- context
- n/a
- tasks
- image
Cohere
Command R7B
RAG-native 7B with tools. Non-commercial license. Great retrieval chat.
- params
- 8B
- license
- CC-BY-NC
- Q4-ish
- 5 GB
- context
- 128k
- tasks
- chat · agent
IBM
Granite 3.3 8B
Apache, long context, enterprise-clean. Solid 8 GB citizen.
- params
- 8B
- license
- Apache-2.0
- Q4-ish
- 5.1 GB
- context
- 128k
- tasks
- chat · code · reason
Qwen
Qwen2.5 7B
Battle-tested 7B. Still a great 6–8 GB default if you want something boring and stable.
- params
- 7B
- license
- Apache-2.0
- Q4-ish
- 4.7 GB
- context
- 32k
- tasks
- chat · code
Ai2
OLMo 2 13B
Fully open (data + code + weights). Q4 ~8 GB. The research-honest pick.
- params
- 13B
- license
- Apache-2.0
- Q4-ish
- 8 GB
- context
- 4k
- tasks
- chat
OpenBMB
MiniCPM5 2B
Sept-2026 edge refresh: 2.5B, 128k context, tool-calling. CPU-first; Q4 ≈ 1.6 GB.
- params
- 2.5B
- license
- Apache-2.0
- Q4-ish
- 1.6 GB
- context
- 128k
- tasks
- chat · code · agent
01.AI
Yi 1.5 9B
Bilingual EN/ZH 9B. Q4 ~6 GB. Still a good 8–12 GB pick.
- params
- 9B
- license
- Apache-2.0
- Q4-ish
- 5.8 GB
- context
- 4k
- tasks
- chat · code
Qwen
Qwen3 4B
Laptop / 6 GB card model. Surprisingly useful for chat and light coding.
- params
- 4B
- license
- Apache-2.0
- Q4-ish
- 2.6 GB
- context
- 32k
- tasks
- chat · code
Google
Gemma 3 4B
Tiny multimodal. Phone-class, also great on iGPU laptops.
- params
- 4B
- license
- Gemma
- Q4-ish
- 2.8 GB
- context
- 128k
- tasks
- chat · vision
Microsoft
Phi-4 Mini
CPU-friendly 3.8B with 128k context. Fine default for 8 GB RAM machines.
- params
- 3.8B
- license
- MIT
- Q4-ish
- 2.5 GB
- context
- 128k
- tasks
- chat · code
Hugging Face
SmolLM3 3B
On-device 3B from HF. Dual-mode thinking, tiny footprint.
- params
- 3B
- license
- Apache-2.0
- Q4-ish
- 2 GB
- context
- 64k
- tasks
- chat · code
Meta
Llama 3.2 3B
Edge / CPU-only. Fine for summarization and short chat on 8 GB laptops.
- params
- 3B
- license
- Llama 3.2
- Q4-ish
- 2 GB
- context
- 128k
- tasks
- chat
every model · vram requirements · quant size · ollama command
model VRAM requirements table — run locally.
Each row lists params, the Q4 VRAM requirement in GB, the smallest quant that runs locally, and the copy-paste Ollama command with a GGUF download link. Compare rows to decide which model is better for your rig. Sizes exclude KV cache — leave ~10% headroom.
| model | params | VRAM requirements (Q4) | runs locally from | ollama command |
|---|---|---|---|---|
| Qwen3.8 27B | 27.8B dense | 17 GB | 17 GB (Q4) | ollama run qwen3.8:27b |
| Qwen3.5 9B | 9B | 5.7 GB | 5.7 GB (Q4) | ollama run qwen3.5:9b |
| Qwen3 8B | 8B | 5.2 GB | 5.2 GB (Q4) | ollama run qwen3:8b |
| Qwen3 32B | 32B | 20 GB | 16 GB (Q3) | ollama run qwen3:32b |
| Qwen3 4B | 4B | 2.6 GB | 2.6 GB (Q4) | ollama run qwen3:4b |
| GLM-5.3 Flash | 321B MoE | 200 GB | 93 GB (IQ2) | llama-cli -hf unsloth/GLM-5.3-Flash-GGUF:UD-IQ2_M |
| GLM-5.3 | 753B MoE | 420 GB | 245 GB (IQ2) | llama-cli -hf unsloth/GLM-5.3-GGUF:UD-IQ2_M |
| DeepSeek-R1 Distill 8B | 8B | 5.2 GB | 5.2 GB (Q4) | ollama run deepseek-r1:8b |
| DeepSeek-R1 Distill 32B | 32B | 20 GB | 16 GB (Q3) | ollama run deepseek-r1:32b |
| DeepSeek-V4 Flash | MoE 284B · 13B active | 160 GB | 76 GB (IQ2) | llama-cli -hf unsloth/DeepSeek-V4-Flash-GGUF:UD-IQ2_M |
| Llama 4 Scout | 109B MoE | 55 GB | 42 GB (Q3) | ollama run llama4:scout |
| Llama 3.3 70B | 70B | 40 GB | 32 GB (Q3) | ollama run llama3.3:70b |
| Llama 3.2 3B | 3B | 2 GB | 2 GB (Q4) | ollama run llama3.2:3b |
| Gemma 3 12B | 12B | 8.1 GB | 8.1 GB (Q4) | ollama run gemma3:12b |
| Gemma 3 27B | 27B | 17 GB | 17 GB (Q4) | ollama run gemma3:27b |
| Gemma 3 4B | 4B | 2.8 GB | 2.8 GB (Q4) | ollama run gemma3:4b |
| Phi-4 14B | 14B | 9 GB | 9 GB (Q4) | ollama run phi4 |
| Phi-4 Mini | 3.8B | 2.5 GB | 2.5 GB (Q4) | ollama run phi4-mini |
| Mistral Small 3.2 | 24B | 14 GB | 14 GB (Q4) | ollama run mistral-small3.2 |
| Devstral Small | 24B | 14 GB | 14 GB (Q4) | ollama run devstral |
| Qwen2.5-Coder 32B | 32B | 20 GB | 16 GB (Q3) | ollama run qwen2.5-coder:32b |
| Qwen3.8-Flash-Next | MoE 180B · 6B active | 100 GB | 48 GB (IQ2) | llama-cli -hf unsloth/Qwen3.8-Flash-Next-GGUF:UD-Q4_K_XL |
| Nemotron 3 Nano 30B-A3B | 30B-A3B | 18 GB | 18 GB (Q4) | llama-cli -hf ggml-org/NVIDIA-Nemotron-3-Nano-30B-A3B-GGUF:Q4_K_M |
| OLMo 2 13B | 13B | 8 GB | 8 GB (Q4) | ollama run olmo2:13b |
| SmolLM3 3B | 3B | 2 GB | 2 GB (Q4) | ollama run smollm3 |
| FLUX.1 schnell | 12B | 8 GB | 8 GB (Q4) | Not an LLM — run via ComfyUI / diffusers |
| FLUX.1 dev | 12B | 10 GB | 10 GB (Q4) | see GGUF download |
| Stable Diffusion 3.5 Medium | 2.5B | 6 GB | 6 GB (Q4) | see GGUF download |
| Whisper large-v3 | 1.5B | 1.5 GB | 1.5 GB (Q4) | whisper.cpp or faster-whisper large-v3 |
| Kokoro 82M | 82M | 0.5 GB | 0.5 GB (FP16) | see GGUF download |
| Nomic Embed Text v1.5 | 137M | 0.3 GB | 0.3 GB (Q8) | ollama pull nomic-embed-text |
| InternVL3 8B | 8B | 6 GB | 6 GB (Q4) | llama-cli -hf unsloth/InternVL3-8B-GGUF:Q4_K_M --mmproj auto |
| MiniCPM-V 4.5 | 8B | 5.8 GB | 5.8 GB (Q4) | ollama run minicpm-v |
| MiniCPM5 2B | 2.5B | 1.6 GB | 1.6 GB (Q4) | llama-cli -hf openbmb/MiniCPM5-2B-GGUF:Q4_K_M |
| Qwen2.5 7B | 7B | 4.7 GB | 4.7 GB (Q4) | ollama run qwen2.5:7b |
| Granite 3.3 8B | 8B | 5.1 GB | 5.1 GB (Q4) | ollama run granite3.3:8b |
| Command R7B | 8B | 5 GB | 5 GB (Q4) | ollama run command-r7b |
| Yi 1.5 9B | 9B | 5.8 GB | 5.8 GB (Q4) | ollama run yi:9b |