This is Part 4 of the Self-Hosting LLMs series — the final installment. Read this standalone or start from Part 1.

You’ve got your framework picked out. You understand how quantization works. Now comes the question that actually determines your experience: which model do you run?

A year ago, the answer was simple — Llama or Mistral, pick a size that fits. Today the landscape is crowded, competitive, and genuinely confusing. There are half a dozen model families, each with multiple sizes, specialized variants, and overlapping claims about being “best.”

This post cuts through the noise. I’ll map out the major model families, show you how they compare on real benchmarks, and give you concrete recommendations by hardware tier and use case.

🏔️ 1. The Major Model Families

As of mid-2026, six model families dominate the open-source landscape. Each has a different philosophy, licensing approach, and sweet spot.

Meta — Llama 4

The model family that started the open-source LLM revolution. Llama 4 introduced a Mixture of Experts (MoE) architecture — a major shift from Llama 3’s dense models.

The key insight: active parameters determine speed, total parameters determine quality. Scout runs at 17B-model speeds but has 109B of learned knowledge to draw from. Maverick’s 10M token context is the longest of any open model.

License: Llama Community License (free for most commercial use, restrictions above 700M MAU)

Best for: Long-context tasks, general-purpose use, established ecosystem support

Alibaba — Qwen 3

The most versatile model family in 2026. Qwen 3 spans eight dense sizes (0.6B to 32B) plus two MoE models (30B-A3B and 235B-A22B), all under Apache 2.0.

The standout: Qwen3-4B matches Qwen 2.5-72B on benchmarks — a model needing 3GB of VRAM competing with one needing 43GB. That’s how fast the field is moving.

All Qwen 3 models feature a /think toggle that switches between chain-of-thought reasoning and fast chat mode.

License: Apache 2.0 (no restrictions)

Best for: The “default recommendation” for most local users — strong across coding, reasoning, and general tasks

DeepSeek — V3 and R1

DeepSeek runs two parallel model lines: V3 for general use and R1 for reasoning.

DeepSeek R1 dominates math and logic benchmarks — 97.3% on MATH-500, with a Codeforces rating in the 96th percentile. The distilled variants (14B and 32B) bring that reasoning capability to consumer hardware.

License: MIT (no restrictions)

Best for: Math, science, complex reasoning, competitive programming

Google — Gemma

Google’s open model family, optimized for efficiency.

Gemma 4 26B-A4B is interesting — 26B total parameters but only 3.8B active, meaning it runs at small-model speeds with large-model knowledge.

License: Gemma Terms of Use (permissive, some restrictions)

Best for: Efficiency-focused deployments, when you need maximum quality per VRAM dollar

Mistral — Devstral and Mistral

Mistral continues to punch above its weight with coding-focused models.

Devstral Small 2 is purpose-built for software tasks — coding, debugging, code review. It fits on a single 16GB+ GPU and ships under Apache 2.0.

License: Apache 2.0 (Devstral, Nemo), Mistral Research License (Large)

Best for: Coding-specific tasks, European language support

Zhipu AI — GLM-5

The newest contender, and currently the strongest all-round open model.

GLM-5.2 requires serious hardware (multi-GPU setups), but the smaller GLM-5 variants are more accessible.

License: Apache 2.0

Best for: Maximum benchmark performance, research-grade tasks

📊 2. Benchmark Comparison

Benchmarks aren’t everything, but they’re a useful starting point. Here’s how the major models compare across key tasks:

General Reasoning

Math and Reasoning

Coding

The Takeaway

No single model wins everything. DeepSeek R1 dominates math. GLM-5 leads coding. Qwen 3 has the broadest size range and best efficiency. Llama 4 has the strongest ecosystem and longest context.

For local use, the models that run on consumer hardware matter more than the 200B+ flagships. That narrows the practical field significantly.

🎯 3. Recommendations by Size Tier

This is the section you’ll actually use. Here’s what to run based on how much VRAM (or unified memory) you have:

Tier 1: 4–8 GB VRAM

Hardware: RTX 3060 (8GB), M1/M2 MacBook (8GB), GTX 1650

My pick: Start with Qwen3-8B for general use. If your tasks are reasoning-heavy, try DeepSeek R1 7B distill. The MoE models (Qwen3-30B-A3B, Gemma 4 12B-A2B) offer remarkable quality for their VRAM footprint.

ollama run qwen3:8b
ollama run qwen3:30b-a3b

Tier 2: 12–16 GB VRAM

Hardware: RTX 4070 Ti (16GB), RTX 3080 (12GB), M2 Pro (16GB)

My pick: Qwen3-14B as the daily driver. Switch to DeepSeek R1 14B when you need step-by-step reasoning for math or debugging. If you can squeeze it, Qwen3-32B at Q3_K_M (~17 GB) is a significant jump in capability.

ollama run qwen3:14b
ollama run deepseek-r1:14b

Tier 3: 24 GB VRAM

Hardware: RTX 4090, RTX 3090, M3 Max (36GB)

My pick: Qwen3-32B is the best general-purpose model you can run on a single consumer GPU. For coding specifically, Devstral Small 2 is purpose-built and excellent. DeepSeek R1 32B is the reasoning king at this size.

ollama run qwen3:32b
ollama run devstral:24b
ollama run deepseek-r1:32b

Tier 4: 48+ GB VRAM (Multi-GPU or Datacenter)

Hardware: Dual RTX 4090, A6000, A100, H100

At this tier, you’re running models that genuinely compete with cloud APIs on quality. vLLM with tensor parallelism becomes the right serving choice.

vllm serve Qwen/Qwen3-235B-A22B-AWQ \
    --tensor-parallel-size 2 \
    --quantization awq

🧪 4. Recommendations by Use Case

Coding Assistant

Your local model needs to understand code structure, follow instructions precisely, and generate syntactically correct output.

Math and Reasoning

When you need step-by-step problem solving, logical deduction, or scientific reasoning.

Document Summarization and RAG

For processing long documents, answering questions over knowledge bases, or building RAG pipelines.

General Chat / Personal Assistant

For everyday questions, drafting emails, brainstorming, and general knowledge.

⚖️ 5. How to Actually Evaluate

Benchmarks tell you what a model can do in ideal conditions. They don’t tell you how it feels to use for your specific work. Here’s a practical evaluation approach:

The 10-Prompt Test

Pick 10 real prompts from your actual work — not toy examples. Run each through 2–3 candidate models and rate:

1. Accuracy — Is the output correct?
2. Instruction following — Did it do what you asked, or what it wanted?
3. Speed — Is it fast enough for interactive use? (Under 1 second first-token latency, 20+ tok/s generation)
4. Format — Does it produce clean output, or wrap everything in unnecessary markdown?

Speed Matters More Than You Think

A model that’s 5% better on benchmarks but half as fast will frustrate you in practice. For interactive use (coding, chat, quick questions), these are rough minimums:

• Time to first token: Under 1 second
• Generation speed: 20+ tok/s for comfortable reading, 10+ tok/s for usable
• Context window: 8K minimum for coding (to fit surrounding code), 32K+ for documents

The MoE Advantage

Mixture of Experts models (Qwen3-30B-A3B, Gemma 4 26B-A4B, Llama 4 Scout) deserve special attention. They route each token through a subset of parameters, giving you:

• Large-model knowledge at small-model speeds
• Lower VRAM usage than their total parameter count suggests
• Better quality per compute than equivalent dense models

The trade-off: MoE models use more disk space (full model weights are stored) and can have slightly less consistent output quality across domains.

🔄 6. The Landscape Will Keep Moving

A year ago, running a competitive local model meant Llama 2 70B at Q4 on a beefy GPU. Today, Qwen3-4B at 3GB VRAM matches what Qwen 2.5-72B at 43GB could do — a 14x efficiency gain in one generation.

This pace will continue. By the time you read this, there may be newer models worth trying. Here’s how to stay current:

• Ollama library — check for new model additions weekly
• Open LLM Leaderboard — HuggingFace’s benchmark tracker
• r/LocalLLaMA — the Reddit community that tests everything first
• Chatbot Arena — blind human preference rankings

The good news: the tools (Ollama, quantization) and concepts you’ve learned in this series stay relevant even as the models change. ollama pull works the same whether the model behind it is Llama 4 or Llama 7.

📚 Series Recap

You made it through the whole series. Here’s what we covered:

• Part 1: Why Self-Host? — The cost math, privacy case, and your first 5-minute setup
• Part 2: Framework Showdown — Ollama, LM Studio, llama.cpp, vLLM — which to use when
• Part 3: Quantization Explained — How a 70B model fits on a 24GB GPU
• Part 4: Open Source Model Landscape ← you are here

If you’ve followed along, you now know:

• ✔️ Why to self-host (cost, privacy, control)
• ✔️ How to set up your tools (Ollama for most, vLLM for production)
• ✔️ What quantization settings to use (Q4_K_M default, GGUF format)
• ✔️ Which model to run (Qwen 3 for most, DeepSeek R1 for reasoning, Devstral for coding)

The local LLM ecosystem is no longer a hobbyist curiosity — it’s a practical, competitive alternative to cloud APIs for a wide range of tasks. And it runs on hardware you probably already own.

Questions, corrections, or model recommendations I missed? Leave a comment below, or find me on GitHub at @scrowten.