Distillation
Distillation trains a small model to imitate a larger one's answers — the reason many of the best small models exist, and why they carry a bigger model's habits, good and bad.
Instead of learning only from raw text, the small "student" model is trained on what a large "teacher" model produces: its answers, its reasoning traces, sometimes the full spread of words it considered. The student never becomes the teacher, but it learns to sound like one far more cheaply than training from scratch.
For local use this is quietly the most important technique of the last few years. A distilled 8B model can hold its own on tasks that once needed a model five times its size, which is exactly what fits on a mid-range card. When a model name carries "distill", that is the claim being made.
The honest caveats: a student inherits its teacher's blind spots along with its strengths, and "distilled" is a description of the recipe, not a guarantee of quality. The hardware story is unchanged — a distilled 8B model is an 8B file, and the finder measures the file.
Where you see it on the radar
Distilled models arrive as small measured files with surprising range — the finder treats them like any other GGUF, so a strong distilled model simply shows up as a good fit on modest hardware.
See what fits a mid-range card →Related terms