AI Glossary Term
What does “Distillation” mean?
Compressing knowledge from a large teacher model into a smaller student model.
How Distillation works
Example
A company wants a chatbot that runs on a phone. It has a large model answer thousands of questions and trains a much smaller model to produce similar answers. The small model is faster and uses less memory, though usually a bit less capable.
Why it matters
Distillation lets teams keep much of a big model's quality while cutting cost and latency. It is one route to running AI on devices or at lower prices.
Common questions
Does the student match the teacher?
Usually not fully. It typically retains much of the teacher's performance on targeted tasks but loses some breadth.
What does the student learn from?
Often the teacher's output probabilities or generated answers, which carry more information than just the right or wrong label.
Are there license concerns?
Yes. Some model providers restrict using their outputs to train competing models, so terms of use are worth checking.