AI Glossary Term

What does “Distilled Model” mean?

Definition

A smaller model trained to imitate a larger model's behavior while using less compute at inference.

Browse the full AI Glossary Explore all free AI guides

How Distilled Model works

Example

A provider releases a smaller 'mini' version of a large model that is trained to imitate it. It answers faster and costs less per request, with somewhat weaker results on hard questions.

Why it matters

Distilled models make strong behavior affordable at scale. You trade some capability for speed and price.

Common questions

Distilled model vs distillation?

Distillation is the process; a distilled model is the resulting smaller model.

Is it as good as the teacher?

Usually close on common tasks but weaker on hard or rare ones.

Why use one?

Lower latency, lower cost, and the ability to run on smaller hardware.

Inference-Time ComputeThe amount of processing power consumed while producing each response.Knowledge DistillationTraining a smaller model to imitate the outputs of a larger model.InferenceThe runtime phase where a trained model generates predictions or outputs.PruningRemoving less important model weights or neurons to reduce size and compute.Test-Time ComputeAdditional inference computation used during response generation to improve quality or reasoning.Usage-Based BillingPricing where costs scale with API calls, tokens, inference time, or consumed compute.

Learn more in our free guides

AI Models Explained Model Collapse Model Lifecycle Model Context Protocol

Test yourself

AI Models Explained Quiz Model Collapse Quiz Model Lifecycle Quiz