What is Distillation?

Training a smaller model to imitate the outputs of a larger model.

Training approaches that let a model keep learning from new data without forgetting prior knowledge.

A method that retrieves external knowledge and feeds it into generation at inference time.

Finding relevant documents or records from a knowledge source for a query.

A record of where data came from, how it was transformed, and where it is used.

A privacy technique that adds statistical noise so individual records cannot be reliably inferred from outputs.