Model Distillation
say it: “MOD-ul dis-tuh-LAY-shun” /ˈmɑdəl ˌdɪstəˈleɪʃən/
Training a small model to copy a larger one.
What is Model Distillation?
Distillation trains a smaller model to copy a larger one's outputs, keeping much of its ability at lower cost.
Level: Master · More AI Basics terms at this level
Updated