Model Distillation

say it: “MOD-ul dis-tuh-LAY-shun” /ˈmɑdəl ˌdɪstəˈleɪʃən/

Training a small model to copy a larger one.

What is Model Distillation?

Distillation trains a smaller model to copy a larger one's outputs, keeping much of its ability at lower cost.

Level: Master · More AI Basics terms at this level

Updated