Mixture of Experts
say it: “MIKS-cher uv EK-sperts” /ˈmɪkstʃər əv ˈɛkspɝts/
A model routing each token to a few specialist parts.
What is Mixture of Experts?
A mixture of experts model has many specialist sub-networks and uses only a few for each token, giving big capacity at lower cost.
Level: Master · More AI Basics terms at this level
Updated