Mixture of experts
What does Mixture of experts mean?
An architecture that routes inputs through selected specialized sub-networks.
Also known as: MoE
A practical example
A router activates a subset of experts for each token.
MICHAEL ORACLE’S AI DICTIONARY
Training & models · Advanced