← Back to the AI Glossary 📖 Models & Architecture

Mixture of Experts (MoE)

A model design where only a small subset of the model's total parameters actually activates for any given input, rather than the whole model running every time.

This lets a model be enormous in total size while staying fast and affordable to run, since each request only uses a fraction of it.
One of 60 free AI glossary terms
Plain-language definitions for the AI jargon you'll actually run into — no email needed, ever, for this section.
Browse the Full Glossary →
More in Models & Architecture
Diffusion Model
A type of generative AI model that creates images (or video/audio) by starting from random...
Transformer
The neural network architecture, introduced in 2017, that made modern large language model...
Small Language Model (SLM)
A language model with far fewer parameters than a frontier model, built to run faster, che...
Encoder-Decoder Architecture
A model design split into two parts — one that reads and understands an input, and a secon...