The part of a transformer model that decides which other words in a sentence are most relevant to understanding the current word.
It's what lets a model correctly figure out that 'it' refers to 'the dog' rather than 'the ball' in a sentence like 'The dog chased the ball until it got tired.'
One of 60 free AI glossary terms
Plain-language definitions for the AI jargon you'll actually run into — no email needed, ever, for this section.
Browse the Full Glossary →