Attention
What does Attention mean?
A mechanism that weights information from different positions or features when constructing a representation.
A practical example
A model relates a pronoun to relevant words earlier in a sentence.
MICHAEL ORACLE’S AI DICTIONARY
Training & models · Advanced