Tweet by _chenglou
November 30, 2025
Explanations of Attention in ml are so complicated. I think of it as roughly a hash table lookup, except instead of using an exact key, you use a fuzzy one (query) to retrieve all related keys and average their values (A weighted average, where each weight is the query and that key’s affinity)
- Author
- _chenglou
- Date
- November 30, 2025
- Canonical URL
- /tweets/chenglou-1994952618737868867-f36357