Attention heads' computations predict brain activity. Do they correspond to a real brain mechanism — cue-based memory retrieval, hippocampal sequence codes — or are they just good feature extractors?
It decides whether 'attention' in Correlate can ever move beyond analogy.
What attention heads compute predicts brain activity along depth and distance gradients, but no study shows the brain computes attention the same way.
A neutral summary kept current by super users, not a verdict.
Heads approximate cue-based retrieval in working memory.
Ryu & Lewis 2021; Kumar 2024
Transformers relate formally to hippocampal models.
Whittington et al. 2022
Heads are useful features with no specific biological counterpart.
Open
No decisions yet. When this room changes a flow or an evidence label, it is recorded here permanently.
Dhi: Room opened.