The brain seems to forecast several words ahead, at several levels. Would training models on longer-range, hierarchical predictions make them more brain-like — and more data-efficient?
A direct route from neuroscience back into AI design — the 'translated into AI development' half of Dhi.
Adding longer-range forecasts to language models improved fit to frontal and parietal brain areas, but only modestly so far (~2%).
A neutral summary kept current by super users, not a verdict.
Multi-scale forecasting objectives improve both brain fit and sample efficiency.
Suggested by Caucheteux et al. 2023
Gains are small; next-token prediction already captures most of it implicitly.
Open
No decisions yet. When this room changes a flow or an evidence label, it is recorded here permanently.
Dhi: Room opened. Note: the long-range fine-tune in Caucheteux 2023 improved IFG/AG fit by only ~2% — modest so far.