2023
Retrieval (RAG)
The workaround the industry lived on while context was short. Now a complement rather than a crutch.
Milestones on the line · MEMORY
Long context
A model can now hold a whole book, an entire codebase or an hour of video in front of it — instead of a couple of pages.
A million tokens stopped being a demo and became a working mode. The model holds a whole project in mind, not a single file.
StatusPASSED
TypeMilestones on the line
Marker2026
What exists today6
Researched
2024
Gemini 1.5 Pro
A million tokens in production — a whole book or an hour of video fits inside.
2025
Repository-scale context
Coding systems read the whole project.
2026-06
MiniMax M3
One-twentieth the compute per token, 9× prefill and 15× decode on million-token context.
In progress
сейчас
Lost in the middle
Long context fits, but attention to its middle sags: the model recalls the start and the end better than what lies between.
Planned
в планах
Memory that persists
Context resets with every new conversation. Memory that lives for years without growing forever is a separate unsolved problem.
Milestones on the line
To the line