Tweet by lateinteraction

February 14, 2026

RLMs are not sub-agents or the ability to iteratively retrieve context. I know because I trained multi-hop models for reasoning & retrieval in 2020, including compaction.* RLMs are the simplest/purest scaffold that understands its own prompts via recursion, not via attention. They support an extremely simple but unusual claim: Models need to be able to access their own conversations with the user and their own horizon symbolically and recursively. The model should be only allowed to understand this long context by *writing code* that launches LLMs, and composing these into the final response. Note that the number of LLM launches can be linear or even bigger in the context size, not a small constant number of sub-tasks. This sounds big until you remember that attention is already quadratic. I'll have to confess that I always found (and still find) the conventional pattern of "sub-agents" rather boring. This is the superficially related structure where the model is given a special tool it can invoke by writing out prompts for and receiving the output. Verbalizing specific individual sub-calls as tool calls token-by-token hides the internal reasoning from the main context, which is an OK outcome for sub-task delegation. But it's a completely unrelated pattern to teaching models to understand their own context/horizon recursively. Sorry I'm a bit of a pedant for understanding concepts precisely, but this seemed needed. *The title is quite literally "Robust Multi-Hop Reasoning at Scale via Condensed Retrieval", arXived on Jan 2nd 2021. It could work for many steps, retrieve text from a massive corpus, compact/condense its own context, and iterate further.

Author
lateinteraction
Date
February 14, 2026
Open on X