Remember everything, inject almost nothing
The genius with amnesia
A language model keeps nothing between one call and the next. What it knows about you is what is on its desk.
On Monday Maya tells her assistant that her child has a serious peanut allergy. The assistant says "Noted!" and it is not lying: during Monday's chat it really does know. On Thursday the knowledge is gone. For Maya's family this is not a small bug. It is a dangerous one.
A language model is stateless: it keeps nothing between one call and the next. When a model is trained, its knowledge is frozen into a very large file of numbers. Talking to it writes nothing into that file. A conversation is only ever placed on the desk, the context window, for the model to read.
Think of the best doctor in the city, who forgets you each time you leave her room. Real doctors solved this long ago. They did not grow a bigger brain. They keep a patient file: a few lines read before you enter, a few lines added after you leave. A memory layer is that file, built for an assistant.
Three fixes that fail
Resend everything, buy a bigger window, or train a model per user. Each fails, and each failure leaves one rule.
Fix 1 is to resend all the history with every message. It works for one session and it has an expiry date. A model charges for every token it reads, so the bill grows with every session. Memory's read cost is flat: three remembered facts of about 25 tokens each is 75 tokens, after ten sessions or ten thousand. Rule 1: store a distilled form, not the raw transcript.
Fix 2 is a bigger context window. It moves the expiry date. It does not change the bill: with a full window, one user asking small questions costs thirty-six dollars a day. And a model finds a fact more easily at the start or end of a long prompt than in the middle, which is where the allergy sentence will be. Rule 2: inject a small, fixed amount per message.
Fix 3 is to train a model for each user. A million users means a million models, and a training run after every chat. Worse, a model's numbers are not a database: we cannot open them and read "Maya is vegetarian", correct it, or prove we deleted it. Rule 3: keep memory outside the model, in a store we can read, edit and delete.
An open-book exam, and every school brings its own examiner
The student is the memory system plus the model that answers. The examiner is another model. Two posters that say 92 and 94 tell us very little.
A memory benchmark is an exam with an open book. The book is a very long conversation. The questions ask about things said in it. The student is the memory system plus the model that writes the answer. The examiner is another AI model that compares each answer with the correct one.
Now imagine every school brings its own student and chooses its own examiner. Each school also decides how many pages of the book the student may open. Then it prints only the final mark on a poster. That is the state of memory benchmarks today.
Three exams matter. LoCoMo: ten very long conversations, about 300 turns each, 1,540 scored questions. LongMemEval-S: 500 questions, each hidden in about 115,000 tokens of chat sessions, including questions the system should decline. BEAM: conversations up to ten million tokens, graded point by point against a rubric.
Five dials that move a score without touching memory
The answer model, the judge, the context budget, the question set, and luck. A score is a claim until it names them.
A benchmark score is produced by a whole pipeline, and memory is only one part of it. We measured each dial on our own system, with the memory untouched.
The judge. We graded one set of 1,540 LoCoMo answers twice. One judge gave 88.5, the other 94.0. The answers were identical. Neither judge is wrong: they draw the line for a paraphrase in different places. The answer model. On LongMemEval, the same memories scored 81.0 with one reader and 88.6 with the next one up.
The question set. A 100-question sample of LongMemEval scored 91; the full 500 scored 81.0 with the same code. Luck. One configuration, run twice, scored 93.51 and then 92.47. So in everything that follows: all questions, both judges where we have them, and the context tokens beside every score.
LoCoMo: the highest published score, on a fifth of the context
96.1 on all 1,540 questions at 1,533 tokens per question. The next three published scores read 6,700 to 45,800.
On LoCoMo, hanumemAI has the highest score we can find published: 96.1 on all 1,540 questions. The answering model is a small open-weight one. The judge is gpt-4o-mini with the prompt from Mem0's own open harness, unchanged.
Agent Zero reports 93.6, reading 45,800 tokens per question. EverMemOS reports 93.05 at about 6,700. Mem0's platform reports 92.5 at 7,000, with gpt-5 as both reader and judge. hanumemAI read 1,533.
The same 1,540 answers under a stricter judge score 91.6. We publish both. The chart shows the claim in one picture: up is more accurate, left is cheaper.
A million tokens of history: ahead of Mem0
BEAM-1M, all 700 questions: 68.5 against Mem0's 64.1, at 40% of the tokens.
BEAM asks whether memory holds at extreme length. At one million tokens per conversation, across all 35 conversations and 700 questions, hanumemAI scores a mean rubric of 68.5 and a pass rate of 73.4, reading 2,716 tokens per question.
Mem0 reports 64.1 with gpt-5 as reader and judge, at 6,700 tokens. Models that read the whole conversation with no memory system score 31 to 34. At 100,000 tokens we score 70.5.
By ability at one million tokens, the strongest are preferences (99) and following instructions (86). The weakest are summarization (53), event ordering (59) and temporal reasoning (64).
LongMemEval: the top group, with a small open model
94.0 on all 500 questions at 2,428 tokens. Every other published score of 93 or more uses a frontier reader.
LongMemEval tests whether memory stays correct when facts change. On all 500 questions hanumemAI scores 94.0 at 2,428 tokens per question; the same answers under a second judge score 91.8.
This is not first place. agentmemory reports 96.2; Agent Zero and Chronos 95.6; Hindsight 94.6; Mem0 94.4. Each of those uses gpt-5, gpt-5.5, Claude Opus 4.6 or Gemini Pro as the reader, at 6,800 to 43,600 tokens per question. hanumemAI's reader is a small open-weight model.
By question type: knowledge update 100, single-session 97 to 100, temporal 97, abstention 87, multi-session 84. Counting across many conversations is the open problem.
What we cannot claim
Not first on LongMemEval or BEAM, no shared judge with anyone, and no separate test set.
The claims that are not ours belong on the same page as the ones that are. First place on LongMemEval-S: agentmemory reports 96.2. First place on BEAM: Hindsight reports 73.4 at 100,000 tokens and 73.9 at a million, reading 17,700 and 23,900 tokens per question.
No like-for-like win. No two vendors share a judge, and none of these numbers, ours included, has been reproduced by a third party. Other systems' figures are as their vendors published them.
No held-out result. The experiments were tuned on the same question sets they are scored on. Failures were read on 179 of the 500 LongMemEval questions only, and the 321 never looked at improved as much (78.5 to 84.4), but there is no separate test set.
A bigger reader did not help
We put gpt-5 in the reader's seat on the same memories. It did not beat the small model.
Would a frontier model on top of the same memory take first place on a second benchmark? We tested it before spending on full runs: gpt-5 as the reader, same memories, same judge, same questions.
On a 100-question sample of LongMemEval, gpt-5 scored 94.0 and the small reader 95.0. On 100 BEAM questions, 67.0 against 69.6. The samples are small, and the answer prompt was tuned with the small reader, so this measures gpt-5 inside our stack, not gpt-5 at its best.
The lesson we took: once the right few lines are on the desk, a small model reads them as well as a large one. The work is in choosing the lines.
What is public, what is not, and where to go next
The results are published. The subject is taught in a book. How it is built is not: ask.
hanumemAI is proprietary. Its scores, token costs and model stacks are public. Its code, its run log and what each experiment changed are not.
The subject is not secret: why models forget, why the obvious fixes fail and what a memory layer must do are taught in AI Agent Memory: Novice to Ninja. How hanumemAI itself is built is not published. This article is the first of two steps; the paper is the second.
Step 2: the paper