Learning in a world of notes

Continual learning is about more than internalizing new knowledge — it’s about learning to leverage the memory systems at your disposal. Text memory, like notes, wikis, and skill files, is the main way today’s agents accumulate knowledge. But having good notes isn’t enough; you need to know how to use them well. Training allows agents to hone their judgement so that they know what their notes are useful for, which notes to read, and when to supplement notes with primary sources. This post studies how training and text memory multiply, making agents more intelligent, cost effective, and robust.

We study the interaction of text memory and training in a simple setting: we give a model a fixed set of high-quality notes and an optimized prompt, and then we train it with reinforcement learning. By learning to use their notes, models achieve substantial gains — anywhere from 7 to 20pp improvement in accuracy across our benchmarks. Trained models outperform both their untrained counterparts and frontier models with the same notes.

It also turns out that training costs roughly 5-30% as much as building a good text memory system. And because the trained models cost less per query at inference time than frontier models of similar accuracy, the cost of training is amortized over queries, paying for itself within days.

Finally, notes are dynamic — agents keep exploring and updating their notes with what they learn. Yet trained models turn out to be surprisingly robust to these changes. A model trained once on an initial version of its notes still beats an untrained one after 23M tokens of text edits. The model learns general strategies for using notes rather than overfitting to their contents, so it doesn’t need to be retrained after every edit.

Together, these results show that training compounds the benefits of text memory. Agents of the future won’t just read and write notes, they will learn to remember.

Setup

Figure 1:Experimental setup

Harness

Fixed during RL

Notes notes/

onboarding-notes.mdTurn 2Current docengineering-onboarding.mdPrevious docdeveloper-setup.mdSuperseded Sep 1, 2026Changed: Python/deps run in a container.Unchanged: local Git/editor; smoke-test suite.

Raw files files/

developer-setup.mdOutdatedOn your laptop:python -m venv .venv
.venv/bin/pip install -r requirements.txt

engineering-onboarding.mdTurn 5Open the repo in its development container.uv sync --frozen
uv run python --version
uv run pytest tests/smoke

RL training

I’m a new engineer. Which setup instructions should I follow, and which are outdated?

Agent

Update model weights

• …
• Turn 2NotesFind the current version and what changedRead notes/onboarding-notes.md
• …
• Turn 5Raw filesRead the setup and verification detailsRead engineering-onboarding.md
• …
• Final answer

  Use the plan from engineering-onboarding.md.

  The local Python steps in developer-setup.md are outdated.

  Keep Git/editor local; run setup and checks in the container.

Reward

Workspaces and tasks. We evaluate on three workspaces.

• Harvey C&H is a synthetic law firm introduced in our previous post, with about 100M tokens covering client matters, including emails, memos, legal documents, and spreadsheets. Tasks require searching across the firm’s prior work, such as finding antitrust matters that received regulatory clearance and locating the supporting documents. We report results on a 125-question subset of the benchmark.¹

• Hex Shorelane Commerce is a data warehouse of a synthetic office-supplies business with realistic inconsistencies in identifiers and definitions. We evaluate on DataBench², data analysis tasks that require querying the warehouse while navigating these inconsistencies. Examples include explaining trends across sales channels and deciding whether to approve increases in paid media spend based on historical data.

• Engram Internal is an October 2026 snapshot of our own Notion workspace, covering research, engineering, product planning, and customer conversations. Tasks require reconciling updates over time and grounding recommendations in the company’s work, such as tracing how team goals evolved across daily stand-ups.

Starting point. We use Qwen 3.8 27B on Harvey C&H and Engram Internal, and GLM-5.3-Flash on Hex. The agents run in the Pi harness with access to the workspace and its notes, and on Hex, the warehouse is reached through Hex’s MCP server rather than a filesystem.

Training compounds strong notes

Figure 2:Training with text notes improves accuracy and efficiency

Base modelBase + notesTrained with notes

Harvey C&H

Rollout tokens per query

Hex DataBench

Rollout tokens per query

Engram Internal

Rollout tokens per query

Rollout tokens per query

We first ask whether reinforcement learning adds anything on top of high-quality notes and a carefully optimized prompt. In each workspace, we compare three methods: the base model working from the raw workspace alone, the base model with notes, and models trained with notes at several values of the cost penalty λ.

Across all three workspaces, the pattern is the same: notes help, and training adds a clear gain on top. On Harvey C&H, the notes already do most of the work, but training still raises accuracy to 80%. The gain is largest on Hex, where notes alone help only modestly and training lifts accuracy by 15 points. These gains don’t have to come at a higher inference cost: with a moderate efficiency penalty, the trained model beats the notes-only baseline on both accuracy and token use in every workspace. More broadly, across penalty settings, the trained models form a frontier above the training-free baselines.

Two ablations round out the picture (Appendix Table 2 and Figure 7). As a control, we train without notes. This still improves on the base model, but falls short of training with notes, so some of training’s gains depend on having notes to work with. Training also helps on top of notes from an alternative, more prose-heavy recipe, so its gains aren’t tied to one way of building notes.

Comparison to frontier models

Figure 3:Trained models compete with the frontier

HarveyHexEngram Internal

Frontier modelFrontier + notesTrained model with notes

The cost of training

Figure 4:Training cost is amortized over inference

HarveyHexEngram Internal

Trained modelQwen 3.8-27B · λ = 0.05 (77.84% accuracy)

Performance-matched frontier modelGPT-6 Sol medium with notes (77.92% accuracy)

Cumulative cost

20 users × 20 queries/day

Notes cost: $1,894.49Training cost: $372.57

Trained models often match frontier models given the same notes, at a much lower cost per query. But training adds an upfront cost that using a frontier model doesn’t. When does that cost pay off?

Because training reduces the cost-per-query at inference time, the trained model pays for itself quickly. Compared with the cheapest frontier model with similar accuracy, it breaks even within days in every workspace—from under a day on Engram Internal to about 9 days on Harvey C&H, assuming a modest workload of 20 users sending 20 queries a day. Beyond that point, using the trained model is cheaper end-to-end than using a frontier model at the same accuracy.

Training endures in a changing world

Figure 5:Trained models are robust to changing notes

Base modelInitial trainingReused modelRetraining

So far, we've investigated a simple setting where we train models to operate with a fixed set of notes. In practice, the world changes. As models work, notes are added, reorganized, and updated with new context. How often would we need to retrain the model? Do models trained to use notes remain effective as those notes change?

To test this, we simulate an agent exploring a workspace and writing notes over time. On Harvey C&H, the note-building agent starts with access to half of the firm’s client matters, and we reveal the rest in stages until it has seen the full workspace. At each stage, it updates the notes with what it finds. By the end, the notes have changed by about 23M tokens, more than three times their original size.

We train a model once on the initial set of notes, built over half of the workspace. As the notes are updated with more context, we reuse the trained model along with the updated notes, and compare it with the base model given the same notes.

The gains from training are surprisingly robust. Even as the updated notes lift the base model’s performance substantially, the trained model beats it at every stage, even after 23M tokens of edits. The reused model’s accuracy also rises as the notes improve even though it was trained on an earlier version, and retraining on the final notes does only slightly better. This suggests that a model learns generalizable strategies, so it doesn’t need retraining after every edit.

Training teaches strategies for using notes

What is the trained model actually learning? We first look at what the trained model carries over as the notes change in the experiment above. One simple behavior: the trained model learned to check what its notes are missing, comparing the matters they list against the workspace while the base model often takes the notes at face value. It doesn’t run this check on every query; it only does so after noticing something suspicious, such as a lookup returning fewer entries than expected (Figure 8, and the trace below). It’s a small habit about how to use notes, not what’s in them, and likely one reason the trained model keeps its advantage as the notes change.

Figure 6:Training teaches strategies for using notes

“What's our latest White Collar & Investigations matter where a litigation hold was actually put in place?”

Base + notes

Accuracy 0% · 64K total tokens

1. Turns 1–2Notes

   Checks the practice areas in the notes. White Collar is not listed.

   Notes excerpt

   matter_metadata.jsonl · practice areas

   Antitrust · Banking & Finance · … Regulatory & Compliance · … Trusts & Estates

   no “White Collar & Investigations”

2. Turn 7Raw documents

   Lists every file that mentions a litigation hold.

   Command & output

   rg -il "litigation hold" matters/

   <19,749 tokens of file paths>

3. Turns 10–21Picks

   Settles on the closest practice the notes list—not White Collar—and reads the hold memos of its candidates.

   Notes excerpt

   matter_metadata.jsonl

   1045-00007 · “practice_area”: “Regulatory & Compliance”

Answer

matter 1045-00007, an SEC enforcement investigation filed under Regulatory & Compliance. The right matter, 1032-00007, is not in these notes at all; they cover only 133 of the 266 matters.

Training + notes

Accuracy 100% · 52K total tokens

1. Turn 1Notes

   Checks the practice areas in the notes. White Collar is not listed.

   Notes excerpt

   matter_metadata.jsonl

   18 Mergers & Acquisitions · 17 Litigation · … 11 Regulatory & Compliance · …

2. Turn 11Raw documents

   Counts which matter folders mention “white collar”.

   Command & output

   grep -rl -i "white collar" … | uniq -c

   66 1016-00009 · 30 1034-00001 · 22 1039-00005 · … 1 1032-00007

3. Turns 13–14Gap check

   Notices missing matters. Compares the notes’ matter list with the folders.

   “These matter IDs don't exist in matter_metadata.jsonl? But the directories do exist.”

   133 meta_ids.txt · 266 dir_ids.txt; then the 133 folder IDs with no notes row

   Comparison command

   comm -23 /tmp/dir_ids.txt /tmp/meta_ids.txt

4. Turn 34Raw documents

   Falls back to raw documents to find an unlisted matter with a confirmed litigation hold.

   Source excerpt

   1032-00007 · internal-investigation-plan-memo.md

   “Litigation hold issued to all custodians and IT systems | October 12, 2025 (completed)”

Answer

matter 1032-00007, an export-control investigation handled by the White Collar & Investigations group; the hold notice was issued October 12, 2025.

We also compare the behaviors of the base model with notes versus the trained model (Figures and example traces are in Trained model behaviors and traces):

• Harvey C&H: the trained model looks things up in the notes more often (26% more calls), but each lookup returns about a third less (36% fewer tokens). These narrower lookups get it what it needs cheaply, while the base model sometimes issues broad queries whose output gets cut off, leading to wrong answers (see the Harvey tab of Figure 10).
• Hex: the trained model makes more notes lookups per question (9.2 vs. 6.8), but each returns less text. These focused searches can find details that broad reads of the same notes miss (see the Hex tab of Figure 10).
• Engram Internal: the number of lookups stays about the same, but each returns less (19% fewer tokens), and more of them follow links from notes to their source files instead of directly searching across both.

What’s next

In this post, we investigated how training can amplify the benefits of text memory. Across three workspaces, training consistently improves a model’s ability to use its notes beyond what prompting alone achieves. That training is also cheap relative to building the notes, paying for itself within days. And a model trained once keeps its advantage as its notes change, even after 23M tokens of edits.

There are many interesting questions to explore from here:

• Learning to construct and use notes: In this blog post, we simplified the setup by first building notes and then training the model, but these need not be distinct stages. Both steps spend significant effort reading and exploring the workspace, and it’s natural to pair them. Several academic works have begun to study this setting^{9, 10}. In doing so, we can learn not only to use the notes, but to develop the notes as well.

• Beyond RL: Likewise, we kept the training recipe as simple as possible in this post, but there are several directions to elaborate upon this baseline, such as other RL algorithms and on-policy (self-)distillation^{11, 12}.

• More flexible tools and harnesses: We’re also developing a wider range of memory systems and harness improvements since different workspaces and use cases may benefit from different kinds of tools^{13, 14}. Our goal is to build agentic systems that work across a wide range of settings, adapting to changing environments by learning to use the tools at their disposal.

Continual learning has many facets, but at its core it is about adapting to complex and changing worlds. Learning to leverage memory systems — learning to remember — is central to this goal. That’s what we’re working toward here at Engram.

Appendix

Training without notes

We report the performance of {notes, no notes} x {trained model, base model} across our three workspaces. We see that training a model without notes—that is, using RL to better navigate the workspace only—offers improvements over the base model without notes. However, it still falls short of training on top of the notes for both Harvey and Hex, and is comparable on Engram Internal.

Training across different notes recipes

Figure 7:Training boosts performance for various text memory systems

Harvey C&H

Our notes recipeAlternative notes recipe

BaseBase + notesTrained

These days, everyone has their own favorite flavor of harness and text memory. We’ve tried many different recipes and chose one that works well across workspaces, but a natural question is whether training’s gains depend on that choice.

To find out, we train a model on top of notes from an alternative recipe for Harvey C&H. Both recipes produce notes that cover similar content, but the alternative notes rely more on prose than on structured data.

Training improves accuracy over the untrained agent using the same notes at every cost penalty. We also see that the trained model with the alternative notes still trails behind the trained model with our notes. This suggests that training doesn’t depend on one particular way of building notes, and that better notes and training add to each other.

Trained model behaviors and traces

Figure 8:Trained models learn to check notes against the raw documents

Harvey C&H

Base model with notesTrained model with notesShare of matters missing (reference)

Checking the notes for gaps. Here we take a closer look at the gap-checking behavior from the earlier section: given notes that cover only part of the workspace, the trained model checks which matters the notes are missing and searches those directly. Figure 8 shows this is a systematic difference between the two models. At each stage of the changing-notes experiment, we count how often each model runs the check. The base model almost never does. The trained model does, and how often it checks tracks how much is missing from the notes. This supports our qualitative observation that the check is triggered by noticing matters missing from the notes rather than run on every query.

Figure 9:Trained models use more efficient memory lookup

HarveyHexEngram Internal

Notes output per lookup

Base model with notesTrained model with notes

General behaviors. Figure 9 plots the distribution of the number of tokens per lookup in the notes. On Harvey C&H, the trained model’s lookups shift dramatically towards shorter outputs; for Engram Internal, there is a slight increase in shorter outputs as well. Hex shows the biggest drop, from 567 to 166 tokens per lookup in a typical trace. Smaller results cost less and fail in fewer ways (Figure 10). A broad query can get cut off, so the agent misses part of the answer (Harvey tab). Or it can bury the one detail that matters: on Hex, the base model reads whole pages and 12K tokens of notes but misses where the subscription delays happen, while the trained model's focused search finds it and answers correctly with 30% fewer tokens (Hex tab).

Figure 10:Example traces from Harvey C&H and Hex DataBench

HarveyHex

“Pull the matters filed to our Real Estate practice where title insurance was in place.”

Base + notes

Accuracy 43% · 98K total tokens

1. Turn 1Notes

   Pulls the full notes row of every Real Estate matter at once. The output is too long, and the tool cuts it off.

   Command & output

   jq -c 'select(.practice_area | test("Real Estate"; "i"))' matter_metadata.jsonl | head -30

   16,308 tokens, ending “[Showing lines 4-8 of 8 (50.0KB limit) ...]”

2. Turn 2Reasoning

   Works only from the part it saw.

   “I found four Real Estate matters: 1011-00007 ... 1026-00004 ... 1027-00004 ... 1030-00003”

3. Turns 10–15Raw documents

   Reads the closing documents of the four it saw in full, 6K to 11K tokens a file.

Answer

two matters, 1026-00004 and 1027-00004. It never gets the notes rows of the other seven Real Estate matters, so it misses three that qualify.

Training + notes

Accuracy 86% · 61K total tokens

1. Turn 5Notes

   Asks the notes for just the IDs of the Real Estate matters.

   Command & output

   jq -r 'select(.practice_area == "Real Estate") | .matter_id' matter_metadata.jsonl

   1036-00004 · 1043-00002 · 1031-00004 · ... all 11 IDs (121 tokens)

2. Turns 9–16Raw documents

   Searches each matter's closing documents for the title-insurance lines.

   Closing-document search

   grep -n -i "title insurance\|title policy\|title premium" 1031-00004/Closing/funds-flow-memo.md

   “Title Insurance — Senior ALTA Loan Policy ...”

   (355 tokens)

   Post-signing search

   grep -rn "title" 1017-00009/Post-Signing/ | head -10

   “Title. A leasehold title insurance policy was issued by Commonwealth Fidelity Title Insurance Co. ...”

   (719 tokens)

Answer

five matters, including the three the base model missed (1036-00004, 1017-00009, 1031-00004). Both models miss one (1043-00003).