Reliable Memory from Unstable Growth: Curating Self-Written Agent Playbooks with Randomized Experiments: what it means for business leaders

Agents that write their own playbooks cannot tell which lessons help, and RBC replaces that self-judgement with a randomized experiment that keeps only the blocks of lessons that measurably raise task success.

1. What the method is

Randomized Block Curation (RBC) is a procedure for deciding which parts of a language-model agent's self-written memory to put in its prompt. Systems such as ACE and Dynamic Cheatsheet let an agent write lessons after each task into a playbook and read the whole playbook later. RBC leaves that writing untouched and replaces the agent's own judgement about which lessons to keep with measurement. It clusters related lessons into blocks, includes or omits each block at random on practice tasks, and estimates every block's effect on task success from the gap between episodes with and without it. It keeps the blocks with a positive effect within a token budget, grows several playbooks because the same procedure can produce a helpful or a harmful one, and serves whichever curated set, whole playbook or empty playbook scores best on fresh practice episodes. Curation itself calls no language model.

2. Why the method was developed

The appeal of textual memory is improvement without retraining, which matters when the model sits behind a proprietary API. The paper shows the mechanism is unreliable as deployed. Served whole, an ACE playbook fell as much as 69.6 points below using no playbook, and regrowing it on the same tasks in a different order could flip a helpful playbook into a harmful one. In the four harmful AppWorld settings the damage came from one family of rules telling the agent to pass an answer to the task-ending call even when none was expected; the agent tagged one such rule helpful 123 times and never harmful. Pruning by the agent's own tags left the harm in three of four cases, because the agent only sees episodes where a lesson was followed, never the counterfactual. Prior context-valuation methods score single entries, which rarely move success beyond noise, and return scores rather than something to serve.

3. Who should care

Anyone in our organisation shipping or buying agents that accumulate memory across tasks should read this. Product and platform engineering own the direct implication: a memory feature that is on by default is a decision, not a safe accumulation, and needs an evaluation loop with observable outcomes. Quality and reliability teams gain a concrete method for a problem they currently cannot audit, since the agent's self-reported confidence does not track its actual effect. Procurement and vendor management should ask any supplier claiming self-improving agents how they verify that the learned memory helps on held-out tasks, and whether that verification happens per growth run. Finance should note the cost shape: RBC trades extra inference episodes and several growth runs for stability, so the compute bill rises in exchange for removing a tail risk of a 70-point regression that nobody would notice until customers did.

4. How the method works

After growth, RBC embeds each lesson with all-mpnet-base-v2 and clusters the embeddings with Ward's method into eight blocks, large enough that a block's presence moves success measurably. Each practice episode draws a task and includes each block independently with probability one half, so every episode doubles as one trial of a two-level factorial design. Because inclusion is random, the difference in mean success between episodes with and without a block is an unbiased estimate of its Banzhaf value, its average gain over whatever else is served alongside it, for all blocks at once. The paper fits this by ridge regression of success on the inclusion indicators with an unpenalised intercept per task to absorb difficulty. Curation keeps blocks with positive estimates, best first, until a budget of 40 percent of the playbook's tokens is filled; if none is positive it serves nothing. Three growth runs each yield a curated set, and a final round of fresh practice episodes picks among the curated sets, the whole playbooks and no playbook.

5. How it was evaluated

The authors tested on two memory systems, ACE and Dynamic Cheatsheet, with three open-weight models, Qwen3.8-27B, Qwen3.5-9B and Gemma-4-31B, across AppWorld, WebShop and ALFWorld, giving eleven ACE settings and eight Dynamic Cheatsheet settings, each with three growth runs. Training, practice and evaluation tasks were disjoint, with practice rounds of 600 to 684 episodes and evaluation over 168, 500 and 134 tasks respectively, three seeds each. Baselines all drew from the same playbooks and differed only in selection: no playbook, the whole playbook, cuts to the same 40 percent budget by the system's own tags or usage counts, by similarity as in ReasoningBank, or at random, and Best-Full, which picks the best whole playbook or none on practice tasks and so isolates what curation adds beyond choosing among runs. Constants were fixed on four ACE settings and left unchanged elsewhere. All models are open weights; proprietary frontier models were not tested.

6. How it performed

Across all nineteen settings RBC averaged 66.9 task success against 55.5 for the whole playbook, 54.1 for no playbook and 53.8 for the agent's own judgement, and beat both reference points on every model and system. On ACE it never fell more than 0.8 points below no playbook, while every baseline fell at least 5.4 below it somewhere. Against Best-Full it scored 67.2 versus 66.1 on ACE while serving 60 percent of tokens against 78, and the lead grew under long growth, where whole playbooks stopped helping and curation still found useful blocks. On Dynamic Cheatsheet, whose rewritten entries are hard to separate, RBC mostly fell back to choosing a whole playbook and matched Best-Full. The authors note the method needs practice tasks with observable outcomes that resemble deployment tasks, costs extra episodes, and was not tested on proprietary models or memory that grows while served. Our call is to pilot: any agent we run with accumulating memory should get a held-out measurement gate before that memory is served, and RBC is the cleanest design we have seen for one. (Source: arXiv 2610.18427, 2026)