Compaction, decay and reinforcement in a two-tier conversational memory: a paired evaluation against retrieval-augmented generation: what it means for business leaders

A paired test on 1,985 long-conversation questions shows that compacting chat history into decaying facts beats transcript retrieval on dates and values at a third less context, and loses where the answer is the speaker's own wording.

1. What the method is

NMAFC, short for Neuromorphic Memory Architecture for Conversational AI, is a memory for long-running assistants that stores compact facts instead of searching the raw transcript. A language model turns each conversation turn into atomic assertions, each naming an entity and stating one thing about it with a date and a speaker. Facts are deduplicated, linked and kept in two tiers: a working tier of dense vectors and a SQLite archive with a full-text index. Every fact carries a weight that decays exponentially with conversation turns and is restored whenever the fact is retrieved; when the weight falls below a threshold the fact is demoted to the archive, never deleted. The paper's second contribution is the test itself: a paired comparison against retrieval-augmented generation in which both systems answer every question in the same window, against the same stores, with the same generator and judge.

2. Why the method was developed

An assistant that talks to a customer for months cannot keep the whole transcript in the prompt, so something must be dropped, summarised or indexed. Retrieval-augmented generation keeps everything and searches at query time, but models lose accuracy on material in the middle of long contexts, and passages that are on topic without holding the answer make output worse. Compacting memories such as MemGPT, MemoryBank, Zep, A-MEM and Mem0 take the other route, yet their published comparisons run each system separately and subtract the totals. That cannot support a paired statistical test, and what compaction loses is hard to measure, because the extractor decides what counts as content before any question is asked. In this study the two systems agree on 1,473 of 1,985 questions, so all the evidence sits in the 512 where they differ. The authors' own earlier claim, from unpaired runs, that their system was several times slower turned out to be wrong.

3. Who should care

Product and engineering leads building assistants or support agents with long customer histories should read this first, because it tells them which questions each memory design answers well. Lookups whose answer is a value, such as when a contract renewed or which plan a customer is on, favour compaction; questions about what a customer actually said favour the transcript. Whoever owns our inference bill should note the 34 per cent cut in retrieved context per question, set against the paid extraction work at write time. Evaluation and data science leads get a protocol that costs one benchmarking run instead of two and separates real effects from noise. Anyone buying a memory product should ask the vendor for paired results that include the adversarial question category comparable published work drops. Compliance and quality teams should know that compacted facts keep what was asserted and drop the customer's wording.

4. How the method works

On the write path, a language model with a tool-calling schema extracts facts from each turn and resolves relative dates, so 'two weekends ago' is stored as an absolute date; it also drops qualifiers and phrasing. Consolidation removes duplicates and marks superseded facts invalid rather than deleting them. Each fact's weight starts at 1 and decays per turn at a rate set by its type: zero for core facts about the user, 0.05 for active context, 0.69 for ephemeral state. Each retrieval resets the weight and flattens the curve, so an active-context fact never retrieved leaves the working tier after 46 turns while one retrieved ten times lasts 206. On the read path, up to five ranked lists from dense, lexical and graph search across both tiers are merged by reciprocal rank fusion, the top 20 facts enter the prompt, and the raw source turns behind the top five are attached to recover lost detail.

5. How it was evaluated

The test bed is LoCoMo, ten synthetic multi-session conversations, with 1,985 questions: 841 open-domain, 446 adversarial, 321 temporal, 281 single-hop and 96 multi-hop. The baseline chunks the transcript, embeds it with the same embedding model, retrieves by similarity and passes the text to the same generator, DeepSeek-V4-Pro, with the same answer instruction. For every question the harness ran both systems back to back and wrote both answers to one row; stores were opened read-only so no question changed what the next one saw. A judge from a different model family, Kimi-K2.6, scored the answers without knowing which system produced them, and token-overlap F1 served as a judge-free check. Differences were tested with McNemar's exact test on the questions where the systems disagreed, with minimum detectable effects quoted for each sample size. The per-question predictions and analysis scripts are public.

6. How it performed

On the four categories comparable work reports, NMAFC scored 71.4 per cent against 64.6, a 6.8-point lead (p = 5.5e-08) with 963 retrieved tokens per question against 1,461. Across all five the lead is 2.0 points and not significant, because the baseline wins the adversarial category by 14.6. Temporal questions carry the gain at 71.7 against 46.1, earned by resolving dates at write time. On those four categories the lead shrinks as the expected answer gets wordier, 9.6 points ahead at one or two words and 3.1 behind past eight, and two prompt changes aimed at the gap gave back on temporal questions what they gained. Latency is indistinguishable through the ninetieth percentile. The authors' limits are a synthetic benchmark, ten conversations, one generator and judge, and run-to-run drift of about four points. Our call is to pilot: trial compaction beside transcript retrieval for date and value lookups, and adopt the paired protocol in our own evaluations. (Source: arXiv 2611.09807, 2026)