Enhancing Local and Global Structure in Compressed AI Agent Memory: what it means for business leaders

A new way to train the codebooks of a standard compressed vector index keeps far-apart memories in order, at the same bytes and search, so themes and deletions drift less while nearest-neighbor retrieval holds.

1. What the method is

Locality-aware quantization, LAQ, is a training method for the codebooks of an optimized product quantization (OPQ) index, a compressed vector store widely used for search at scale. It keeps the bytes per item, the PCA basis, the rotation and the search procedure exactly as OPQ has them and changes only the codebook values, so it replaces an OPQ index directly and adds no query-time operation. What is new is the target. Earlier quantizers train on reconstruction, nearest neighbors or query relevance; LAQ also trains on whether near and far items keep their order at 30 to 50 percent of the memory. It needs no labels or queries. The author claims it is the first quantizer whose objective comes from a proven bound on this far end, a claim we have not checked independently.

2. Why the method was developed

Compressed memories are accepted today on nearest-neighbor checks such as recall, yet agents also consolidate records into themes, delete a client's items and map how companies relate, and those operations compare each item with the whole memory. A recall check does not see that damage. At 96 bytes per passage on a financial memory, rescored binary codes reach a local fidelity, the chance-corrected recall at 10 neighbors, of 0.97 while moving 10.1 percent of passages to another theme; PCA with 8-bit codes scores only 0.77 and moves 1.7 percent. Rescoring repairs what a query sees but leaves the stored codes, and everything computed on them, unchanged. Private memories also have no labels to test against, and earlier learned quantizers either need labels or queries, or aim at nearest neighbors rather than the far end.

3. Who should care

Engineers who run vector stores and agent memory are the first audience, especially anyone on an OPQ index, because the swap changes no byte budget or query path and needs only a training job. Teams choosing compression should add a far-end score to their acceptance tests, since a recall number alone can approve codes that reshuffle themes. Product owners of features built on clustering, deduplication, summarized briefings or company maps should ask which vectors those features read. Legal and compliance should note that deletion by meaning, when metadata are incomplete, draws on the same far-end structure, so a compression choice can change what a removal request actually removes. Infrastructure and finance keep the same per-item cost, while buyers of managed vector services should ask vendors what they measure beyond recall, since rescoring will not cover this.

4. How the method works

The author proves that a compression cannot lose many neighbors at a given scale without reversing many cross pairs, where a cross pair is one near and one far item relative to an anchor. Across the 30 to 50 percent band, the neighbor loss is at most 2 to 2.18 times the square root of the violation rate. That count is a step function, so it is bounded by a hinge loss on distance gaps, which gradient descent can lower. Training starts from a fitted OPQ index and minimizes normalized reconstruction error plus hinges at a middle and a far group of scales, drawing half the pairs uniformly and half around the rank boundary. Hard assignments run forward and soft assignments backward for 1,000 steps. A 2,000-item monitor set picks the checkpoint with the best far-end score, provided reconstruction error stays within 1.15 times the start.

5. How it was evaluated

Two label-free scores are read from the vectors before and after compression: local fidelity, and global fidelity, the same statistic averaged over 30 to 50 percent of the memory. A measurement study applied 27 compressions in each of 18 runs across five memories and six encoders, comparing retrieval, theme consolidation and deletion against the uncompressed memory. These operations were modeled on the vectors directly, with no language model in the loop. LAQ was then tested on companies or conversations held out from training: FinQA, LoCoMo and Multi-Doc-2025 with three retrieval-tuned encoders, plus Amazon Titan V2 on Multi-Doc-2025, giving 40 runs, and 12 more on the FinDER financial memory, run after the recipe was fixed. Each run compares LAQ with its own OPQ start at 96 or 128 bytes. Baselines also included PQ, RQ, LSQ++, PCA with 8-bit codes, RaBitQ, rescored binary codes and ScaNN. A million MS MARCO passages tested scale.

6. How it performed

Global fidelity rose in all 52 held-out runs and local fidelity in 46; across the first 40 runs, where local fidelity did not rise it stayed within 0.0023 of OPQ. Gains are modest with retrieval-tuned encoders, 0.005 to 0.017, and larger with Titan V2, 0.032 to 0.038. At 128 bytes on one split, OPQ moved 8.3 percent of passages to another theme and LAQ 3.5, but over 16 runs the mean fell only from 2.8 to 2.1 percent. Evidence recall matched OPQ. Reconstruction training alone raised the far end in 0 of 52 runs, so the far hinge does the work. A million-passage training run took 175 seconds and reproduced bit for bit. Our copy of the paper cuts off before the later results and discussion, so we have not seen the author's stated limits, and this is single-author work built on the author's own measures. Our call: pilot LAQ wherever an OPQ index also serves clustering or deletion. (Source: arXiv 2611.08341, 2026)