Chunk Recomputation under Real Document Revisions: Measuring Update Cost and Retrieval Quality of Fixed, Recursive, and Content-Defined Chunking: what it means for business leaders

When a document in a retrieval index is edited, the way it was cut into chunks decides how much unchanged text must be re-embedded, and fixed-size windows waste the most.

1. What the method is

The paper is a measurement study, not a new algorithm. The authors define an update cost for retrieval-augmented generation called the recompute fraction. When a document is revised and chunked again, it is the share of the new version's embedding-input tokens whose complete input is not byte-for-byte identical to an input produced from the previous version. They measure it on a bilingual benchmark of 120 pairs of adjacent Chinese and English Wikipedia revisions, with 240 questions whose evidence is tied to character offsets in each version. Three stateless chunkers are compared: non-overlapping fixed-size windows of 304 tokens, recursive paragraph-first chunking with a 400-token target, and content-defined chunking (CDC) through the Steadlith library with 176 to 352 token bounds. Each is scored on both update cost and evidence recall under a fixed retrieval budget. The point is that update cost is a second axis that static chunking benchmarks never show.

2. Why the method was developed

Chunking strategies are almost always tuned on a frozen snapshot of a corpus, yet the knowledge bases we build retrieval on are edited all the time. Every chunk is embedded on its own, so when an edit moves a chunk boundary, the chunks on either side become new inputs even though their words have not changed. In one English article from the benchmark, an edit touching 0.13% of the text caused fixed-size windows to re-embed 49% of the document, against 8% for paragraph-first chunking. Earlier work left gaps. Content-defined chunking came from file synchronisation and deduplication. Its text adaptation, Steadlith, reported reuse only on synthetic edits with hashed representations. KARA keeps the previous chunk layout as state and counts chunks rather than tokens. Dynamic vector indexes such as FreshDiskANN and SPFresh update vectors efficiently but do not decide which vectors need to change in the first place.

3. Who should care

The platform and ML infrastructure teams that run retrieval over living content are affected most: internal wikis, policy libraries, product documentation, support articles, contracts. They choose the chunker and pay for re-indexing, and this work gives them a metric to add to their evaluation harness. Whoever owns embedding spend, whether that is FinOps or the platform lead, should know that chunker choice can change re-embedding workload by tens of percentage points with no visible change in answer quality. Note that the paper measures model input, not dollars or latency. Search-quality owners get some reassurance: in this setup the cheaper chunkers did not visibly lose evidence. Teams indexing Chinese or other unsegmented text need to know that the CDC implementation failed outright on some articles. When we buy rather than build, procurement can ask vendors how they chunk and how much they re-embed when a document is edited.

4. How the method works

The mechanism being measured is an embedding cache keyed on the exact bytes of each chunk, including the model's special tokens. The cache holds only the previous version of the same document, and a chunk that differs by a single character counts as a miss. The recompute fraction sums tokens over the distinct missed inputs and divides by tokens over all distinct new-version inputs. A no-edit control must return zero for every document, and 20 hand-computed test cases checked the calculation. Why the methods differ comes down to boundary drift. An insertion shifts every fixed window that follows it. Paragraph-first chunking resynchronises at the next paragraph break. CDC picks cut points from local word-level Rabin fingerprints, but its minimum and maximum lengths can make it drift for several chunks after an edit. Retrieval uses a frozen BGE-M3 encoder with exact dot product, packing whole chunks into a 2048-token budget and scoring binary evidence recall.

5. How it was evaluated

The authors collected 149 adjacent revision pairs. Six were used to calibrate chunk lengths, 120 form the primary cohort (62 Chinese, 58 English), and 23 with unresolved source concerns were kept only for diagnostics. Each version has 1,024 to 20,000 tokens of body text. Edits were sorted into small, medium, large and extreme bands, using a word- or character-level difference proxy, before any method was run. Lengths were calibrated so that median chunk lengths sat within 15% across methods. The realised ratio came out at 1.17. Human annotators labelled 480 version-level answers, without independent adjudication, leaving 330 scorable labels on the 116 articles where all three chunkers succeeded. Differences were bootstrapped over articles with 2,000 resamples. Further checks were a cost-only grid of six chunk-length targets per method from 256 to 496 tokens, a cost-only check on 30 arXivEdits papers, and controlled 1% insertions at the start, middle and end of development articles.

6. How it performed

At the frozen settings, fixed-size chunking re-embedded 68% of new input in Chinese and 57% in English. Recursive chunking re-embedded 56% and 38%, and CDC 54% and 43%. Evidence recall stayed within two points across methods (English 96.7, 96.8 and 95.0%; Chinese 97.4, 98.3 and 98.3%). For small English edits the gap widened: 46% for fixed windows against 25% for recursive. Fixed windows were the most expensive at every length-matched setting tested. Whether recursive or CDC ranked cheaper flipped with chunk length and language. Overlapping fixed windows recomputed about the same, embedded 10 to 26% more input and lost 3 to 4 recall points in English. CDC failed on 4 Chinese articles without word segmentation. The authors' limits: one source, one encoder, one budget, unadjudicated labels, no KARA comparison, and workload rather than price. Our call: act now, by retiring fixed windows for edited corpora and adding the recompute fraction to our chunker evaluations. (Source: arXiv 2610.31019, 2026)