Copies Are Not Confirmations: How News Duplication Fools LLM Judges and Continual Learners: what it means for business leaders

Roughly one news article in five is a copy of another outlet's story, small LLM judges count those copies as confirmations, and collapsing each copy chain to one report before judging measurably reduces the falsehoods a learning system accepts.

1. What the method is

Copies Are Not Confirmations is a measurement study with a defence attached. It shows that much of the news is the same story republished, that LLM judges treat those reprints as separate confirmations of a claim, and that a system that keeps learning from a news stream behind such a judge admits more falsehoods as a result. In a sample of 200,000 English articles from CC-News, a near-duplicate detector links 18.9% to a near-copy at another outlet. The defence is to change the evidence, not the model: group detected copies into chains and show the judge only the earliest report from each chain. A second detector, an entity-novelty test, catches copies that a language model has reworded, which text similarity misses entirely. The authors are explicit that this measures textual dependence and establishes neither true source independence nor truth.

2. Why the method was developed

Any system that updates itself from incoming text has to decide which claims to believe, and the obvious heuristic is that a claim reported many times is better supported. News breaks that heuristic. Wire stories, content shared across one owner's titles, syndicated features and press releases put one source's claim on dozens of sites, and journalists count that as one source. Earlier research showed that repetition shifts what language models believe, both in training and in the prompt, but it did not separate copies from independent sources and did not test a defence inside a learning pipeline. The authors' own previous paper found that a continual learner is only as safe as the judge approving its updates and proposed corroboration across independent chains as future work. This paper tests that proposal, and finds simple counting rules fail in both directions.

3. Who should care

This touches every team of ours that lets a model weigh retrieved or streamed text. Data and platform engineers running ingestion, retrieval or memory pipelines over news and web content should read it as a statement that deduplication is an evidence-counting step, not only a storage saving. Teams that use LLMs as judges, for fact checks, market and competitive intelligence, or automated knowledge-base updates, should know that small local models can be talked into a claim by three reprints. Security should treat one origin plus ten copies as a cheap poisoning attack on any pipeline that learns from public text. Whoever selects models should note that the judges behaved very differently: provenance metadata in the prompt helped one of four reliably. Product and legal owners of customer-facing answers inherit the residual risk, because coordinated sources still get through.

4. How the method works

The copy detector is standard near-duplicate detection. Each article's first 2,000 characters are cut into five-word shingles and hashed with MinHash using 64 permutations; locality-sensitive hashing finds pairs from different outlets with estimated Jaccard similarity of at least 0.6, and linked pairs are joined into chains. Lines that occur in at least 20 articles are stripped first, because shared boilerplate such as paywall and login text caused every false match in the labelled sample. Before judging, the pipeline shows the judge the earliest report from each chain, at most ten. For reworded copies, the entity-novelty test extracts named entities with spaCy, plus quoted passages, from the first 2,500 characters and computes the share of the later report's items absent from an earlier look-alike. A score below 0.4, a threshold fixed before the first run, marks a copy, on the reasoning that rewrites keep names and numbers while independent reporting adds new ones.

5. How it was evaluated

The evaluation has three parts. Copying was measured on a random sample of 200,000 English articles from 6,877 outlets in CC-News, 2017 to 2019, with 300 labelled pairs and recall tested on 200 edited reprints and 200 rewrites each by phi4, Llama 3.2 and Gemini 2.5 Flash. Judge behaviour was tested on 320 claims, including 80 lies about invented people, shown to four judges, Llama 3.2 3B, phi4 14B, Gemini 2.5 Flash and Gemini 2.5 Pro, with one to ten reports, differently worded reports and provenance notes. The learning study fine-tunes a Qwen2.5 1.5B student with a LoRA adapter over a simulated 30-day stream holding 400 truths, 100 legitimate updates and 171 lies under copy, paraphrase and collusion attacks, mostly across three seeds. Detector errors are drawn at measured rates; the detectors are not run on the stream's text. Baselines are counting rules, a simulated judge with 0.9 accuracy, TF-IDF, embedding models and cross-encoders.

6. How it performed

Three identical reprints raised the share of lies about unknown people accepted from 0.06 to 0.71 for Llama 3.2 and from 0.21 to 0.90 for phi4, and both ignored a note that the reports shared one source. Gemini 2.5 Pro obeyed the note but then rejected true claims too, and Gemini Flash accepted 93% of lies from a single report. Showing one report per chain cut accepted copied lies by 0.22 to 0.24 for the local judges, 0.16 for Pro and 0.10 for Flash, costing at most 4.2 points of accepted truths on average. Text similarity caught 92% of edited reprints and no LLM rewrites; the entity test cut reworded lies accepted by phi4 from 0.67 to 0.20, or 0.40 against a rewriter inventing details. Colluding sources pass untouched, the stream is simulated, and nothing was validated on live news. Our call: pilot copy collapse ahead of any LLM judge in our ingestion pipelines, and watch the rest. (Source: arXiv 2611.09203, 2026)