On the Risks of using LLM-Generated Tests for Regression Testing: what it means for business leaders

A study of 145 pull requests in SciPy, Qiskit and pandas finds that LLM-generated regression tests lock in bugs three to six times more often than they catch them, and those locked-in bugs almost never surface later.

1. What the method is

The paper is an empirical study, not a tool. It examines what happens when a large language model writes regression tests for a code change that happens to contain a bug. The authors name the central failure a fault-enforcing test: a test that passes on the buggy code and fails once the bug is fixed, so it protects the defect rather than the intended behavior. They contrast it with a fault-revealing test, which fails on the bug and passes on the correct code, and with weak tests that pass on both and invalid tests that fail on both. Using this four-way classification, they measure how often each kind appears when a coverage-guided, pull-request-aware generator built on Claude Haiku 4.5 writes tests, and then follow the fault-enforcing ones through months of later changes in three mature open-source Python projects.

2. Why the method was developed

Writing regression tests is one of the most expensive parts of software maintenance, and LLM tools now promise to generate them automatically from a pull request, with vendors and papers reporting large coverage gains. The authors point out a design flaw shared by these tools. A developer whose new test fails asks whether the code or the test is wrong. The generators instead assume the code is correct and repair or discard any failing test, so a bug in the change is never surfaced. Because the model infers expected values from what the code does rather than what it should do, a buggy implementation produces a test that specifies the bug. The paper was written to quantify that risk in realistic project histories, and to check whether such tests get corrected over time or quietly accumulate into a new kind of technical debt.

3. Who should care

Engineering leaders at any organization adopting AI coding assistants or automated test generation should care most, since the finding bears directly on whether a green test suite can be trusted after an AI-assisted merge. Heads of quality assurance and platform teams who set merge policies need to know that a passing generated test is not evidence of correctness. Vendors building code-generation and test-generation products are addressed explicitly, with concrete recommendations on how to redesign acceptance criteria. Maintainers of scientific, financial and data-processing libraries, where the studied projects sit, face the same exposure. Anyone running a vibe-coding workflow in which developers accept whatever the model produces is the population the study simulates, and the results suggest where review effort should be concentrated.

4. How the method works

The authors took merged pull requests from SciPy, Qiskit and pandas and injected 60,780 small artificial faults, such as flipped comparison operators or changed constants, into the modified functions. They kept only faults that the project's own test suite failed to catch and that some test could in principle detect, which left 3,483, 3,446 and 10,609 mutants respectively. For each, the model saw the pre-change code, the buggy post-change code and the diff, never the correct version, and wrote a test targeting the changed lines with up to six rounds of execution feedback. Each test was then run on both the buggy and the correct code and classified. Fault-enforcing tests were carried forward through every later pull request to see whether they kept accepting the bug, and in a separate stress test, one fault per later pull request was piled into the same codebase to measure whether a dozen simultaneous bugs changed any test's verdict.

5. How it was evaluated

The evaluation covers 145 pull requests, 50 each from SciPy and Qiskit and all 45 eligible from pandas, each run in an isolated container. Three questions structure it: how often generated tests enforce versus reveal faults, whether enforcing tests persist across later commits, and whether accumulated faults remain enforced and escape the developer-written suite. Persistence used 2,480 fault-enforcing test-fault pairs followed through up to 49 later pull requests and up to 14 months, with a Kaplan-Meier survival estimate to handle unequal observation lengths. Accumulation ran 246 randomized trials. Confidence intervals come from bootstraps that resample whole pull requests, and the authors state limits: faults are mutants rather than real bugs, the fault-revealing rate is likely optimistic, only one model and three Python projects were studied, and the accumulation test is a deliberately adverse scenario rather than a model of real fault arrival.

6. How it performed

Fault-enforcing tests made up 16.9 percent of generated tests in SciPy, 8.4 percent in Qiskit and 16.9 percent in pandas, against fault-revealing shares of only 2.6, 2.4 and 4.8 percent, a ratio of 3.5 to 6.5 that held in every project and for every mutation operator. In 92 of 145 pull requests at least one injected bug was locked in. Most remaining tests were weak or invalid. Over the later history, 82.8 to 90.9 percent of enforcing tests still accepted their fault at the end, and among tests whose code was not rewritten, 99.5 to 100 percent survived. Developer tests caught the fault for 36 percent of SciPy pairs but only 6 to 8 percent in Qiskit and pandas. With up to 24 simultaneous faults, other bugs changed a test's verdict in at most 0.4 percent of evaluations, and the project suites confirmed detection of just 14 to 30 percent of accumulated faults. (Source: arXiv 2610.11835, 2026)