Are You Sure Your LLM Is Sure? Repairing and Auditing Confidence: what it means for business leaders

A workflow from TCS Research and MBZUAI that recalibrates an LLM confidence score without target labels, admits the fix only when held-out evidence certifies it, and then tests for the systematic errors the score still cannot see.

1. What the method is

RADAR is a three-stage procedure for the confidence score that decides whether an LLM answer is accepted, sent to review, or regenerated. The repair stage, called Probe-Anchored Isotonic Calibration or PAIC, fits a monotone remapping of the existing score using the predictions of a separate correctness probe on unlabeled prompts from the deployment population, so no target labels are needed to fit. The adoption stage is a paired test on held-out labeled prompts that certifies a candidate score only when it lowers squared loss, with a finite-sample bound on the chance of adopting a non-improvement. The audit stage is a kernel test over residual errors, clustered by prompt so that several answers to the same question are not treated as independent evidence. The theoretical backbone is a decomposition of squared risk into outcome noise, information hidden by the score, mapping error, and a pooling gap introduced by preserving score order.

2. Why the method was developed

Standard recalibration methods such as temperature scaling, Platt scaling and isotonic regression fix the numbers a score reports, but they need labeled data from the population where the model is deployed, which under distribution shift we often do not have, and they can only correct errors visible through the score itself. The authors give the sharp example: if every answer carries the same confidence and a fixed fraction of them systematically confuse minutes with hours, expected calibration error can be exactly zero while the failure remains perfectly predictable. Correcting a score and recovering distinctions the score hides are different problems, and earlier work tended to conflate them. The paper also observes that agreement with a probe cannot settle whether a repair helps, since a probe agrees with itself whether or not it is right, and that answer-level calibration tests reject correctly calibrated scores whenever sibling answers share mistakes.

3. Who should care

Anyone in our organisation who gates LLM output on a confidence threshold is touched by this. Platform and MLOps teams own the serving interface where a single scalar drives triage queues and auto-accept rules; PAIC fits that interface because it serves one monotone map of the existing score and keeps its validated ranking. Model evaluation and risk functions gain a defensible answer to the question of whether a recalibration should ship, since the adoption test replaces eyeballing a reliability diagram with a certified one-sided comparison. Compliance and audit owners get the diagnostic stage, which names the features along which residual errors hide. Product owners of review workflows should note the paper's finding that lower prediction loss does not guarantee cheaper review decisions. Buyers of hosted models, who may see token log-probabilities but never hidden states, are the population the scalar-map constraint was designed for.

4. How the method works

PAIC trains a correctness probe on labeled source prompts, then on unlabeled target prompts fits an isotonic regression of the probe's predictions against the original score, weighting each prompt equally regardless of how many answers were sampled. The authors prove that PAIC's excess risk is bounded by the probe's score-conditional bias plus a fitting term, not by the probe's overall accuracy: errors that cancel within a score level are harmless, while a constant offset persists no matter how many calibration prompts are used. For adoption, frozen candidate scores are compared on a held-out labeled pool, optionally sharpened with prediction-powered inference, and a betting-based test certifies a candidate when its wealth process crosses a threshold set by the error budget. For auditing, residual features are averaged within each prompt and kernel products are taken only across different prompts, with a multiplier bootstrap at the prompt level providing the null distribution.

5. How it was evaluated

Experiments ran on Llama 3.2 3B, Qwen 2.5 3B and 7B, Mistral 7B and a 32B extension, across BoolQ with 3,270 validation questions, ARC Challenge with 1,172 test questions, MATH with transfer to 777 unlabeled level-3 prompts, and constructed inventory-tracking problems with controlled presentation changes. Shift studies used 500 labeled source prompts and 500 unlabeled target prompts and compared PAIC against source-fitted label-isotonic, Platt and beta maps, importance-weighted isotonic, source-distilled isotonic, group maps in the style of Chen et al., direct probes served with their features, and a target-label isotonic yardstick. Thirteen predictors were compared on the fixed-choice tasks. Adoption and audit procedures were tested on 28 constructed settings with 2,000 repetitions each and on 47,400 cached raw-score allocations. Loss throughout is squared loss, and all guarantees assume sibling-preserving splits with fitted components frozen before comparison.

6. How it performed

Under difficulty, task and subject shifts PAIC had lower mean risk than every source-fitted scalar baseline in all six settings and won 67 of 72 prespecified contrasts, 47 of them after global adjustment, though direct probes beat it wherever their features could be served. On MATH transfer Llama improved while Qwen did not, and on presentation-change controls PAIC lost to a label-fitted comparator in four of six contrasts. Betting adoption picked a candidate with lower held-out risk in 42.95 percent of 47,400 allocations against 15.09 percent for one-sided empirical Bernstein, with no false adoption observed. The learned auditor rejected 126 of 2,000 graded-null repetitions, above nominal, which the authors flag rather than explain. The paper also shows repaired scores that reviewed every answer at some cost settings. Code is not yet released. Our call: pilot the adoption test on one gated workflow now, and treat PAIC as a candidate rather than a default. (Source: arXiv 2610.22139, 2026)