PRISM: Calibrating Black-Box Judges with Counterfactual Probes: what it means for business leaders

A rubric-free way to make a black-box model judge agree better with human raters: measure how its score moves when told to weigh an attribute more or less, then regress that movement onto human labels.

1. What the method is

PRISM, short for Probe-based Residual Inference of Scoring Misalignment, is a method for correcting the scores of a language-model judge so they agree better with human raters, without opening the model or asking it to rate individual criteria. The judge scores a piece of text once as normal, then scores the same text again under instructions to weight a named attribute, such as fluency or length, more heavily or less heavily than usual. The differences between those probed scores and the plain score form a row of sensitivity features the authors call a surrogate Jacobian. A ridge regression fitted on human-labelled examples learns one coefficient per probe for that judge. At inference the correction is the weighted sum of the measured responses, subtracted from the plain score. Only overall scores are ever requested, so no rubric has to be written or maintained.

2. Why the method was developed

We use model judges because human evaluation is slow and expensive, yet judges have known tendencies: they reward verbosity, favour certain answer positions, prefer their own generations, and on subjective tasks can agree with each other while diverging from people. The standard remedy is a rubric: define criteria, ask the judge to rate each one, and either average the ratings, as G-Eval does, or fit a model on them to human labels, as SAJA does. Rubrics must be written and maintained per task, and recent work finds model-generated rubrics unreliable. The authors asked a different question: whether a judge's reaction to being told to care more or less about something reveals the biases already present in its unprompted score. If it does, a correction can be learned from those reactions alone, with no criterion-level ratings and a probe bank that need not reproduce the rubric at all.

3. Who should care

This touches anyone in our business who scores generated text at scale with a model rather than a person. Our evaluation and quality teams can use it to tighten agreement between an off-the-shelf judge and our own raters using labels we already collect. Teams that train or fine-tune with model feedback, including reward-model and preference-data work, should note that a biased judge feeds biased signal into training, and a learned residual correction is a cheap layer to add. Product owners who buy a judge as a service should understand that the method works on black-box APIs and never needs weights. Platform and finance leads should expect inference cost to rise, because each scored item needs probed scores on top of the baseline. Research leads can read the fitted coefficients as a diagnostic of which attributes a given judge over- or under-weights relative to our raters.

4. How the method works

For each attribute in a probe bank, two prompts keep the content, task, scale and output format fixed and add one sentence telling the judge to weight that attribute more, or less, heavily than it normally would. The judge's score under each prompt minus its baseline score gives two oriented responses per attribute, kept as separate columns rather than collapsed into one difference, which preserves asymmetric reactions. One-sided context probes, such as scoring for a particular audience, contribute a single column. Scores and responses are centred within each group of candidates that share a source. Ridge regression on human labels fits shared coefficients, with the penalty chosen by grouped cross-validation and the baseline coefficient fixed at one, so an item with no probe response keeps its score; a variant learns that coefficient too. Diagnostics showed every attribute in a 14-probe bank moved scores above repeat-to-repeat noise, and that averaging eight candidate orders stabilises batched scoring.

5. How it was evaluated

The main comparison used two public benchmarks: SummEval, 100 news articles with 16 summaries each rated by experts on coherence, consistency, fluency and relevance, and HANNA, 96 story prompts with 11 stories each rated by three people on six criteria, with the human target averaged across criteria. The judges were Jev 1.13, a typed-probability decision model, and Sol, with Claude Sonnet 4.5 and Gemma 4 31B reported in the appendix. Baselines were the uncalibrated judge, SAJA with broad and granular rubrics fitted by random forest, and averaged G-Eval criterion scores. The metric is Spearman rank correlation within each document or prompt, averaged across contexts, using five held-out folds and 2,000-sample bootstrap intervals. Four further benchmarks, Topical-Chat, HelpSteer2, MT-Bench and JudgeBench, were scored with Jev on pairwise accuracy. The authors note that query and token budgets were not matched across methods, and that SummEval was also used during development.

6. How it performed

On all four main settings, both PRISM and its learned-baseline variant significantly improved rank agreement over the uncalibrated judge by paired bootstrap intervals, and were competitive with the rubric-based methods, with the best method varying by judge and task; the table values are not legible in the text available to us, so we cannot quote the correlations. Swapping rubric-derived quality probes for style probes such as length and extractiveness gave similar agreement, and combining both added little. On the four additional benchmarks the uncalibrated judge already beat SAJA, and PRISM's gains were small with intervals including zero. Fitted coefficients differed across judges: Jev and Sol both corrected fluency the same way but accuracy and grounding in opposite directions. The authors flag the added probe-call cost, untested transfer across tasks, model versions and rater pools, and possible benchmark contamination. Our call: pilot it on one judge where we already hold human labels. (Source: arXiv 2610.18389, 2026)