What Makes a Social Agent's Interactions Effective? Learning from Verifiable Outcomes in Paired Worlds: what it means for business leaders

A benchmark that checks whether the plans conversational agents agree on actually work shows that highly rated conversations often fail, and a paired-world training method turns those checks into a learning signal.

1. What the method is

The paper contributes two things. HICO-Bench is a benchmark of 180 everyday coordination scenarios across six families, such as splitting a bill, approving a request, agreeing a quantity, or booking a shared resource. Two agents with private information talk, then each independently writes down the plan it believes was agreed. A program checks Feasibility, meaning the two records match, the outcome kind is right, and the terms satisfy both parties' private conditions. An LLM judge checks Integrity, meaning the conversation actually supports the record. Passing both is a Verified Success. Every scenario comes in two worlds that share the public context but differ in one private fact, chosen so that no plan is correct in both. Paired-Credit Policy Optimization, PaCPO, is a reinforcement learning method that turns the end-of-conversation verdict into per-turn credit by checking tentative plans across the two worlds. It lifts Qwen3-8B's Verified Success from 20.4% to 28.3%.

2. Why the method was developed

Most evaluations of social and conversational agents score the transcript. SOTOPIA and its successors ask an LLM judge to rate goal completion and believability, and several training methods optimize those ratings. A rating does not check whether the plan the agents settled on works, and the authors find that, pooled across thirteen models, 36% of conversations the judge scores highly for goal completion fail Verified Success. Negotiation and tool-use benchmarks do verify outcomes by rule, but in fixed domains where the decisive facts are given up front rather than elicited in conversation. On the training side, a verified outcome arrives only after the last turn, so the learner cannot tell which turn earned it. Step-level credit methods such as GiGPO and GraphGPO need the same state to recur across rollouts, which rarely happens in free-form dialogue, and judge-distributed rewards inherit the judge's noise. The paired worlds give a verifiable signal about which turns mattered.

3. Who should care

This touches anyone whose agents commit to something on a user's behalf: scheduling assistants, approval and authorization flows, procurement and ordering bots, customer service agents that agree refunds or bookings. Product and engineering leads should take the headline finding seriously: a goal-completion score from an LLM judge, whether a vendor's or our own, can rank models in a different order from whether the agreed plan is actually valid. QA and evaluation teams get a concrete recipe, independent structured records from both sides plus a rule checker, that we can apply to our own transcripts without any training. ML teams doing reinforcement learning post-training on dialogue get a credit-assignment technique for sparse terminal rewards. Risk and compliance should note the Integrity failures: agents inventing authority they lack, or recording a plan that contradicts what was said, are exactly the behaviours that cause disputes with customers.

4. How the method works

Each scenario has a public context, a private card per role, and a verifier holding both roles' conditions. The tested policy is rolled out in both worlds against a frozen partner, with one terminal reward: full credit for Verified Success, partial credit if only one check passes. After every turn, PaCPO probes the policy, outside the conversation, for the plan it would record if the dialogue ended now, and checks that tentative plan against the rules of both worlds. A gate opens only once both tentative plans are well formed, the worlds admit no common plan, and the differing private fact has been disclosed in both conversations, as marked by a frozen LLM annotator. Credit is positive when each world's plan fits its own world but not the other; identical plans in both worlds earn nothing. These credits form a potential, and potential-based shaping redistributes the terminal reward across turns without changing the total. Updates use PPO with a learned value baseline and LoRA adapters.

5. How it was evaluated

The authors benchmark thirteen models, from Qwen3-8B to GPT-6 Astra and Gemini 3.7 Flash, on 240 test tasks: 60 held-out scenarios, two worlds, two roles, each played against a frozen Qwen3-8B partner. Feasibility is computed by program; Integrity and the seven SOTOPIA ratings are judged by Qwen3.8-Max, a different model from the Qwen3.8-27B judge used during training, and a second judge, Kimi-K3, agrees with it on 90% of Integrity decisions. For training, Qwen3-8B is tuned with LoRA on 384 conversations per method and compared with PPO, GRPO, Dr.GRPO, GiGPO and GraphGPO, all given the same terminal reward and the same dialogue budget. Further checks include ablations of the paired credit, self-play where both sides use the trained checkpoint, zero-shot transfer to SOTOPIA scored by GPT-4o, training on MultiWOZ and CaSiNo, and a blinded pilot with two expert annotators on 24 dialogue pairs.

6. How it performed

Verified Success across the thirteen models runs from 20.42% for Qwen3-8B to 82.50% for Gemini 3.7 Flash, with GPT-6 Astra at 80.83%, so even frontier models fail roughly one task in five. Ratings and outcomes diverge: GPT-5.6 Luna beats Gemini 3.7 Flash on judged goal completion yet reaches only 52.08% Verified Success. PaCPO takes Qwen3-8B to 28.33% against 25.00% for PPO and 26.25% for GRPO, with the gain coming from Integrity rather than Feasibility; SFT followed by PaCPO reaches 31.67%. Removing the paired world, shuffling credit timing, or opening the gate before disclosure each costs two to three points. Gains concentrate in bounded quantities and authorization; resource allocation and both scheduling families barely move. On CaSiNo negotiation, agreement rises from 36% to 64%. The limits are one 8B model, six structured families, an LLM judge for Integrity, and a human study of 24 pairs. Our call: pilot outcome verification on our own agent transcripts now, and watch PaCPO until it is shown on larger models. (Source: arXiv 2610.20195, 2026)