Towards User Simulators That Stay On Task And Still Sound Natural: what it means for business leaders

An open 8B user simulator trained by reinforcement learning against a constantly retrained real-versus-fake discriminator. It produces usable multi-turn test conversations three times as often as the best earlier simulator, at a fraction of the cost.

1. What the method is

UserLM-2.0 is an 8-billion-parameter user simulator from Cornell and Microsoft Research. It is a model that plays the human side of a multi-turn conversation with an AI assistant, given a high-level task intent. It starts from Llama3.1-8B-Instruct, fine-tuned on real user turns from WildChat. It is then post-trained with multi-turn reinforcement learning (GRPO) on a reward that combines two things: task fidelity, where an LLM judge checks how many of the intent's sub-goals the simulated user got across, and behavioral naturalness. The advance is how naturalness is scored. Instead of an LLM judge, a small 395M-parameter ModernBERT-Large classifier (the discriminator) decides whether a conversation came from a real user. It is retrained at intervals during reinforcement learning on the simulator's latest output, in the adversarial style of a GAN, where a generator and a detector train against each other. The weights are public as microsoft/UserLM-2.0. The paper also proposes a per-conversation validity metric that requires a simulated dialogue to be both on task and natural.

2. Why the method was developed

Assistants are increasingly used for long multi-turn tasks, and they get worse as conversations go on. To test and train for that, we need simulated users, and both existing approaches fail on one of two counts. A frontier assistant prompted to act as the user stays on task but writes polished, exhaustive messages no real user sends: GPT-5.6-Sol passed the naturalness test only 10.2% of the time. A model fine-tuned on real user turns sounds human but wanders, ends conversations early and drops requirements. The best earlier one, SFTUser-14B, got across about 61% of sub-goals. A simulator that leaves out requirements makes an assistant look broken when it is not. One that does not behave like real users gives conclusions that do not carry over to real users. The obvious fixes fail too: the authors show that asking an LLM judge to reward naturalness does not work, and a fixed discriminator gets gamed within about 75 training steps.

3. Who should care

This matters to anyone who evaluates or trains a conversational product. ML and evaluation engineers who run multi-turn benchmarks or regression suites for chat assistants and support bots get a cheap, open simulator that produces usable test conversations. QA and product teams can stress-test how our assistant handles users who reveal requirements gradually, repeat themselves and write briefly, rather than the front-loaded briefs frontier models write. Teams that fine-tune assistants with reinforcement learning against simulated users get a training partner that is harder to game. Finance will notice the cost: the authors estimate $0.11 per 100 valid conversations, against $205.20 with GPT-5.6-Sol. Our privacy and legal leads should note the authors' own warning that fine-tuning a simulator on our customers' transcripts risks it repeating their private information. Teams building agents that use tools or act in an environment should know the model has only seen plain chat.

4. How the method works

Training begins with supervised fine-tuning on real user turns, which gives UserLM-2.0-SFT. Reinforcement learning then samples conversations against Qwen3.5-9B as the assistant and scores each one on four components. Coverage is the share of sub-goals an LLM judge finds the user got across. The simulator sees a detailed intent while the judge scores against a more abstract version, so paraphrasing the brief does not pay. Quality is a pass-or-fail check against copying the intent, repeating earlier turns or slipping into the assistant role. If it fails, the whole reward is zero. Termination rewards ending with an end tag only after all sub-goals are met and penalizes quitting early. Naturalness is also pass-or-fail: does the discriminator, seeing only the user turns plus the intent, score the conversation as real above a threshold? The discriminator is retrained at fixed intervals on equal numbers of real and simulated conversations, so it keeps catching the simulator's newest tricks.

5. How it was evaluated

The data is the non-toxic part of the WildChat-4.8M chat logs, cut down to 199,249 conversations with more than one user turn. Qwen3-30B extracted the intents and GPT models the sub-goals. The test set has 2,000 intents spread evenly across one to seven sub-goals, with four simulated conversations for each, all run against Qwen3.5-9B as the assistant. The baselines are six prompted assistants (Llama-3.1-8B, Qwen3.5-9B, GPT-5.6-Luna, GPT-5.6-Sol, DeepSeek-V4-Pro and Kimi-K2.6), five trained simulators (USP, UserLM-8B, SFTUser-14B, HumanLM-Opinion and the authors' own SFT model) and held-out real conversations. Task fidelity is measured by quality, termination and coverage, with GPT-5.6-Luna judging coverage. Naturalness is measured by three detectors: a discriminator trained for each simulator, a general one, and the commercial AI-text detector Pangram-4. A conversation counts as valid if it passes quality, ends properly, reaches 80% coverage and fools at least one detector. Ablations test LLM-judge and fixed-discriminator rewards for naturalness.

6. How it performed

UserLM-2.0 produced valid conversations 46.2% of the time, against 27.8% for its own SFT starting point, 14.1% for SFTUser-14B and 6.7% for GPT-5.6-Sol. It had the highest coverage (88.0%) and naturalness pass rate (71.8%) of any simulator. With seven sub-goals it stayed valid 12.6% of the time, while every earlier simulator fell below 1%. How naturalness is rewarded matters: LLM-judge or fixed-discriminator rewards left validity between 7% and 10.5%, below the 20.7% reached with no naturalness reward at all. Real conversations remain far ahead at 80.2% valid, and real users spend 34 words per sub-goal against the simulator's 59. The authors name the limits themselves: chat only, with no tools or environments; a single assistant model in both training and testing; and metrics built on their own judges and detectors. Our call: pilot it as a cheap source of simulated users for our multi-turn evaluations, checking it against real transcripts before we trust what it reports. (Source: arXiv 2610.31512, 2026)