Concordat: Typed Sessions for LLM Agents: what it means for business leaders

A small Python library stops an LLM agent from acting before a customer says yes by withholding the tools rather than asking, and the benchmark that missed the violations shows why our own tests might miss them too.

1. What the method is

Concordat is an open-source Python library, about 850 lines with no required dependencies, that treats an LLM agent as an object whose interface is a finite-state protocol, borrowing from typestate and session types. A developer writes an ordinary agent class, and decorators on its methods declare the protocol state in which each tool is legal and the state each call leads to. The library derives the protocol from the class, checks it when the class is defined, and enforces it on every model turn by offering the model only the tools legal in the current state. Calls outside that set are rejected before they have side effects. Decisions reserved for humans can be requested by the model but never taken by it. Token, tool-call and money budgets are charged per turn and split between agents on handoff so no allowance is spent twice. A subclass that could bypass its parent's protocol is rejected at definition time. Runners exist for the Anthropic Messages API and the OpenAI Responses API.

2. Why the method was developed

Agents are now allowed to cancel bookings, issue refunds and change accounts, and the rules governing those actions, such as obtaining an explicit yes before any change, usually live as a sentence in the system prompt. The model sees every tool on every turn and is trusted to use them in the right order. That holds until a customer pushes back, and a model eager to be helpful may read 'just do it and stop asking' as permission. The booking often ends up as the customer wanted, so outcome checks see nothing wrong, and the broken policy goes unnoticed. Earlier remedies fall short in different ways. Session types and typestate assume every party can be type-checked, which a model cannot. Runtime guardrails such as AgentSpec, Progent and ShieldAgent inspect a tool call after the model has chosen it. Constrained decoding works within a single response, not across a conversation. Concordat removes the illegal choice before the model can make it.

3. Who should care

Anyone in our company who owns an agent that writes to a system of record is touched first: customer operations, billing and account management, where a refund or cancellation made over a customer's objection becomes a complaint, a chargeback or a regulatory finding. Risk and compliance gain something they can inspect, because the protocol renders as a state diagram, so a reviewer can check the process without reading code, and a subclass cannot quietly add a shortcut around a required approval. Our platform and ML engineering teams get a small library that composes with the Anthropic and OpenAI SDKs they already use. QA and evaluation teams should note the sharper finding: the benchmark's own reward missed most violations, so our outcome-based test suites may be blind in the same way. Finance gets budgets that are split, not copied, across agent handoffs. Procurement should ask vendors how confirmation is enforced, not whether it is prompted.

4. How the method works

A protocol is a finite set of states, a set of named actions, an initial state, final states and a transition relation. The library rejects protocols with unreachable states, dead ends, or final states that still allow actions. On each turn the runner computes the actions allowed in the current state and passes only those tool definitions to the model, with parallel tool calls disabled so each call sees the state the previous one left. A call the model makes anyway is returned to it as an error and never reaches the tool. Because the visible tool list changes, the system prompt says tools are offered step by step and each tool result lists the actions now allowed. The author added both after a model distrusted an earlier result whose tool had vanished. A human step is requested by the model but decided by an application-supplied approver, with approve and reject as distinct transitions. Inheritance is checked by a breadth-first walk pairing each child state with the parent states it could correspond to, rejecting any route to a parent state the parent does not allow.

5. How it was evaluated

The study uses tau-bench release 1.0.1, airline domain, all 50 tasks, in which a simulated LLM customer talks to the agent and a task passes when the final database matches the expected one and the required information was communicated. The airline policy requires listing the action details and obtaining an explicit yes before any database update. The standard agent is the benchmark's own agent. The Concordat agent is the same agent under a two-state protocol derived from the benchmark's tool annotations in a 120-line adapter: lookup tools in both states, writing tools only after a customer message containing yes, yep or yeah. To create pressure, every customer scenario gained an instruction to refuse the first confirmation request with pushback that never says yes, and to say yes only when asked again. Models were Claude Haiku 5.5 as agent with Claude Sonnet 5.5 as customer, three runs pooled into 128 completed task-runs, and gpt-4.1 in both roles for one run. Every flagged change was read by hand.

6. How it performed

Under pressure the standard agent made database changes without a yes in 29 of 128 Haiku task-runs and 13 of 50 gpt-4.1 tasks. The Concordat agent made none, mostly because the writing tools were never offered. In the first Haiku run, 8 of the 13 violating tasks still earned full benchmark reward. Task success did not differ significantly. Haiku passed 105 against 112 of 128, and 11 of the 13 losses were transfers to a human after repeated refusals, which three added prompt sentences addressed before the gpt-4.1 run, where Concordat passed 22 against 20 and transferred 7 customers against 11. Cost was lower with Haiku, $1.49 against $1.70, and higher with gpt-4.1, $3.47 against $3.13, because conversations ran longer. The author's limits are real: the gate and the metric share one keyword test for yes, so the zero is guaranteed relative to that test, and there is one domain, two models, few trials, a prompt change between models and simulated customers. Pilot it on one agent that writes to a system of record, with a structured confirmation in place of the keyword. (Source: arXiv 2610.37461, 2026)