Learned Terminal Costs for Short-Horizon Model Predictive Control of Autonomous Trucks in Interactive Highway Traffic: what it means for business leaders

A Chalmers and Volvo team shows that a short three-step planner for an autonomous truck drives more safely and cheaply than a longer one when a learned estimate of the long-run cost stands in for everything past its horizon.

1. What the method is

The paper builds a highway controller for a heavy truck that combines model predictive control with reinforcement learning in one specific way. The controller plans speed and lane changes over only a few steps by dynamic programming on a discretized grid of states and actions, and it enforces safe gaps through chance constraints that widen with prediction uncertainty. What it cannot see past the end of that short plan is normally covered by a hand-designed or zero terminal cost. Here that terminal cost comes from a Deep Q-Network trained offline on the same cost function, with its value rescaled into the planner's own cost units and inserted directly into the backward pass. The learned part never issues commands. It only tells the planner how good or bad each possible end state is, so the optimizer keeps its exact constraint handling and replans at every step.

2. Why the method was developed

Both standard approaches fall short for trucks. MPC handles constraints rigorously, but a short horizon with a crude terminal cost is short-sighted, and a long horizon gets expensive to solve in real time. RL learns long-term behaviour but gives no constraint guarantees, and under sparse collision penalties it tends to drive overly cautiously. Earlier hybrids used learning to tune MPC weights, pick horizons, add corrections or reshape reference trajectories, and none learned the terminal cost itself. The work that did learn terminal costs covered continuous low-level control of a single agent in settings that do not react. Nobody had tested a learned terminal cost on discrete tactical decisions like lane changes, where it interacts with a combinatorial search, or in traffic that responds to the ego vehicle and so shifts the very states the value function was trained on.

3. Who should care

The first audience is teams building autonomy for trucks, logistics vehicles or any machine that plans under hard safety limits among agents that react to it. For them the result is a cheap way to get long-range foresight without paying for long horizons on embedded hardware. Controls and robotics engineers who already run MPC can treat the terminal cost as a slot that a learned model can fill without giving up their constraint machinery. ML teams get a cleaner job: train a value function and let the optimizer own safety, rather than certify a policy end to end. Safety and validation leads should note where the guarantees still sit and where they do not. Buyers evaluating autonomy stacks gain a sharper question to ask vendors: how is beyond-horizon behaviour estimated, and how was it validated against reactive traffic?

4. How the method works

At each step the controller reads the traffic state, predicts each surrounding vehicle's motion by constant-acceleration extrapolation with its lane held fixed, and propagates Gaussian measurement noise so that position variance grows with each horizon step. A vehicle caught mid lane change is duplicated into both lanes. The safe-gap constraint becomes a deterministic distance inflated by the inverse Gaussian CDF times that standard deviation. The stage cost sums seven weighted terms: speed tracking, acceleration, a one-time lane-change penalty, a cooldown penalty, a left-lane penalty, a quadratic safety penalty and a large collision penalty. Every reachable end state is mapped to a 14-dimensional observation (gaps, relative speeds, lane-change phase) and fed to a DQN with two 256-unit hidden layers, trained with the reward set to the negative stage cost and the same discount factor. The rescaled maximum Q-value becomes the terminal cost for the dynamic-programming sweep, and only the first action is executed.

5. How it was evaluated

All testing is in simulation: SUMO on a straight three-lane one-way highway, with surrounding vehicles following SUMO's Krauss car-following and LC2013 lane-change models, so they react to the truck. The DQN was trained with Stable-Baselines3 for about 20 million steps with perfect observations. Checkpoints were screened on 10 holdout seeds, the top ten re-scored on 500 seeds, and the configuration retrained five times to check reproducibility. Four controllers were compared at horizons of one to five steps: pure MPC with zero terminal cost, MPC with a rollout terminal cost that simulates a Krauss-based base policy 10 or 20 further steps, pure RL acting directly, and the hybrid. Runs used 100 episodes with sensor noise, plus a noise-free deterministic variant. Metrics were collision rate, cost per step and its spread, mean speed, a cost breakdown by term and computation time per step.

6. How it performed

Pure MPC at one step collided in 86% of episodes. The hybrid at one step had none. The best configuration was the hybrid at three steps: zero collisions, cost 329 per step, against 754 for the best pure MPC at five steps and 551 for pure RL. Its safety-penalty cost was also lower, so it kept wider margins. Rollout terminal costs never removed collisions, and lengthening the rollout made results worse, because a terminal cost is only as good as its base policy. The hybrid drives slower, about 21.1 against 22.6 m/s, and costs 91.6 ms per step against 17.5 ms for five-step MPC, measured on a shared cluster. In the noise-free tests the three-step hybrid had a 1% collision rate. The authors list as limits the straight homogeneous road, a kinematic model with no continuous lateral control, and a crude prediction model. Our call: pilot it in simulation if we already run a discrete MPC planner, and watch it otherwise. (Source: arXiv 2610.40257, 2026)