$\nabla$Mimic: Ultra-Efficient Humanoid Control with Differentiable Simulation: what it means for business leaders

By training on the simulator's own exact gradients, made stable and cheap, one GPU learns a humanoid motion-tracking policy that matches the previous best score in 20 minutes instead of 1.2 hours.

1. What the method is

∇Mimic is a way to train humanoid motion-tracking controllers from exact gradients of the physics simulator. It comes with NablaEngine, a GPU simulator for the Unitree G1 that computes MuJoCo's full contact dynamics and their hand-derived analytic derivatives. PPO, the usual method, treats the simulator as a black box and learns only from reward scores. This method instead learns from how each action changes the return, measured by backpropagating through short windows of simulated steps. Two algorithmic pieces make this stable. The first clips each sample's gradient relative to the batch median at every backward step. The second regresses the policy onto action targets whose size is fixed in radians. One policy trained on all 40 LAFAN1 motion clips on one GPU reaches the previous state-of-the-art score of 0.82 in 20 minutes. SuperMimic, a world-model method, reports 1.2 hours to get there. Both the method and the engine are open source.

2. Why the method was developed

Nearly every humanoid tracker is trained with PPO, which needs enormous numbers of samples. Even with massively parallel GPU simulation, a whole-body policy takes hours to days, and the SONIC project used about 21,000 GPU-hours. Differentiating through the simulator promises far more information per sample, but on contact-rich control it has failed. SHAC, the standard first-order method, stops on exploding gradients within minutes in every G1 run. The authors trace this to single samples near contact events, such as feet almost touching, where the derivative has no upper bound. A global norm clip either lets that one sample through or shrinks every other sample's gradient to nothing. SuperMimic avoided the problem by using gradients of a learned world model, which are smooth and cheap but describe a different system. The paper argues SuperMimic won on price, not signal: exact gradients reached 0.82 in about 3,200 updates against roughly 41,000, but existing simulator adjoints were too slow to pay off.

3. Who should care

Teams that train robot controllers in simulation are the direct audience. If we build locomotion or whole-body policies, this moves the compute bill from GPU-days toward GPU-hours, and engineers can test an idea within an afternoon instead of overnight. Robotics leaders choosing humanoid platforms should know the speed is tied to the Unitree G1. On another robot we would get the algorithm but not the engine, because NablaEngine's kernels are hand-specialized and would need similar engineering for each new body. Infrastructure and finance owners should note the hardware: one RTX PRO 6000 workstation GPU instead of a multi-GPU cluster for this class of task. ML teams outside robotics can borrow one idea, clipping each sample relative to the median inside backpropagation through time, for any differentiable pipeline where rare samples blow up. Teams buying teleoperation or motion-tracking stacks should ask vendors how their controllers are trained and what retraining costs.

4. How the method works

Each update samples a batch of start states from a replay buffer, simulates short windows on NablaEngine with a recorded tape, and backpropagates the window return through the analytic adjoint. At every backward step, each sample's action gradient is clipped to a fixed multiple of the batch median norm before it flows further. This touches 9 to 21 percent of samples and removes the explosive tail. The clipped gradients become target actions, scaled so the average change across the batch is a set number of radians. The policy is then fitted to these targets with several minibatch steps of a separate Adam optimizer. No sample's target can move more than 0.13 rad. A critic trained with TD(lambda) estimates the return beyond each window. The policy outputs offsets to the reference pose rather than absolute joint commands. NablaEngine runs one simulated world per GPU warp, computes a whole control step's adjoint in a single kernel, differentiates the contact solver through its optimality conditions, and freezes derivatives below 1 mm near the two contact cases that diverge.

5. How it was evaluated

The authors train one policy on all 40 LAFAN1 clips retargeted to the G1: 264,705 frames, or 2.45 hours of motion. The robot has 29 actuated joints and is controlled at 50 Hz. Each clip is evaluated once from its first frame on MuJoCo Warp, a different simulator from the one used for training, with a deterministic policy and no noise. The score is the full-clip normalized tracking return, so a fall forfeits the rest of the clip. Wall-clock time counts training only, on one RTX PRO 6000, with three seeds per configuration. The baselines are PPO on the same engine in two configurations, run as single seeds for six hours; SHAC with its published gradient guard and with relaxed guards; and SuperMimic. Because SuperMimic's code is not public, its numbers are the ones its paper reports on an H100. Ablations remove one component at a time, and a separate run trains the same algorithm on MuJoCo Warp's experimental adjoint to separate the engine's contribution from the algorithm's. All results are in simulation.

6. How it performed

∇Mimic reaches a score of 0.82 in 0.34 hours, 0.854 at one hour and 0.875 at three hours, with no falls on any of the 40 clips at the end. The three seeds agree closely. PPO on the same engine reaches 0.64 after three hours and 0.70 after six, with eight clips ending in falls, so cheaper samples did not close the gap. SHAC with its published guard stops at 0.06 hours with a score of 0.373, and with relaxed guards it reaches 0.816 at three hours. Removing per-sample clipping made all three seeds diverge. On MuJoCo Warp, per-update learning curves overlap, but updates take 1.63 seconds against 0.37, so the engine changes speed, not learning. The authors list the limits: one robot, flat ground, LAFAN1 only, no hardware deployment, an unmeasured bias from clipping, single-seed PPO runs, and a SuperMimic comparison made on a different GPU from reported numbers. Our call: pilot it if we train controllers for the G1, and otherwise watch until it covers more robots, terrain and real hardware. (Source: arXiv 2610.41654, 2026)