Sim-to-real transfer

Why simulation is unavoidable

Robot learning needs far more interaction than real hardware can economically supply. A real robot collects experience at wall-clock speed, needs supervision, breaks, wears, and occupies a space someone is paying for. A simulator runs faster than real time, in parallel, without consequence, and can reset instantly.

That difference is not marginal. It is the difference between thousands of trials and hundreds of millions. Any method requiring large-scale interaction — reinforcement learning especially — is simulation-dependent in practice.

The reality gap

A policy trained in simulation and deployed on hardware usually performs worse, sometimes catastrophically worse, and the reasons cluster predictably.

• Contact and friction. The hardest thing to simulate and the thing manipulation depends on most. Coefficients vary with surface, wear, temperature and contamination, and contact solvers approximate rather than resolve.
• Compliance and deformation. Grippers flex, objects deform, cables resist. Rigid-body simulation is fast because it ignores most of this.
• Actuator behaviour. Real motors have backlash, stiction, torque ripple, thermal derating and controller dynamics. Simulated ones usually do not.
• Latency. Real systems have sensing delay, communication delay and control delay. A policy trained without them learns timing that does not exist.
• Sensor realism. Simulated cameras lack the noise, motion blur, rolling shutter, exposure behaviour and lens artefacts of real ones; simulated depth lacks the characteristic failure patterns of real depth sensors.
• Mass properties. Published inertias and centres of mass are approximations, and assembly tolerances move them.
• The long tail. Spilled liquid, a loose label, a bent tote, glare through an opened door. Simulators contain what someone modelled.

Techniques for crossing it

Domain randomisation. Rather than making simulation accurate, make it varied: randomise friction, mass, lighting, textures, latency and sensor noise across a wide range so the policy must be robust to all of them. Reality then becomes one more sample. This is the most broadly used approach and it works well — at the cost of a policy that is conservative, because robustness across a wide range trades away performance at any point in it.

System identification. Measure the real system and tune the simulator to match: identify actual friction, inertia, actuator response and latency, then train against that calibrated model. Precise where it applies, and expensive per platform.

Real-to-sim-to-real. Collect real data, use it to correct the simulator, train, deploy, collect again. An iterative loop that converges rather than a single crossing.

Learned residuals. Train the bulk of the behaviour in simulation, then learn a correction term on real hardware that accounts for what the simulator got wrong. Keeps real-data requirements small.

Domain adaptation. Align the representations the policy uses across simulated and real inputs, so that the features it depends on look the same in both.

Conservative deployment. Lower speeds, larger margins, tighter envelopes and earlier refusal. Not a transfer technique, but the one most responsible for simulation-trained systems working in the field.

What a simulator actually is

Two components are doing separate jobs, and they fail differently.

The physics engine integrates rigid-body dynamics and resolves contacts. Contact resolution is the hard part: real contact involves deformation, micro-slip and friction that varies with pressure and surface state, while a solver approximates it as constraints resolved per timestep. Different engines make different approximations, which is why the same scene behaves differently across simulators and why a policy can be tuned to a solver's artefacts rather than to physics.

The renderer produces the images a vision-based policy consumes. Photorealism helps, but the systematic differences — noise characteristics, exposure behaviour, motion blur, depth-sensor failure patterns — matter more than visual fidelity, because those are the regularities a network latches onto.

A simulator being fast, parallel and resettable is what makes it useful. Those same properties are why it is a model rather than a measurement.

Mistakes that are easy to make

• Tuning until it works in simulation. A policy fitted tightly to a solver's behaviour has learned the solver, not the task.
• Omitting latency. Training without realistic delay produces timing that cannot be reproduced, and it usually shows up as instability rather than as an obvious error.
• Randomising the wrong things. Randomising textures while leaving friction fixed produces a policy robust to appearance and fragile to physics.
• Evaluating in the training scene. Real-world evaluation in the exact environment the simulation modelled measures very little.
• Not counting the real data. A result presented as sim-to-real, after substantial fine-tuning on hardware, is a hybrid result and should be described as one.
• Treating a reset as free. Simulation resets instantly; a real failure requires someone to walk over, which is a cost that shapes what is deployable.

What transfers and what does not

A rough and reasonably reliable guide:

• Transfers relatively well: navigation and collision avoidance, free-space motion, locomotion over modelled terrain, high-level sequencing, anything geometry-dominated.
• Transfers poorly: contact-rich manipulation, force-controlled tasks, deformable objects, precise insertion, anything where the outcome depends on friction, and anything where small timing differences change the result.

Noticing the pattern is useful: simulation is good at geometry and weak at physics that involves touching things. Which is unfortunate, because touching things is most of manipulation.

How to evaluate a sim-to-real claim

• How many real-world trials, on how many distinct object instances?
• Was the real evaluation in the same environment the simulation modelled, or a different one?
• How much real data was used in fine-tuning, and does the claim account for it?
• What was the intervention rate, not just the success rate?
• Were speeds or margins reduced for the real deployment relative to simulation?
• What were the failure modes, and were they anticipated by the simulation?

A simulation result with no real-world evaluation is a simulation result. It can be genuinely informative, and it is not evidence about hardware.

Why this determines what gets deployed first

The map of what transfers predicts the order in which robotics arrives commercially, and it matches what is actually happening. Structured environments with rigid objects and geometric tasks — logistics, material handling, inspection routes — are where simulation-trained capability works soonest. Unstructured environments with deformable objects and force-dependent outcomes are later, not because anyone chose that sequence, but because the reality gap is narrowest in the first case and widest in the second.

Choosing a simulator

There is no best one, and the trade-offs are stable enough to be worth stating. The questions that actually discriminate:

• Contact fidelity versus speed. Solvers that model contact more carefully run slower, and reinforcement learning needs volume. This is the central tension and it has no clean resolution.
• Parallelism. Whether thousands of environments can run simultaneously, ideally on the same hardware as training, which is what made modern locomotion results practical.
• Rendering quality. Matters for vision-based policies, irrelevant for state-based ones, and expensive either way.
• Robot model availability. Whether your specific robot has a validated description, or whether somebody has to build and verify one.
• Determinism. Whether a run can be reproduced exactly, which matters a great deal for debugging and is often sacrificed for speed.
• Ecosystem fit. Whether it connects to the middleware and tooling you already use.

A reasonable default is to use a fast, parallel simulator for training and a slower, higher-fidelity one for validation before touching hardware — accepting that neither is reality and that the second is a better predictor than the first.

What good sim-to-real practice looks like

The teams that transfer reliably tend to do the same unglamorous things: they measure the real system's latency and actuator response and put those in the simulator rather than guessing; they randomise the parameters they have measured to be uncertain rather than randomising everything uniformly; they validate in an environment the simulation did not model; they deploy conservatively at first and tighten later; and they treat every real-world failure as a specification for what the simulation got wrong, feeding it back rather than working around it.