Imitation learning for robots

The idea, and why it dominates

Imitation learning trains a robot to reproduce behaviour from demonstrations rather than from a reward signal. A human performs the task, the system records observations paired with actions, and a policy learns the mapping between them.

It dominates practical robot learning for an unglamorous reason: it is the only approach where the data collection is obviously tractable. Reinforcement learning needs millions of interactions and a reward function somebody has to design. Imitation needs a person doing the task correctly, which is something most organisations already have.

How demonstrations are collected

• Teleoperation. A human drives the robot directly, through a controller, a leader-follower rig, or a VR interface. The data is in the robot's own action space, which is what makes it directly usable, and the collection rate is bounded by human time.
• Kinesthetic teaching. Physically guiding the robot's arm through the motion. Intuitive, limited to robots that are backdrivable and tasks where a person can reach.
• Human video. Abundant and attractive, with two hard problems: no action labels, and a human body that does not map onto the robot's. Active research, not a solved pipeline.
• Scripted or planned policies. Using a classical controller as the demonstrator in simulation, which scales well and inherits that controller's limitations.

The methods, and how they differ

Behaviour cloning is the straightforward version: supervised learning from observation to action. Simple, effective where data is plentiful and the task is consistent, and subject to the compounding problem below.

Action chunking predicts a short sequence of future actions rather than a single step. This produces smoother motion and reduces compounding, because the policy commits to a trajectory segment instead of re-deciding at every timestep from a slightly drifted state.

Diffusion policies generate actions through an iterative denoising process. Their advantage is handling multimodality: when several different motions are each correct, a model trained to predict the average produces the one motion that is wrong. Averaging two valid ways around an obstacle gives you the path straight through it. Diffusion represents the distribution instead of its mean, which matters more than it sounds.

Vision-language-action models extend imitation with a pre-trained vision and language backbone, inheriting object and instruction understanding from internet-scale data and learning only the mapping to motion from demonstrations.

Compounding error, the problem at the centre

This is the structural weakness of the whole approach and it is worth understanding precisely.

A policy trained on demonstrations has only ever seen states a competent demonstrator visits. In deployment it makes a small error, which puts it in a state slightly off the demonstrated distribution. From there its next action is slightly worse, which puts it further off. Error compounds because the policy's own mistakes move it into territory it has no training for, and recovery behaviour was never demonstrated — precisely because the demonstrator never needed it.

The mitigations are all partial: collect demonstrations that include recoveries, deliberately perturb the robot during collection so the demonstrator has to correct, add interactive correction where a human intervenes during rollout and those corrections become training data, or use action chunking to reduce the number of decision points.

What it is good at

• Tasks a human can demonstrate and a specification cannot easily capture — the ones where "you know it when you see it" is the honest description.
• Getting to a working first version quickly, with hundreds rather than millions of trials.
• Contact-rich behaviour where a human's feel for the task transfers through the demonstration.
• Narrow, repeated tasks, which is where most commercial value currently sits.

What it is bad at

• Exceeding the demonstrator. The policy is bounded by the demonstrations. Inconsistent demonstrations produce an inconsistent policy.
• Recovery. Unless failures and corrections were deliberately demonstrated, the policy has no model of getting out of trouble.
• Generalising beyond the distribution. New object categories, new environments and new layouts are where performance falls off.
• Long horizons. Chained steps compound both error and the absence of reliable success detection.
• Knowing it has failed. Demonstrations contain successes. A policy trained only on success has no representation of failure, which is why these systems often continue confidently after the task has gone wrong.

How much data does it take

There is no universal number, and the honest ranges are wide enough to be worth stating anyway. A narrow, well-constrained task with consistent presentation can produce useful behaviour from a few hundred demonstrations. A task with real variation in objects, positions and lighting generally needs thousands. Anything approaching open-ended behaviour is beyond what demonstration collection alone reaches, which is the whole reason cross-embodiment pooling and pre-trained backbones exist.

What moves the number more than anything else is how much variation the deployment actually contains. Teams routinely underestimate this, because the collection session happens in a tidied version of the real environment. The variation that matters is the variation on a normal Tuesday, not the variation during the demo.

Evaluating an imitation policy

Offline metrics on held-out demonstrations are close to useless for predicting deployed behaviour, because the compounding problem means the states that matter are the ones no demonstration contains. A policy can match held-out actions closely and still fail in rollout.

What actually tells you something:

• Rollout success over many trials, on real hardware, with objects and positions randomised rather than reset to the demonstration setup.
• Intervention rate — how often a human had to step in, and for what.
• Failure taxonomy, counted. Which failures dominate decides what to collect next.
• Behaviour at the edges: a novel object, an unusual pose, a partial occlusion.
• Whether failures were detected, since an undetected failure propagates into every downstream step.

Practical guidance on data

• Consistency beats volume. Five hundred consistent demonstrations usually beat two thousand inconsistent ones. Decide the approach before collecting, not during.
• Demonstrate the exceptions. The distribution of your demonstrations should resemble the distribution of reality, including the awkward cases, or the policy will meet them for the first time in production.
• Record everything. Full observation streams, not just the action trace. You will want to retrain with different inputs later.
• Log the operator and the conditions. When a subset of data turns out to be bad, you need to be able to find it.
• Expect to re-collect. The first collection usually teaches you what you should have collected.

Interactive correction, and why it works

The most effective answer to compounding error is also the most direct: let the policy act, and have a human correct it while it does. The corrections become training data, which means the dataset grows precisely in the states the policy actually visits rather than the states a demonstrator would have chosen.

This addresses the structural problem head-on. Offline demonstrations only cover the expert's trajectory; interactive correction covers the policy's own distribution, including the drifted states where it needs help most. Several rounds of this typically beat a much larger offline dataset, because the data is targeted rather than merely abundant.

The costs are real. It requires a human present during rollout, on hardware, with a safe way to intervene mid-motion — which on anything that can injure or damage means a considered interface rather than a person grabbing the arm. It also requires the corrections to be recorded in a way that distinguishes them from ordinary demonstrations, since they carry different information.

Diagnosing a policy that will not improve

When more data stops helping, the cause is usually one of a short list, and working through it in order saves weeks:

• Inconsistent demonstrations. Multiple strategies in the dataset, so the policy is learning an average that works for none of them. Check whether different operators did it differently.
• Calibration drift mid-collection. The dataset silently contains two different geometries.
• A missing observation. The policy cannot see something the demonstrator relied on — depth, force, a view the operator had and the camera does not.
• Distribution mismatch. The evaluation setup differs from the collection setup in a way nobody documented.
• The task needs force feedback and the demonstrations were collected without it, so they encode the wrong behaviour.
• It is a detection problem, not a policy problem. The policy is fine and does not know it has already failed.