Robot foundation models

The idea

A foundation model is trained broadly, then adapted to specific tasks. Applied to robotics, the proposition is that a single model trained across many robots, environments and tasks will transfer to new ones better than a model trained narrowly on each — and that robot capability might therefore accumulate the way language capability did.

That is a genuine claim with real supporting evidence and real limits, and it is worth separating the two.

Why it was not obvious this would work

Language models had a corpus. Robotics does not. Every unit of robot interaction data has to be produced by a robot acting in the world, at wall-clock speed, on hardware that costs money and wears out. Worse, data collected on one robot was long assumed to be close to useless on another, because the embodiment differs: different kinematics, different grippers, different cameras, different action spaces.

If that assumption held, there could be no foundation model for robotics — only a series of per-robot models.

What the cross-embodiment evidence showed

The assumption turned out to be too pessimistic. Pooling demonstration data across many robot types and training a single policy on the union produced models that performed better on the original robots than models trained on each robot alone. The Open X-Embodiment collaboration is the clearest public example: a dataset assembled from many institutions and robot platforms, with models trained across it showing positive transfer.

That result is the empirical basis for the whole category. It does not say capability is fully portable; it says data from different embodiments is not wasted, which is the weaker claim that actually matters for building anything.

How these models are built

• Pre-trained vision and language backbones. Most approaches start from models already trained on internet-scale image and text data, inheriting object recognition and instruction understanding rather than learning them from robot data.
• Action representation. The design decision that distinguishes approaches: actions may be discretised into tokens, predicted as continuous vectors, or generated by a diffusion or flow-matching head. This choice affects precision, smoothness and inference cost.
• Robot demonstration data. Teleoperated demonstrations remain the primary source, supplemented by simulation and, increasingly, human video.
• Post-training. Adapting a generalist model to a specific robot and task, which in practice is where most deployed performance comes from.

As of 2026 the landscape includes open models from the research community alongside commercial generalist policies from several well-funded labs, with new releases arriving at a pace that makes any specific ranking obsolete within weeks. The approaches are more durable than the leaderboard.

What they are good at

• Variation within a familiar distribution. A new object that resembles training objects, a rearranged scene, a differently worded instruction.
• Instruction following. Mapping natural language to a task, inherited largely from the language backbone.
• Reducing per-task engineering. Adapting a generalist model is often less work than building a bespoke pipeline, which changes the economics of trying something.
• Semantic generalisation. Benefiting from concepts present in internet data but absent from any robot dataset.

What they have not solved

This is the part usually compressed into a single sentence about "ongoing work", and it deserves more.

• Reliability. Success rates that are impressive in a paper are frequently well below what a production process requires. The gap between a good demonstration and an acceptable intervention rate is the central unsolved problem.
• Verification. There is no practical method to characterise what a large learned policy will do outside its training distribution, which is precisely the situation deployment creates.
• Failure detection. Knowing that the model has failed, promptly and reliably, is less developed than the acting itself — and it is what long-horizon tasks depend on.
• Precision and force. Tasks requiring tight tolerance, controlled force or stable contact remain substantially harder than free-space motion.
• Latency and compute. Large models impose inference cost that interacts badly with control-loop deadlines and with the compute actually available on a machine.
• Evaluation. Benchmarks are not standardised, protocols differ, and reported numbers are frequently not comparable between papers, let alone to a factory floor.
• Safety. A learned policy cannot currently be relied on to encode its own refusal conditions. That responsibility sits outside the model, in a supervisory layer.

Post-training is where the deployed performance comes from

The generalist pre-trained model gets most of the attention and supplies a minority of the performance that actually ships. What makes a policy work on a specific robot doing a specific job is the adaptation stage: demonstrations collected on that machine, in that environment, with that gripper, covering the exceptions that occur there.

This matters commercially as well as technically. If most of the value is in post-training, then the data collected during a deployment is a significant asset, and the question of who owns it, who may learn from it, and whether an operator's data improves a competitor's robot becomes a term to negotiate rather than an afterthought. Ventures that defer that question tend to answer it in whichever way is convenient once there is data to argue about.

Open and closed models

The category has both. Openly released model weights let a team inspect, fine-tune, run locally and verify claims independently, at the cost of doing the integration work themselves. Commercial generalist policies are typically easier to adopt, better supported, and opaque in precisely the ways that matter for assurance.

For anything deployed around people, the ability to characterise behaviour matters more than headline capability, which argues for whichever option can actually be inspected and tested. That is a judgement about assurance, not about licensing ideology, and it should be made per deployment rather than as a policy.

How to read a claim about one

A useful set of questions, applicable to any announcement in this area:

• How many trials, and was the figure a success rate or a best run?
• What was the intervention rate, and who was intervening?
• On which robot, with which gripper, and in whose environment?
• Were the objects and scenes in the training distribution?
• What happened in the failures, and were they detected?
• What is the task-level success rate, as distinct from the per-step rate?
• Is the evaluation reproducible by someone else?

A result that cannot answer those is a demonstration, which is a legitimate thing to publish and a poor basis for a deployment decision.

What this means practically

Foundation models changed the default approach to robot capability, and that is a real shift rather than a hype cycle. They did not remove the need for the rest of the stack: hardware abstraction, uncertainty handling, success detection, safety supervision, evaluation on real hardware, and the operational machinery to deploy and roll back. A generalist policy inside a system with none of those is a research artefact, not a product.

The useful framing is that the model is one layer among several, and currently the fastest-moving one. The layers around it are what turn it into something an operator can rely on.

Sources

• Open X-Embodiment: Robotic Learning Datasets and RT-X Models