Onboard compute for robots

Why it runs on the robot

The obvious architecture is a thin robot and a powerful server. It is wrong for most of what a robot does, for three reasons that do not go away with better networking.

Latency. A control loop closing at several hundred hertz cannot wait for a round trip. Even on an excellent local network, the tail of the latency distribution — not the average — is what determines whether the loop is stable, and the tail is what wireless is worst at.

Availability. Wireless coverage in industrial buildings is uneven in ways that are difficult to predict and expensive to fix. A robot that stops when it loses signal is a robot that stops.

Safety. Anything in a safety function has to be deterministic and self-contained. A network is neither.

What does move off-robot: fleet coordination, map distribution, logging and analysis, model training, and anything a human is going to look at. The split is not cloud versus edge. It is: things with a deadline run locally, things without one do not.

The real budget

Compute on a robot is constrained by four things, and the one everybody quotes is the least binding.

• Power. Every watt of compute is runtime removed from the battery. On a mobile robot this is a direct trade against the thing the robot exists to do. A hundred-watt compute module on a robot with a modest battery is a meaningful fraction of the duty cycle.
• Thermal. Heat has to leave. A sealed enclosure on a warehouse floor in August is a different thermal environment from a bench in an air-conditioned lab, and a module that throttles is a module whose published figures no longer apply. This is where most sizing errors show up, and they show up in summer.
• Mass and volume. On an arm or a legged platform, compute mass competes with payload and changes the dynamics.
• Cost at fleet scale. A difference of a thousand dollars per robot is a rounding error on one unit and a line item on two hundred.

What is available

NVIDIA Jetson Orin. The default for robotics for several years. The AGX Orin module sits at the top of the family at up to around two hundred and seventy-five trillion operations per second with sixty-four gigabytes of memory, and the Orin NX and Orin Nano step down in capability, power and price. The family's real advantage is the software: CUDA, TensorRT and an ecosystem where most robotics-relevant models already run.

NVIDIA Jetson Thor. The Blackwell-generation successor, which became available in August 2025. The T5000 module is quoted at up to around two thousand and seventy teraflops of FP4 compute with a hundred and twenty-eight gigabytes of memory in a hundred and thirty watt envelope, with the developer kit starting at three and a half thousand dollars; NVIDIA claims up to seven and a half times the AI compute of Orin and three and a half times the energy efficiency. Lower-tier T3000 and T2000 modules followed. Read those figures carefully — the headline number is FP4, a four-bit format, and comparing it against an INT8 figure from the previous generation is not a like-for-like comparison. The relevant question is throughput on your model at your precision, measured.

Industrial x86. A compact fanless PC. Unexciting, widely available, straightforward to develop on, and frequently the right answer when the workload is classical perception and planning rather than large neural inference.

Microcontrollers and real-time targets. Beneath all of the above sits the hardware that actually closes the control loops and implements the safety functions. Deterministic, modest, and not running Linux.

Alternatives. Qualcomm's robotics platforms and various accelerators exist and are credible on paper. The practical question is always whether the models you intend to run are supported by the toolchain, and how much of your engineering time goes into finding out.

The three-tier split

Nearly every serious robot settles into the same architecture, arrived at independently by people who have been burned.

Tier one: the safety controller. Separate hardware, certified where required, implementing the stop functions. It is not running your application code and it does not share a processor with it. Its independence is the entire point. The guide on functional safety covers why.

Tier two: real-time control. Motor control, balance, force control. Hard deadlines, measured in hundreds of microseconds to a few milliseconds. A real-time operating system or dedicated hardware; a missed deadline is a fault, not a slow frame.

Tier three: perception, planning and policy. Soft deadlines, large compute, Linux, the GPU. A late frame here degrades behaviour rather than breaking it.

Collapsing tiers two and three onto one processor to save cost is a recurring mistake. It works in the lab and produces intermittent, unreproducible timing faults in the field, which are among the most expensive defects to chase.

Sizing it honestly

Published performance figures are measured under conditions that do not resemble a robot. The only sizing method that works is to run your actual models, at your actual input resolution and frame rate, on the candidate module, inside something like your enclosure, at the ambient temperature you will really see, with everything else that has to run at the same time also running. Then measure sustained throughput after thirty minutes, not peak throughput in the first ten seconds.

Budget headroom. A system sized exactly to today's workload has nowhere to put the model that is two per cent better, and the thing that will certainly happen over the life of the product is that the models get larger.

Quantisation and compilation — reduced precision, TensorRT or an equivalent — routinely deliver severalfold improvements and are nearly always worth doing before buying a larger module. Verify accuracy after quantising rather than assuming it survived.

What goes wrong

Thermal throttling discovered in production rather than in testing. Power draw that quietly halves runtime. A toolchain that does not support one layer in the model, found after the hardware is bought. Peak figures used for sizing where sustained figures were needed. And the architectural one: safety-relevant work placed on the same processor as the application, which is cheap on the bill of materials and expensive everywhere else.

What still belongs off the robot

Deciding what runs locally is only half the architecture. The other half is what leaves, and when.

Three things sensibly live off-robot. Coordination, where a fleet manager allocates work and resolves contention between robots — latency-tolerant by nature, since a traffic decision a hundred milliseconds late is fine. Logs and telemetry, which are the only reason a field failure is ever diagnosable, and which should be written locally first and uploaded opportunistically so a network outage costs you nothing. Model training and map building, neither of which has any business running on a robot.

The design rule that holds up: the robot must be able to complete the task it has already accepted with the network entirely absent. If losing connectivity mid-mission means an immediate stop, the architecture has a dependency it should not have, and every wireless dead spot in the building becomes an operational incident.

A sizing method rather than a number

Any specific recommendation here would be wrong within a year, so here is the procedure instead.

List every process that must run concurrently and what each one costs: camera drivers and image pre-processing, the perception models at the resolution and frame rate you actually need, state estimation, planning, the control stack, logging, and the operating system itself. Measure each in isolation on the candidate hardware, then together — the total is reliably worse than the sum, because of memory bandwidth contention that no datasheet mentions.

Then apply three multipliers. Run it hot, in the enclosure, for an hour, and use the sustained figure. Add headroom for the model you will want in eighteen months, because you will. And check the memory ceiling separately from the compute ceiling, since running out of memory is a hard failure where running out of compute is merely slow.

If the result is uncomfortably close to the limit of a module, buy the next one up. The cost difference is small against the cost of discovering the problem after a hundred units have shipped.