AI Bits #7: What OpenVLA demonstrates—and what it does not

Correction, 9 September 2026: this issue previously overstated cross-robot transfer and used an incorrect improvement figure. It now distinguishes the reported experiment from deployment speculation. The original March issue date is retained; this is a review of earlier research, not a new model announcement.

The source and the result

OpenVLA, version 3, is a 2024 paper by Moo Jin Kim and collaborators. Its abstract describes a 7-billion-parameter model trained on 970,000 real-world robot demonstrations. It reports a 16.5-percentage-point absolute task-success advantage over RT-2-X across 29 tasks and multiple embodiments. This is the authors’ result, not an MLAI replication.

The project page describes image/language inputs and tokenised actions decoded into continuous actions. It distinguishes direct WidowX/Google robot evaluations from fine-tuned Franka demonstrations. That distinction does not support a claim that any robot can be controlled by a prompt without integration or adaptation.

Section 6 identifies single-image input limitations, inference-throughput constraints and task success typically below 90%. Those limitations concern this paper’s model, not every later robotics system. Sources checked 9 September 2026.

Ask three different questions

This is an editorial reading framework, not a deployment checklist. Do not operate physical equipment from this article. A qualified robotics team must assess the actual system and environment; an event discussion or model benchmark cannot provide that approval.

Percentage points are not relative percent

Fictional arithmetic, not OpenVLA trial data: if one system succeeds on 40 of 100 attempts and another on 50 of 100, the success rates are 40% and 50%. The absolute difference is 10 percentage points; the relative increase is 10 ÷ 40 = 25%. Neither number says which failures occurred or whether a new task will behave similarly.

When summarising a paper, retain its stated unit. Do not reverse-engineer unreported trial counts from an aggregate percentage. Also check whether the reported average weights tasks equally or pools attempts; these need not give the same answer.

Exercise: what can a short demonstration tell you?

Fictional scenario: a clip shows an arm moving three objects into a tray. The caption says “general-purpose robot”. You have no trial log or training description. Start with “the clip shows three object transfers”, not “the robot reliably handles unfamiliar objects”.

• Ask whether these objects, instructions and positions appeared in training or adaptation data.
• Ask how many attempts were recorded and how the clip was selected.
• Ask whether a person reset the scene, corrected an action or chose the next instruction.
• Ask what counts as success and whether damage, delay or intervention is recorded separately.
• Record missing answers as unknown—not evidence of either success or fraud.

The exercise can be completed by reading a caption and listing missing information. It is not a test of a real robot and does not establish that the hypothetical system fails.

Build a robotics research discussion record

Paper/model version and publication date:
Claim in my own words:
Exact supporting section, table or figure:
Baseline, metric and units:
Task set, trial count and aggregation method:
Robot, observations and action interface:
Direct evaluation or adapted model:
Failures, interventions and uncertainty reported:
What the result does not establish:
Question to investigate next:

A complete record may contain unknowns. Keep them visible and return to the methods or supplementary material before repeating the claim. Do not describe your reading as reproduction of an experiment.

Turn the headline into a useful community question

At a relevant MLAI event, try: “Which part of this result depends on the evaluation setup?” Bring the paper version and one unresolved question. You do not need to own a robot or seek contract work to join a research conversation. Check each event’s topic, level, location and online availability before registering.

Editorial scope

This revision checked the abstract, project explanation and paper limitations. It is not a line-by-line replication or a current comparison of robotics products. The earlier buying suggestions and speculative forecasts have been removed rather than presented as demonstrated findings. Examples and exercises are labelled; no deployment, cost saving or human-review approval is claimed.

Frequently Asked Questions