Paper summary: MolmoAct2: Action Reasoning Models for Real-world Deployment

Problem

According to the abstract, VLA models fall short for real deployment: frontier models are closed, open ones need expensive hardware, reasoning-augmented policies are too slow, and fine-tuned success rates aren’t dependable.

Method

Five advances over the first MolmoAct, from the Allen Institute (whose Molmo VLM is summarised here in Sept 2024):

1. MolmoER: a VLM backbone specialised for spatial and embodied reasoning, trained on 3.3M samples with a specialise-then-rehearse recipe.
2. Three new datasets for low-to-medium-cost robots, including MolmoAct2-BimanualYAM (720 hours of bimanual teleoperation, described as the largest open bimanual dataset) and filtered DROID and SO100/101 subsets.
3. OpenFAST: an open action tokenizer trained on millions of trajectories across five embodiments.
4. A flow-matching continuous-action expert grafted onto a discrete-token VLM through per-layer KV-cache conditioning.
5. MolmoThink: adaptive-depth reasoning that re-predicts depth tokens only where the scene changes.

Key results

The abstract describes the most extensive empirical evaluation in the series. The detailed numbers are in the paper.

Limitations

From the paper: actions are predicted in fixed-horizon chunks (e.g. 30 steps at 30 Hz) and executed open-loop. Smoothness across chunk boundaries isn’t enforced, so velocity or acceleration jumps can appear at the seams.