ToyBlocks: A controlled world for robot learning and evaluation

The Allen Institute for AI (Ai2) has introduced ToyBlocks, an open simulation framework in which a robot arm pushes, flips and stacks lettered blocks on a tabletop. Announced November 6 on Hugging Face, it is the first of what Ai2 calls FoundationWorlds: controlled environments paired with automatic task and demonstration generation and behavior metrics, so researchers can change training data, repeat tests and trace failures. Ai2 says the system runs hundreds of plans in parallel on one GPU, reaching up to 24 times real-time, without human teleoperation.

Ai2's early experiments expose clear limits. Popular action models, including ManiFlow, π0.5 and MolmoAct 2, generalized poorly at first despite large-scale pretraining. An agent using Claude Opus 5.5 solved all 30 classical block-stacking tasks, but handled only 63% of search-and-spell tasks correctly, often confusing look-alike letters or misorienting them. Letting the agent write its own skill descriptions raised success from 24% to 63%, versus 36% with descriptions from a human expert. Among world models, Wan 2.2 predicted block motion and outcomes most accurately.

Models trained only in simulation also transferred to a physical Franka arm. ManiFlow's individual skills succeeded 80.0% of the time in simulation and 81.1% in the real world, while the combined Molmo 2 and ManiFlow system managed 55.5% and 50.0%, the latter from just 30 episodes, which Ai2 cautions needs more trials.

The release includes code, two trained models and 85,336 long-horizon demonstrations covering nearly 1,600 hours. A larger dataset is targeted within weeks, giving other labs a shared, reproducible testbed for studying why robots fail.