Mind2Act World: A Diagnostic Benchmark for Embodied Mind, Physical Action, and Mind-Action Composition in Robotic Manipulation
TL;DR A simulated benchmark of 30 bimanual manipulation tasks that tests remembering (Mind), executing (Act) and combining the two (Mind2Act) separately, with 4,500 demonstrations for fine-tuning. Across nine agents, VLAs and world-action models, the best agent succeeds in 52.7% of Mind and 39.0% of Act episodes but only 12.0% of Mind2Act ones.
What they do
When a manipulation policy fails a long task, standard benchmarks cannot say whether it forgot something, executed badly, or failed to link the two. Mind2Act World splits 30 simulated tasks into three views of ten: Mind (decisions from earlier observations, with routine execution), Act (precise or timed execution of a stated operation) and Mind2Act (workflows where each physical outcome decides the next step). It then evaluates agents and end-to-end policies on the same tasks.
How it works
- Tasks. Isaac Sim with two ARX X5 arms, parallel grippers, a head camera and two wrist cameras. Each task has three difficulty levels in Clean and Random (clutter, materials, lighting) scenes, giving 180 configurations. Success comes from physical predicates in the simulator, plus a 0-100 progress score.
- Data. 4,500 scripted demonstrations (50 per task and level, 75.8 h) for fine-tuning end-to-end policies.
- Systems. Four VLAs and two world-action models are fine-tuned; three multimodal-LLM agents act through motion tools without training. Simulation pauses during inference.
- Diagnosis. Rules, an LLM judge and human review assign each failed episode’s first unrecoverable error to Mind, Act or composition, summarized in per-model profile cards.
Results
- Overall, GPT-6 Astra leads agents at 34.6% success and InternW0 leads end-to-end policies at 8.7%; the other seven systems reach 2.9-7.0%. Eight tasks are never completed in 56,700 episodes.
- On Mind2Act, GPT-6 Astra reaches 12.0%, the other two agents complete no episode, and end-to-end policies reach 0.2-3.6%; X-VLA drops from 19.6% on Act to 0.2%.
- In Random scenes the four VLAs keep 86-94% of their Act successes but only 28-41% of their Mind successes.
- Execution errors are the largest failure group: 41.4% (G0.5), 37.2% (InternW0) and 55.3% (GPT-6 Astra).
Limitations
The views use different tasks, and Mind2Act workflows are longer, so a low Mind2Act score cannot yet be separated from horizon length or an early execution error. Paused simulation means timed tasks do not test real-time response, and agents get a motion planner and four times the physics budget. All tasks are tabletop and simulated on one platform; transfer to real robots is unmeasured.
Videos
Figures



Abstract
Embodied intelligence aims to let robots perform complex physical tasks with human-like adaptability and reliability, which requires three complementary capabilities: history-based cognitive reasoning, precise and timely physical execution, and the composition of reasoning and execution through environmental feedback. Yet existing benchmarks either evaluate reasoning and execution within the same tasks or assess them separately without their composition, so they reveal little about whether a failure comes from reasoning, execution, or their composition. We introduce Mind2Act World, a diagnostic benchmark of 30 simulated manipulation tasks from five real-world application domains. Each task has three difficulty levels and two scene settings (Clean and Random), yielding 180 configurations. The tasks form three views: Mind tests decisions based on past observations, Act tests precise and timely execution, and Mind2Act combines both in closed-loop tasks where action outcomes guide later decisions. Comparing the three views shows whether a system lacks a capability or fails to compose capabilities it already has. A diagnosis harness further attributes the failed episodes of three representative systems to Mind, Act, or their composition and summarizes each system in a profile card. Our evaluation of multimodal agents, vision-language-action models (VLAs), and world-action models (WAMs) shows that competence in a single view does not carry over to Mind2Act tasks. Even the strongest agent succeeds in 52.7% of Mind and 39.0% of Act episodes but in only 12.0% of Mind2Act episodes. The strongest agent retains task state but struggles with sustained contact, whereas end-to-end policies execute practiced motions but retain little task state. These findings point to composing reasoning and execution through environmental feedback as the main open problem.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
