← Paper Digest

MoRE: Mixture-of-Experts for Multi-Physics Robotic World Models

Ziyu Zhu, Shaowei Liu, Albert J. Zhai, Henry Che, Wei-Chiu Ma, Shenlong Wang

November 2026

Real-world MPC: pushing a box to a goal pose, then moving a toy sloth by grasping it (3x speed).

TL;DR A 3D world model that predicts how object point clouds move under a robot tool and routes each interaction to one of three experts (rigid pushing, deformable pushing, deformable grasping). Without being told the interaction type it matches per-regime expert models, and in simulated MPC it succeeds in 50% of trials against 20% for DINO-WM.

What they do

Learned dynamics models usually target one material, tool or task. MoRE is one 3D world model for rigid and deformable objects, pushed or grasped. It routes each interaction to a physics-specific motion expert and serves as the dynamics model in sampling-based MPC.

How it works

  • Geometry-only inputs. Object and action are both point clouds. The tool’s points come from forward kinematics, so the model does not depend on the robot’s joints. It predicts 3 future steps from 8 history steps.
  • Hard-routed experts. A frozen Uni3D encoder and a transformer motion encoder feed a gate that picks one expert per step. The rigid expert outputs an SE(3) transform; two deformable experts (pushing, grasping) output per-point displacements.
  • Data. The experts warm up on about 20K simulated episodes. The model is then finetuned on a 1:1 mix with about 7K real episodes from DROID and PGND.
  • MPC. Random probing pushes give the gate a motion history. Candidate actions are then scored by Chamfer distance to a goal point cloud.

Results

  • Without interaction labels it matches privileged per-regime experts. Chamfer distance is 0.048 vs 0.048 for rigid pushing, 0.027 vs 0.032 for deformable pushing and 0.039 vs 0.038 for deformable grasping; AdaptiGraph and ParticleFormer score 0.052-0.143.
  • Against DINO-WM: Chamfer distance 0.048 vs 0.071, and 50% vs 20% success over 10 simulated MPC trials.
  • On DROID it beats PointWorld in all three regimes, and the gate picks the right expert 96.7% of the time.
  • In real box pushing and toy-sloth pose matching it reaches lower error than a heuristic MPC and a single-transformer model (reported only as plots).

Limitations

It handles one manipulated object at a time and does not model mass, friction or elasticity, so an empty and a full bottle look alike. The gate needs probing interactions before planning, and errors accumulate over long rollouts.

Videos

Real-world dynamics predictions for rigid and soft objects in one scene, from a single model.
MPC rollouts in simulation, where a tool pushes household objects toward goal poses.

Figures

From Figure 1: MoRE predicts action-conditioned 3D dynamics by routing tool-object interactions to specialized motion...
From Figure 1: MoRE predicts action-conditioned 3D dynamics by routing tool-object interactions to specialized motion experts, covering rigid and deformable objects under prehensile and non-prehensile contact.
From Figure 2: Current and past object states and actions, all point clouds, are encoded into point tokens by two enc...
From Figure 2: Current and past object states and actions, all point clouds, are encoded into point tokens by two encoders. A gating network then selects one motion expert per physical regime to predict the next 3D object state. P = prehensile, NP = non-prehensile.
From Figure 4: Real-world MPC on (a) box pushing and (b) toy manipulation. Left: error across planning steps; right: ...
From Figure 4: Real-world MPC on (a) box pushing and (b) toy manipulation. Left: error across planning steps; right: success rate versus error threshold. MoRE reaches lower error and higher success than the baselines.
Abstract

A robotic world model should predict how objects evolve under robot actions and physical dynamics. While recent 3D world models have shown strong predictive capabilities, they still struggle with the heterogeneous dynamics arising from diverse object interactions. To address this in a unified framework, we decompose robotic interactions into distinct physical regimes and assign each to a specialized motion expert. We introduce MoRE, a 3D world model with a mixture of motion experts organized around rigid, deformable, and prehensile dynamics. This design enables a single model to capture diverse physical behaviors while preserving regime-specific priors. Trained on mixed simulation and real-world data, MoRE integrates with Model Predictive Control (MPC) to support diverse manipulation tasks. Experiments show that MoRE improves prediction accuracy across physics regimes, leading to stronger real-world MPC performance. Code, data, and project website are available at https://alwaysleepy.github.io/MoRE-WM-site/.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.