Memory-Efficient Parallel Policy Learning for Collapse-Reactive Retreat via Hybrid Terrain Modeling

TL;DR Train a Go2W wheeled quadruped to back out of ground that collapses under it, by replacing collapsible terrain with per-robot virtual contact forces in simulation. This fits over 100x more parallel environments than rigid-block terrain, converges over 50x faster, and the policy transfers zero-shot to four physical collapse scenarios.
What they do
At disaster sites, ground can give way only once a robot is on it. The paper trains one proprioceptive policy for a Unitree Go2W wheeled quadruped that drives normally and, when a wheel loses support, keeps its balance and backs out to firm ground. The obstacle is training: thousands of parallel robots each need their own collapse. If they share one collapsible rigid terrain, one robot’s collapse changes another’s ground (“indirect interference”); giving every robot its own block or particle terrain exhausts GPU memory.
How it works
- Hybrid terrain modeling. Isaac Lab simulates the robot and firm terrain as rigid bodies. The collapsible region is a real hole in the floor, where virtual contact forces support the wheels.
- State-Multiplexed Contact Model. All environments share the reference geometry, a penalty contact law with regularized friction, and the state-transition rule. Each keeps only a small collapse state (trigger flags, support ratio, subsidence, slip direction), so no terrain objects are duplicated.
- Three collapse primitives: Touch (support loss after a load trigger), Sink (subsidence under load, then collapse) and Slip (a lateral push, then support loss).
- Policy. PPO with an asymmetric actor-critic: the actor sees only proprioception, while the critic also gets distances to the opening. Training uses a terrain curriculum and domain randomization.
Results
- Only this model ran 16,384 environments on one RTX 4090. Convergence took about 1/4 the time of 50 cm rigid blocks and 1/70 that of 10 cm blocks, at about 90% sim-to-sim success, comparable to the finest grid.
- Sharing one rigid terrain between just two environments caused interference in about 28% of episodes and cut success from about 70% to 50%.
- In simulation it avoided falls in over 90% of trials at every speed, compared with under 20% for estimator-triggered retreat (Naive, RTDD). It reached over 80% on unseen particle terrain.
- On the real robot, with no extra training, policies averaged over 4 of 5 retreats under Touch, Sink and Particle Slip, and fewer under Slip. Naive almost never succeeded under Touch and Sink.
Limitations
Proprioception alone can mistake a downward step for a collapse: Stepped Downhill had the most false retreats, and training steps were at most 0.08 m. How faithfully the virtual contact model matches real ground failure is not quantified. Retreat works mainly after forward entry, and virtual forces act only on the wheels.
Videos
Figures




Abstract
At disaster sites and on unstable slopes, robot contact can collapse uninspected terrain, requiring retreat upon support loss. Massively parallel reinforcement learning of collapse-reactive retreat requires independent state--action experiences. However, on shared rigid-body collapsible terrain, one robot's collapse alters others' contacts without direct interaction. We call this indirect interference. Duplicating terrain objects prevents this but is memory-intensive, limiting parallel sampling. Our key insight is that policy learning does not necessarily require rigid-body collapse modeling, which is feasible but computationally expensive; collapse-induced contact responses can suffice. We therefore propose a memory-efficient parallel policy-learning framework for collapse-reactive retreat via hybrid terrain modeling. Hybrid terrain modeling combines rigid-body simulation of non-collapsible terrain with virtual contact forces for collapsible regions. A State-Multiplexed Contact Model shares terrain geometry and a contact-response law while maintaining environment-specific collapse states, enabling independent experiences without terrain objects for collapsible regions. The framework trains a proprioceptive policy for both normal locomotion and collapse-reactive retreat. It supports over 100 times as many parallel environments as fine-resolution rigid-block terrain within the same GPU memory limit. This scalability and improved sample efficiency enable over 50 times faster policy convergence with comparable retreat performance. Sim-to-sim and real-robot experiments demonstrate retreat under multiple collapse conditions. A supplementary video is available at: https://youtu.be/_RrhwjzEPE0
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
