Matt Strong

Matt Strong

CS PhD at Stanford University

← Paper Digest

ReContact: Reconstruct and Retarget Human Contacts from Video for Contact-Rich Humanoid Manipulation

Zhongkai Gu, Yang Liu, Wei Zhu, Mitsuhiro Hayashibe

November 2026

Real G1, front and side views, holding the box on its raised thigh during the regrasp.

TL;DR ReContact turns a human video into an object-centric contact plan (which body part touches which point of the object, pressing in which direction) and trains a humanoid policy rewarded for following it. In simulation it completes a thigh-supported box regrasp into a two-palm bottom hold in 83.5% of episodes versus at most 0.16% for adapted baselines, and it runs nine tasks on a real Unitree G1.

What they do

Humanoids often pass an object’s support between body parts, e.g. resting a box on a raised thigh while the palms slide underneath. Copying such contact transitions from human video fails when the body hides the object, and a policy that tracks the object trajectory can still hold it at the wrong points. ReContact makes the contacts explicit in an object-centric contact plan.

How it works

  • Contact plan. Per phase, the active robot-link/object pairs, each with an anchor on the object, a pressing direction and a stick or slide mode. GPT-6 Astra drafts it from exocentric video; egocentric plans come from reconstructed hand-object contacts.
  • Reconstruction and retargeting. Contact priors tie the object pose to the supporting body part through occlusion; whole-body optimization fits the G1 while keeping the anchors and directions.
  • Control. A residual policy over the frozen SONIC whole-body tracker, trained with PPO, is rewarded for contact near each anchor with force along the planned direction.
  • Task families. The placement-dependent segment is re-solved at new object placements, and one policy trains on all of them.

Results

  • Thigh-supported regrasp in simulation: 83.5 ± 5.2% two-palm success over three seeds. Adapted baselines complete 24.3-65.7% of episodes but form the hold in at most 0.16%, squeezing the box between the hands instead.
  • Tube transfer: one policy on 247 re-solved references reaches 97.8% success across the workspace, against 20.4% for the best single-reference policy.
  • Real Unitree G1: seven contact-rich manipulation and two loco-manipulation tasks, including a chain of table pickup, regrasp, about 3 m of loaded walking, reverse regrasp and floor placement.

Limitations

The paper reports no success rates for the hardware tasks, where object poses come from fiducial markers, the head camera or poses latched at start. Each task family needs its own residual policy.

Videos

Real G1 with a box at a table, then walking with it under a safety gantry.
Rendered simulation of the G1 taking a box from a table and supporting it on its thigh.

Figures

From Figure 1: Overview of ReContact. (a) An exocentric video supplies body and object motion. (b) GPT-6 Astra review...
From Figure 1: Overview of ReContact. (a) An exocentric video supplies body and object motion. (b) GPT-6 Astra reviews sparse keyframes against the video, and contact-constrained whole-body optimization turns them into humanoid-object references. (c) The object-centric contact plan for the thigh-supported regrasp lists, per phase, the paired robot and object surfaces with their contact points, contact directions and sticking or sliding modes. An objective that does not specify the pressing direction also accepts a side squeeze. (d) The plan defines the rewards of a residual controller for contact at the planned point and along the planned direction. (e) Re-solving the placement-dependent segment at new object placements gives a task family for one policy.
From Figure 4: Only ReContact moves the box from thigh support to a two-palm bottom hold. Vertical contact force on t...
From Figure 4: Only ReContact moves the box from thigh support to a two-palm bottom hold. Vertical contact force on the box from the right thigh and each palm, divided by the box weight, in 48 episodes per controller; lines are medians and bands interquartile ranges. Shading marks the palm slide and the final bottom hold. Panel titles give two-palm successes.
From Figure 7: Workspace coverage from one reference in the tube-transfer task. Top: success of the task-family polic...
From Figure 7: Workspace coverage from one reference in the tube-transfer task. Top: success of the task-family policy inside the generated workspace (outline) and in the 6 cm band outside it; white points mark the 247 references and the star the source tube position. Bottom: per-position outcomes of policies trained on the source reference alone (local starts, workspace-wide starts, no start randomization) and on the family.
Abstract

Contact transitions allow humanoid robots to redistribute object support across the whole body, enabling manipulation beyond hand-only grasps. Human videos provide a scalable source of such transitions for robots to learn from. However, body occlusion disrupts object pose tracking and corrupts retargeted motion. Moreover, accurately reproducing the object trajectory does not ensure that the robot applies stabilizing forces at the intended contacts, risking slips or drops in later phases. To address these challenges, we propose ReContact, which reconstructs and retargets human-object interactions through an object-centric contact plan. The plan specifies contact phases, contact pairs, anchors, pressing directions, and sticking or sliding modes. Contact priors stabilize object pose reconstruction through occlusion. A residual policy over a frozen whole-body tracker is rewarded for making contact at planned anchors with forces aligned to the specified directions. In simulation, on a thigh-supported regrasp task, ReContact achieves the planned two-palm bottom hold in 83.5 ± 5.2% of episodes over three training seeds, versus 0.16% for the best adapted baseline. One policy trained on 247 references re-solved for new placements raises in-workspace tube-transfer success from 20.4% (source reference only) to 97.8%. On a real Unitree G1, policies execute seven contact-rich manipulation and two loco-manipulation tasks.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.