Humanoids and mobile manipulation
Humanoid whole-body control and teleoperation, and mobile robots that manipulate.
Start here: seminal papers
Biped Walking Pattern Generation by using Preview Control of Zero-Moment PointSeminal
IEEE International Conference on Robotics and Automation (ICRA), September 2003
The authors model a humanoid as a cart on a table and generate its center-of-mass trajectory with a preview controller that tracks the upcoming ZMP reference. A second preview pass corrects the simple model's error on the full HRP-2P multibody model, cutting the maximum ZMP error from 2.3 to 1.2 cm (x) and from 1.6 to 0.4 cm (y), and a simulated HRP-2P walks on spiral stairs.
Zero-Moment Point: Thirty Five Years of its LifeSeminal
International Journal of Humanoid Robotics, March 2004
The originators of the Zero-Moment Point restate it as the point on the sole where the ground reaction has no horizontal moment, and give a two-step test that computes it and checks it against the support polygon. They separate the real ZMP from the center of pressure and from a "fictitious" ZMP outside the foot, which signals that the robot is tipping over.
Capture Point: A Step toward Humanoid Push RecoverySeminal
IEEE-RAS International Conference on Humanoid Robots (Humanoids), December 2006
Defines the Capture Point, the spot on the ground where a pushed biped must step to come to a complete stop, and computes it in closed form for a linear inverted pendulum with a flywheel body. In simulation, a planar biped walking at 0.5 m/s stops in a single step by lunging its body.
DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills★ Must read
April 2018
DeepMimic trains physically simulated characters with RL to imitate motion clips, rewarding closeness to the reference pose plus an optional task goal. One recipe learns 24 humanoid skills, from walking to backflips, and adding a task reward raises target-strike success from 19% (imitation only) to 99%.
AMASS: Archive of Motion Capture as Surface ShapesSeminal
April 2019
MoSh++ fits the SMPL body model, including hands and soft-tissue motion, to raw mocap markers and uses it to convert 15 mocap datasets into one archive, AMASS, with 42 hours, 346 subjects and 11,451 motions. On 4D scans recorded alongside mocap, it recovers body shape to 7.4 mm, versus 12.1 mm for the original MoSh.
Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward CompositionSeminal
November 2020
Writing gaits as periodic costs on foot forces and foot speeds gives one reward function, with no reference motions, that trains walking, running, galloping, hopping and skipping on the Cassie biped. The policies transfer from simulation to hardware, and a single policy moves between all two-beat gaits and standing.
AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control★ Must read
April 2021
AMP replaces hand-designed motion-tracking rewards with a discriminator trained on unstructured mocap clips, whose score is added as a style reward to a simple task reward in RL. With a 56-clip locomotion dataset, a simulated humanoid picks walking, jogging or running gaits on its own and reaches 0.90 normalized return on target heading, versus 0.46 and 0.63 with walk-only or run-only data.
Habitat 2.0: Training Home Assistants to Rearrange their HabitatSeminal
June 2021
Habitat 2.0 is a physics-enabled home simulator with an interactive apartment dataset (ReplicaCAD) and a rearrangement benchmark for a Fetch robot, running at 25,734 steps per second on one 8-GPU node (850x real-time). On its tasks, flat RL fails at long-horizon rearrangement, a hierarchy of RL skills does better, and sense-plan-act is more brittle.
Deep Whole-Body Control: Learning a Unified Policy for Manipulation and LocomotionSeminal
October 2022
One RL policy controls both the legs and the arm of a low-cost quadruped with an arm mounted on top, so the legs bend and stretch to extend the arm's reach. In real-world cup picking it succeeds 80% of the time on easy and hard points, versus 30% and 10% for a built-in MPC plus arm IK baseline.
Real-World Humanoid Locomotion with Reinforcement Learning★ Must read
March 2023
A causal transformer that reads the last 16 steps of proprioception and actions is trained with RL in simulation and deployed zero-shot on the full-sized Digit humanoid. Over one week of full-day outdoor testing it never fell, and it recovers from tripping on steps it never saw in training.
Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement LearningSeminal
April 2023
A 20-joint Robotis OP3 miniature humanoid learns 1v1 soccer with deep RL in simulation and plays on real robots zero-shot. Compared with the robot's scripted controllers it walks 181% faster, turns 302% faster, gets up in 63% less time and kicks 34% faster.
TidyBot: Personalized Robot Assistance with Large Language ModelsSeminal
May 2023
A user gives a few example object placements, an LLM summarizes them into general rules, and a mobile manipulator uses those rules to tidy a room. The approach reaches 91.2% accuracy on unseen objects in a text benchmark, and the real robot puts away 85.0% of objects.
Perpetual Humanoid Control for Real-time Simulated AvatarsSeminal
May 2023
A single physics-based humanoid controller (PHC) imitates 98.9% of the 11,313 AMASS training clips without any external stabilizing force, and gets up after falls to resume tracking. It drives avatars from noisy video pose estimates (about 90% success on H36M) or language-generated motion without resets.
Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body TeleoperationSeminal
January 2024
A $32k bimanual mobile manipulator whose operator, tethered to the base at the waist, drives the wheels while puppeteering both arms, plus behavior cloning co-trained on existing static ALOHA data. With 20-50 demos per task, ACT reaches 80-95% success on six of seven household tasks.
Expressive Whole-Body Control for Humanoid RobotsSeminal
February 2024
One RL policy for the Unitree H1 humanoid imitates the upper body of about 780 CMU MoCap clips while its legs only track the clip's root velocity and orientation. This beats full-body imitation on velocity tracking in simulation (reward 318.7 vs 246.1) and runs on the real robot, which dances, high-fives and shakes hands with people.
Humanoid Locomotion as Next Token PredictionSeminal
February 2024
Train a small causal transformer to predict the next observation and action of humanoid walking trajectories, masking actions where the data has none (motion capture, YouTube videos). Trained offline on 27 hours of walking data, it walks a full-sized Digit humanoid around San Francisco zero-shot and tracks commands better than the RL policy that generated its labeled data.
Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation★ Must read
March 2024
H2O trains one RL policy that makes a Unitree H1 humanoid copy a human's whole-body motion in real time from a single RGB camera, after a privileged simulated imitator filters the retargeted AMASS motions down to the ~8.5k of 10k the robot can perform. In simulation it tracks 72.5% of the retargeted motions, versus 67.9% when trained without that filtering.
HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and ManipulationSeminal
March 2024
A MuJoCo benchmark of 27 tasks (12 locomotion, 15 whole-body manipulation) for a Unitree H1 humanoid with two Shadow Hands and a 61-D action space. DreamerV3, TD-MPC2, SAC and PPO stay below the success threshold on most tasks, and every method completes 0 of 4 kitchen subtasks.
OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and LearningSeminal
June 2024
OmniH2O trains one whole-body humanoid policy that tracks only head and hand positions, so a Unitree H1 with dexterous hands can be driven by VR, an RGB camera, language or GPT-4o. It tracks 94.1% of 14k motions in simulation (H2O 87.5%), and diffusion policies trained on its 40 minutes of teleoperation data succeed 8/10 on average across four tasks.
HumanPlus: Humanoid Shadowing and Imitation from HumansSeminal
June 2024
A humanoid learns to copy a person's body and hand motion live from one RGB camera, using an RL policy trained on 40 hours of human motion data; the copied demonstrations then train vision-based skill policies. With up to 40 demonstrations, a 33-DoF humanoid completes six tasks, from folding a sweatshirt to putting on a shoe and walking, at 60-100% success.
Humanoid Parkour LearningSeminal
June 2024
One depth-vision RL policy lets a Unitree H1 humanoid jump, leap, climb stairs and clear hurdles, trained on fractal-noise terrain instead of motion references or foot-raising rewards. On the real robot it jumps onto 0.42 m platforms, leaps 0.8 m gaps and runs at 1.8 m/s.
Open-TeleVision: Teleoperation with Immersive Active Visual FeedbackSeminal
July 2024
A VR teleoperation system that streams stereo video from an actuated camera on a humanoid's head to the operator's headset, while the robot's head, arms and hands follow the operator's. ACT policies trained on 10-20 of its demonstrations reach 87-100% stage success on four long-horizon tasks on a Unitree H1.
HOVER: Versatile Neural Whole-Body Controller for Humanoid RobotsSeminal
October 2024
One humanoid policy that accepts any mix of keypoint, joint-angle and root commands, distilled from a full-body motion-imitation teacher under random command masks. On a Unitree H1 it beats the single-mode specialists it replaces, for example in 11 of 12 real-world tracking metrics.
ExBody2: Advanced Expressive Humanoid Whole-Body ControlSeminal
December 2024
ExBody2 trains a whole-body motion-tracking controller for the Unitree G1 humanoid on human motion capture, automatically dropping motions the robot cannot follow and finetuning specialists for motion groups such as dance. On the real robot it tracks selected motions with 0.107 rad mean joint error, against 0.140 rad for the best baseline.
ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body SkillsSeminal
February 2025
ASAP learns a residual "delta action" model from real humanoid rollouts so the simulator reproduces real dynamics, then fine-tunes motion-tracking policies inside it. On a Unitree G1 this cuts global tracking error from 61.2 to 50.2 mm on a kick and from 159 to 112 mm on an unseen motion.
GR00T N1: An Open Foundation Model for Generalist Humanoid RobotsSeminal
March 2025
An open 2.2B-parameter vision-language-action model for humanoids that pairs an Eagle-2 VLM with a flow-matching diffusion transformer, pre-trained on real robot data, human videos, simulation and generated videos. Post-trained on a real GR-1 humanoid, it averages 76.8% success versus 46.4% for Diffusion Policy, and 42.6% with only 10% of the data.
TWIST: Teleoperated Whole-Body Imitation SystemSeminal
May 2025
A MoCap system streams an operator's whole-body motion to a Unitree G1, and one tracking policy trained with RL plus behavior cloning turns it into balanced skills like crouching to lift a box, kicking a ball and dancing. Adding 150 in-house MoCap clips to training cuts summed tracking error on unseen motions from 2.09 to 1.52.
BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided DiffusionSeminal
August 2025
One RL motion-tracking recipe with shared hyperparameters lets a Unitree G1 reproduce cartwheels, spin-kicks, sprints and dances from human motion. A latent diffusion model distilled from these trackers is then steered at test time toward joystick, waypoint, obstacle-avoidance and keyframe tasks it never trained on. In a 77-person study, people judged its walking and running more human-like than Unitree's own controller 70.8% of the time.
Recent
Social Humanoid: Socially Responsive Human-Humanoid Interaction
November 2026
A Unitree G1 watches a person through an RGB-D camera and generates a whole-body reaction (handshake, high-five, hug) with a diffusion model over motion tokens shared by humans and the humanoid. On the real robot it succeeds in 90-100% of 30 trials per interaction, with 308 ms median latency.
Perceptive Humanoid Motion from Simulation
November 2026
HALO trains a Unitree G1 humanoid only on synthetic vision-language-motion data rendered in 1,000 procedurally generated rooms. Zero-shot, it sits, crouches, crawls, sidesteps through doors and pushes boxes in six unseen real homes, succeeding in 91% of 360 trials.
Playing Soccer Like Humans Do: Learning Versatile and Agile Humanoid Soccer via Latent Motion Priors
November 2026
SoccerBot trains one skill-conditioned RL policy that moves, dribbles, traps and kicks a ball on a Unitree G1, acting as a residual on a latent motion prior distilled from human soccer motion capture. In simulation it kicks successfully 97.0% of the time and dribbles 89.9%, and it transfers zero-shot to the real robot with motion-capture ball tracking.
Ctrl-F: Force as an Interface for Humanoid Loco-Manipulation
November 2026
Ctrl-F gives a humanoid motion tracker a 3-D force command for each hand or foot, trained without objects, plus a planner that sets those forces from the object's state. In simulation it keeps hand-task completion above 90% at 1.5x nominal object mass, where force-blind trackers drop to zero, and a real Unitree G1 pushes and kicks 20 kg boxes.
HLO-AMP: Hierarchical Loco-Manipulation from Object-Aware Motion Priors
November 2026
A Unitree G1 learns bimanual box pick-and-place from a low-level controller trained to imitate human motion capture with the box in the simulation, steered by a high-level RL policy. One deployed policy carries boxes up to 7 kg and recovers from drops. In simulation it succeeds 80.7% of the time, vs 50.3% for a whole-body controller trained without the object.
Coordinating Humanoid Loco-Manipulation through Action-Conditioned Latent Foresight
November 2026
A humanoid policy samples several whole-body and hand command sequences, predicts in latent space where each would lead, and executes the one that makes the most visual progress and is most temporally consistent. On a Unitree G1 it succeeds 85.7% on seven tasks (65.7% with unseen objects) versus 70.0% (48.6%) for GR00T N1.7.
ReflexArc: Bottom-Up Integration of Instantaneous Tactile Reflexes into Whole-Body Humanoid Control
November 2026
A full-body tactile suit on a Unitree G1 triggers whole-body withdrawal reflexes that reach the motors within 10 ms, and the learned whole-body policy is trained to keep balance without cancelling them. Compared with the same reflex delayed by 20 ms, the first force peak drops by 34.1% and the contact impulse by 2.15x.
MABEL: An Open Anthropomorphic Mobile Bimanual Platform for Teleoperation and Learning
November 2026
An open-source 56-DOF mobile bimanual robot with dexterous hands, a pitching torso, a lift and an active neck, for a US$8.1k parts cost. Apple Vision Pro head and hand tracking drives the whole body with 137 ms video latency, and an ACT policy trained through the same controller succeeds in 8 of 10 grasp-and-place trials.
FEIJOA: Task-Driven Grounding and Execution for Mobile Manipulation
November 2026
A language-model agent for mobile manipulation that grounds each instruction against a raw RGB-D scan when it arrives, and checks every skill's feasibility before running it and its effect afterwards. It resolves 94.2% of 103 open-ended grounding queries exactly (68.0% for the best prebuilt map) and completes 35 of 50 real tasks on Spot.
Pixel: A Co-Designed Full-Stack Robot System for Learning General-Purpose Mobile Manipulation
November 2026
Pixel is a wheeled humanoid that costs under $15,000. The robot, its teleoperation interfaces, a wearable human-demonstration rig, the whole-body controller and the policy were designed together, so human and robot data share one action space. Policies trained only on wearable-rig demonstrations reach 82.0% average task progress on three household tasks, and collecting those demonstrations is 4.2-6.2x faster than teleoperation.
Pixel-UMI: Learning Environment-Aware Whole-Body Manipulation from Sensorized Human Demonstrations
November 2026
Pixel-UMI turns head-and-gripper human demonstrations into whole-body robot training data by retargeting them against a 3D reconstruction of each scene, so the policy learns postures that clear obstacles. Across five real mobile-manipulation tasks it reaches 87.0% average task progress, and scene-aware retargeting alone adds 64 points.
TAMPAgent: Agentic Belief-Space Planning for Mobile Manipulation
November 2026
A language-model agent writes belief-space task-and-motion plans, and perception, grasp and motion tools must certify each action's constraints before the robot moves. On simulated kitchen tasks with an RB-Y1 mobile manipulator it reaches 77.2% success, against 57.2% for Code as Policies and 61.1% for Graph as Policy.
ReContact: Reconstruct and Retarget Human Contacts from Video for Contact-Rich Humanoid Manipulation
November 2026
ReContact turns a human video into an object-centric contact plan (which body part touches which point of the object, pressing in which direction) and trains a humanoid policy rewarded for following it. In simulation it completes a thigh-supported box regrasp into a two-palm bottom hold in 83.5% of episodes versus at most 0.16% for adapted baselines, and it runs nine tasks on a real Unitree G1.
New papers stay for 14 days.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.