PUPPET: Physical-Unified Platform for Presence, Embodiment, and Teleoperation

TL;DR PUPPET lets a remote user drive a real TIAGo mobile manipulator from inside a VR replica of the lab, kept in sync by motion capture, and automatically checks each trial's recorded data. In a 30-person study, VR cut post-learning task time to 68.7 s from 95.0 s with 2D camera teleoperation and raised validation-ready trials from 66.0% to 85.6%.
What they do
A teleoperated trial can finish its task and still leave an incomplete or inconsistent data record. PUPPET opens a physical robot lab to remote users through a continuously synchronized VR replica of the robot, workspace and task objects, and checks every trial’s record automatically. A within-subject study compares it with in-person and 2D camera-based operation of the same robot and task.
How it works
- Embodied control. A Meta Quest 3 user’s tracked hand drives the TIAGo end-effector, headset orientation drives the robot’s head and a joystick drives the base. The server closes the loop at 100 Hz over WiFi 6, with a median command-to-motion feedback latency of 30.6 ms.
- Synchronization. OptiTrack motion capture (120 Hz) tracks the robot base and task objects and streams their poses into the Unity scene, with a median physical-to-virtual delay of 7.15 ms.
- Validation. Each trial is saved as a ROS bag plus 30 fps multi-view video. Six rule families check completeness, temporal continuity, signal plausibility and execution consistency, and mark each trial Pass, Review or Fail.
Results
Thirty participants each moved a cylinder from a table to a shelf three times per condition, in counterbalanced order:
- Post-learning completion time was 68.66 s in VR, 94.96 s with 2D teleoperation and 72.10 s in person, a 27.7% reduction from 2D (p = .011). Every trial in every condition completed the task.
- 85.6% of VR trials were validation-ready versus 66.0% for 2D (no trial failed outright). 89.7% of post-learning VR trials needed no technician intervention, versus 67.2% for 2D and 93.1% in person.
- Compared with 2D, VR lowered NASA-TLX workload (2.26 vs 2.81) and raised usability (3.78 vs 2.43) and task confidence (4.43 vs 3.50).
Limitations
The conditions differ in both view and control scheme (2D and in-person used keyboard and joystick), so the effects of spatial visualization and embodied control cannot be separated. The study uses one robot, one task and one lab. The virtual lab is modeled by hand and depends on motion capture, and marker loss caused tracking anomalies in 6.7% of VR trials.
Figures




Abstract
Remote access to robotics hardware does not by itself guarantee usable experimental data. A teleoperated trial can complete the task and still produce an incomplete or inconsistent record. PUPPET aims to solve this problem by presenting a cyber-physical framework that opens a physical robotics laboratory through a continuously synchronized VR replica of the robot, stranding environment, and task objects. In addition, PUPPET integrates a trial validation pipeline built into the experimental workflow. We evaluated PUPPET in a within-subject study with 30 participants, who performed the same table-to-shelf object-transfer task in person, through 2D teleoperation, and through PUPPET. VR reduced post-learning completion time by 27.7% over 2D (68.7 s vs. 95.0 s), statistically indistinguishable from in-person execution (72.1 s). It also produced more validation-ready (85.6% vs. 66.0%) and intervention-free (89.7% vs. 67.2%) trials. With PUPPET we show that modality shapes the remote task performance and the reusability of the resulting experimental record.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
