Posts by Collection

digest

A New Approach to Linear Filtering and Prediction Problems

Published:

What they do Wiener’s theory specifies the optimal filter by its impulse response, which is hard to synthesize, poorly suited to machine computation, and needs a new derivation for each generalization. Kalman restates...

Hybrid Position/Force Control of Manipulators

Published:

What they do Joint sensors on an imperfectly rigid, geared arm are not precise enough for assembly, but contact forces reveal small relative position errors. Building on Mason’s framework of natural and artificial con...

Articulated Hands: Force Control and Kinematic Issues

Published:

What they do Salisbury and Craig work through the kinematic and control design of the Stanford-JPL hand: three fingers with three joints each. Motors sit on the forearm and drive each finger through four Teflon-coated...

Measure of Manipulability for Robot Manipulators

Published:

What they do Choosing an arm’s mechanism and link lengths, and where and in what posture it should work, had been done by intuition and experience. Yoshikawa earlier proposed the measure of manipulability, w = √det(J ...

A Robust Layered Control System for a Mobile Robot

Published:

What they do Mobile-robot control systems were usually built as a chain of functional modules: perception, modelling, planning, task execution and motor control. Brooks slices the problem by behavior instead, into “le...

A Computational Approach to Edge Detection

Published:

What they do Canny writes the goals of an edge detector as functionals of the filter’s impulse response and optimizes them: good detection (high signal-to-noise ratio), good localization, and one response per edge. Fo...

Adaptive Control of Mechanical Manipulators

Published:

What they do Computed torque decouples and linearizes a manipulator only if its model parameters match the real arm, yet loads and friction are often unknown and change over time. This paper adds an adaptive element: ...

A Combined Corner and Edge Detector

Published:

What they do To build a 3D model of the world from a moving monocular camera, the tracked features have to be discrete and detected the same way in every frame. Canny edges fail this test. When an edge’s response sits...

Passive Dynamic Walking

Published:

What they do Walking machines of the time generated their gait with active control. McGeer studies the other extreme: a biped whose gait comes from gravity and inertia alone. Started by hand on a shallow slope, it set...

Object Modeling by Registration of Multiple Range Images

Published:

What they do To build a complete 3D model of a physical object from several range images, the views must first be brought into one frame. Earlier systems either assumed the transformation between views was known from ...

Planning Optimal Grasps

Published:

What they do Force closure says only whether a grasp can resist every external wrench. Ferrari and Canny make it quantitative: a good grasp resists every wrench without large finger forces. They define two quality cri...

Good Features to Track

Published:

What they do A feature tracker only works if its windows can be tracked well and stay attached to one point in the world. Shi and Tomasi derive the selection rule from the tracker itself instead of from a separate “in...

A New Extension of the Kalman Filter to Nonlinear Systems

Published:

What they do The extended Kalman filter (EKF) linearises the process and observation models so that the linear Kalman filter equations apply. The authors point out two drawbacks: linearisation can make the filter unst...

Rapidly-Exploring Random Trees: A New Tool for Path Planning

Published:

What they do Probabilistic roadmaps need a local planner that connects pairs of configurations, often thousands of them. For nonholonomic and dynamical systems each connection can be as hard as designing a nonlinear c...

Monte Carlo Localization for Mobile Robots

Published:

What they do Kalman filters track a robot’s pose well but cannot represent ambiguity or relocalize after a failure. The authors’ earlier grid-based Markov localization handles arbitrary densities, but costs a lot of m...

Bundle Adjustment: A Modern Synthesis

Published:

What they do A survey of photogrammetric bundle adjustment written for computer vision implementors. Bundle adjustment refines a reconstruction so that 3D structure and camera pose and calibration are jointly optimal ...

Robotic Grasping and Contact: A Review

Published:

What they do A survey of robot grasping research in the two decades before 2000, focused on the mechanics of grasping and finger-object contact. It covers both restraining objects (fixturing) and manipulating them wit...

Robot Dynamics: Equations and Algorithms

Published:

What they do A review of robot dynamics from the recursive Newton-Euler algorithm up to 2000. It covers inverse dynamics (torques from a trajectory, for feedforward control), forward dynamics (accelerations from torqu...

A Flexible New Technique for Camera Calibration

Published:

What they do Classical calibration needs an object of known 3D geometry, usually two or three orthogonal planes. Self-calibration needs no target but is not always reliable. Zhang’s method sits in between: the camera ...

Hybrid Zero Dynamics of Planar Biped Walkers

Published:

What they do A planar biped with no ankle actuation has one more degree of freedom than actuators, and its model is hybrid: swing dynamics plus an impact map when the swing leg lands. The paper defines the hybrid zero...

Zero-Moment Point: Thirty Five Years of its Life

Published:

What they do Thirty-five years after the Zero-Moment Point was first used implicitly, its originators restate what it means. They had found imprecise or incomplete ZMP definitions in humanoid conference papers, often ...

Distinctive Image Features from Scale-Invariant Keypoints

Published:

What they do Image matching needs local features that survive changes in scale, rotation, viewpoint, illumination and noise, and are distinctive enough to find their match in a large database. This paper fully develop...

GraspIt! A Versatile Simulator for Robotic Grasping

Published:

What they do GraspIt! is a public, interactive grasping simulator. A user imports hands, arms, objects and obstacles, closes a hand on an object, and sees the grasp’s contacts, quality and weakest direction update as ...

Efficient Bipedal Robots Based on Passive-Dynamic Walkers

Published:

What they do Passive-dynamic walkers have no motors or controllers. Powered only by gravity, they walk stably down shallow slopes with humanlike gaits, but they cannot walk on level ground. The authors built three lev...

Histograms of Oriented Gradients for Human Detection

Published:

What they do Dalal and Triggs ask which image features best support detecting people, with a sliding-window linear SVM as a fixed test bed. They propose dense grids of locally normalized histograms of gradient orienta...

Visual Servo Control, Part I: Basic Approaches

Published:

What they do Visual servo control puts camera data in the loop that moves a robot. This tutorial, Part I of two, covers the basic schemes for a 6-DOF eye-in-hand camera, a fixed goal and a motionless target. Every sch...

Capture Point: A Step toward Humanoid Push Recovery

Published:

What they do A pushed humanoid has to decide whether to step and where. The paper defines a Capture Point as a ground point where, if the biped puts its base of support there and keeps its center of pressure on it, it...

Parallel Tracking and Mapping for Small AR Workspaces

Published:

What they do PTAM tracks a calibrated hand-held camera in a small, mostly static, unknown scene (such as a desk) for augmented reality, with no prior map or initialisation target. Incremental systems such as EKF-SLAM ...

Robotic Grasping of Novel Objects using Vision

Published:

What they do Grasping methods of the time needed a 3-d model of the object, which stereo rarely delivers for textureless, transparent or reflective objects. This paper skips the model: a learned classifier marks where...

iSAM: Incremental Smoothing and Mapping

Published:

What they do iSAM treats SLAM as smoothing: the full trajectory and all landmarks form one least-squares problem whose information matrix stays sparse. Rather than refactoring it at every step, as their batch square r...

g2o: A General Framework for Graph Optimization

Published:

What they do Graph-based SLAM and bundle adjustment both minimize a sum of squared errors in which each term links two state variables. That makes the problem a graph: nodes are poses or landmarks, edges are measureme...

Sampling-based Algorithms for Optimal Motion Planning

Published:

What they do PRM and RRT find a feasible path with probability approaching one, but little was known about how good that path is as samples grow. This paper proves that PRM, RRT and k-nearest PRM with fixed k do not c...

KinectFusion: Real-Time Dense Surface Mapping and Tracking

Published:

What they do KinectFusion builds a dense 3D surface model of a room in real time from a handheld Kinect, using only the depth stream and a commodity GPU. Every depth frame is fused into one global volumetric model, an...

ORB: An Efficient Alternative to SIFT or SURF

Published:

What they do SIFT is too slow for real-time systems such as visual odometry and for low-power devices such as phones. ORB (Oriented FAST and Rotated BRIEF) is a cheaper replacement built from the FAST keypoint detecto...

Brain-Computer Interface Controlled Robotic Gait Orthosis

Published:

What they do People with paraplegia from spinal cord injury (SCI) cannot walk, and the authors’ earlier EEG work only moved an avatar in virtual reality. Here a noninvasive EEG brain-computer interface (BCI) drives a ...

Deep Learning for Detecting Robotic Grasps

Published:

What they do Grasp detection from a single RGB-D image: find the oriented rectangle where a parallel-plate gripper should close. Earlier detectors ranked candidate rectangles with hand-designed features. This paper le...

Estimating Uncertain Spatial Relationships in Robotics

Published:

What they do Robots get spatial information as uncertain relative measurements in many frames. Earlier methods kept worst-case min-max error bounds, which become very conservative when many pieces are combined. The st...

Data-Driven Grasp Synthesis - A Survey

Published:

What they do A survey of data-driven grasp synthesis: methods that sample candidate grasps and rank them using experience (heuristics, simulation, human demonstration or real robot trials), as opposed to analytic meth...

Efficient Model Learning for Human-Robot Collaborative Tasks

Published:

What they do A robot working next to a person should adapt to their style, but learning a new model for every user is slow. Observing that human teams follow only a few high-level strategies, the authors learn a small...

LSD-SLAM: Large-Scale Direct Monocular SLAM

Published:

What they do Direct monocular methods align images on raw intensities rather than keypoints, but earlier ones were pure visual odometry with no global map or loop closure. LSD-SLAM builds a consistent large-scale map ...

Real-Time Grasp Detection Using Convolutional Neural Networks

Published:

What they do Grasp detection on the Cornell dataset was done with sliding-window classifiers that score many small patches, taking 13.5 s per image. This paper applies one large CNN once to the whole RGB-D image and p...

Shared Autonomy via Hindsight Optimization

Published:

What they do In shared autonomy the robot often does not know which object the user wants. Instead of predicting the most likely goal and assisting toward it (predict-then-blend), this paper treats assistance as a POM...

End-to-End Training of Deep Visuomotor Policies

Published:

What they do Robot policy search usually sits on top of a hand-built vision pipeline that is trained separately and never adapted to the task. This paper asks whether training perception and control jointly works bett...

You Only Look Once: Unified, Real-Time Object Detection

Published:

What they do Detectors such as DPM and R-CNN run a classifier over sliding windows or region proposals, with each stage trained separately, and are slow. YOLO replaces the pipeline with one convolutional network that ...

Recurrent Network Models for Human Dynamics

Published:

What they do Predicting human motion calls for a learned model of human dynamics. The Encoder-Recurrent-Decoder (ERD) puts nonlinear encoder and decoder networks before and after stacked LSTM layers and trains all thr...

Continuous control with deep reinforcement learning

Published:

What they do DQN needs a max over discrete actions; discretizing a 7-DoF arm with three levels per joint already gives 2187 actions. DDPG builds instead on the deterministic policy gradient (DPG): an actor network map...

Deep Kalman Filters

Published:

What they do A classical Kalman filter is linear with Gaussian noise. replace its transition, action and emission functions with neural networks. Every variant is learned the same way: stochastic backpropagation on a...

Structural-RNN: Deep Learning on Spatio-Temporal Graphs

Published:

What they do Many problems are posed as spatio-temporal graphs (st-graphs): nodes for components such as body parts, humans or objects, edges for their interactions in space and over time. Structural-RNN (S-RNN) is a ...

Deep Residual Learning for Image Recognition

Published:

What they do Once networks with batch normalization start converging, adding layers past some depth makes them worse: on CIFAR-10 a 56-layer plain network has higher training error than a 20-layer one, so this is not ...

Value Iteration Networks

Published:

What they do Deep RL policies are usually reactive: CNN features map straight to actions, with no planning step. A value iteration network (VIN) puts a differentiable form of value iteration inside the policy. It lear...

Revisiting Active Perception

Published:

What they do A historical review and position paper on active perception. It focuses on vision but is meant to apply to other senses. The authors define an active perceiver as an agent that “knows why it wishes to sen...

Structure-from-Motion Revisited

Published:

What they do Incremental SfM rebuilds 3D scenes from unordered photos by registering one image at a time, triangulating points and running bundle adjustment (BA). Existing systems (Bundler, VisualSFM) often leave many...

Cooperative Inverse Reinforcement Learning

Published:

What they do Inverse reinforcement learning (IRL) assumes the human acts optimally as if no robot were watching, which rules out teaching. Cooperative IRL (CIRL) is instead a two-player Markov game: the human observes...

Generative Adversarial Imitation Learning

Published:

What they do The setting is imitation from expert trajectories alone, with no expert queries and no reward. Behavioral cloning suffers from compounding errors, and inverse RL runs RL in an inner loop only to return a ...

Deep Visual Foresight for Planning Robot Motion

Published:

What they do Visual foresight replaces the usual segment-model-simulate pushing pipeline with one learned model of future camera images. The user clicks a pixel on an object and a goal location for it; the robot picks...

CAD2RL: Real Single-Image Flight without a Single Real Image

Published:

What they do Learning collision avoidance by trial and error is impractical on a fragile quadrotor. CAD2RL trains a vision-based policy entirely in simulated 3D hallways and flies it on real drones without ever seeing...

Human-Robot Mutual Adaptation in Shared Autonomy

Published:

What they do Shared-autonomy assistance usually assumes the user’s goal is right. Here the robot knows a better goal and steers the user there only if they seem willing to follow; otherwise it complies to keep their t...

Enabling Robots to Communicate their Objectives

Published:

What they do People anticipate a robot better if they understand the objective it optimizes. The robot here picks which environments to show its optimal behavior in, so that a person watching infers the right objectiv...

Mask R-CNN

Published:

What they do extends the two-stage Faster R-CNN detector to instance segmentation. Next to the existing class and box branches, each region of interest (RoI) gets a third branch: a small fully convolutional network t...

One-Shot Imitation Learning

Published:

What they do Instead of one policy per task, this paper trains a single network that takes one demonstration of a block-stacking task plus the current state and outputs an action. A demonstration of a new layout (e.g....

On human motion prediction using recurrent neural networks

Published:

What they do Deep RNN models of human motion (ERD, LSTM-3LR, SRNN) were judged on short-term angle error and on long-term plausibility. The authors show that on Human3.6M a zero-velocity baseline, which just repeats t...

Deep reinforcement learning from human preferences

Published:

What they do Many goals are hard to write as a reward or to demonstrate. Here a non-expert human instead watches pairs of short clips of the agent and picks the better one. A reward model is fit to these comparisons a...

Grasp Pose Detection in Point Clouds

Published:

What they do Grasp pose detection finds 6-DOF parallel-jaw grasps directly in a point cloud, with no object segmentation, CAD model or pose estimate. The paper combines a candidate sampler, a new grasp descriptor and ...

Hindsight Experience Replay

Published:

What they do Robot RL usually needs a carefully shaped reward. (HER) learns from a binary success signal instead. A failed attempt to reach goal g still shows how to reach the state the agent actually ended up in, so...

Emergence of Locomotion Behaviours in Rich Environments

Published:

What they do Locomotion rewards are usually hand-shaped and brittle. This paper asks whether a rich environment can replace the shaping: train simulated bodies on varied obstacle courses with a reward that is mostly f...

Proximal Policy Optimization Algorithms

Published:

What they do Reusing a batch for several policy gradient steps tends to cause destructively large updates. TRPO limits each update with a KL constraint, but it is relatively complicated and does not fit architectures ...

One-Shot Visual Imitation Learning via Meta-Learning

Published:

What they do Learning a visuomotor skill from raw pixels usually takes many demonstrations per task. Meta-imitation learning (MIL) meta-trains a policy on demonstrations of many other tasks so that a gradient update o...

Social Attention: Modeling Attention in Human Crowds

Published:

What they do A robot moving through a crowd has to predict where the people around it will walk. Earlier models such as Social LSTM assume only people in a local neighbourhood affect each other. The authors argue that...

Asymmetric Actor Critic for Image-Based Robot Learning

Published:

What they do Policies trained in simulation must run on camera images in the real world, but learning from images alone is slow because of high dimensionality and partial observability. The simulator, however, knows t...

Trajectory Deformations from Physical Human-Robot Interaction

Published:

What they do Under impedance control a human can push a robot off its path, but the robot’s desired trajectory stays fixed, so the human has to keep pushing to change its behavior. This paper lets forces applied durin...

Future Person Localization in First-Person Videos

Published:

What they do The paper introduces a new task, future person localization: from a short wearable-camera clip of a person, predict where they will appear in the next frames, as a step toward assistive navigation in crow...

Riemannian Motion Policies

Published:

What they do Reactive obstacle avoidance often just adds many local policies together, and they fight: symmetric obstacles cancel out, heavy obstacle weights make the arm sluggish, and nullspaces need hand engineering...

Open3D: A Modern Library for 3D Data Processing

Published:

What they do An image can be loaded and shown with a few lines of OpenCV, but 3D data has no equally easy library; the authors describe PCL as bloated and largely dormant. Open3D is written from a clean slate around t...

Shared Autonomy via Deep Reinforcement Learning

Published:

What they do Earlier shared-autonomy methods infer the user’s goal, but they assume known dynamics, a known set of goals and a known model of how the user acts. This paper drops all three. A model-free deep Q-learning...

World Models

Published:

What they do The agent is split into a large world model, which learns without rewards to compress game frames and predict how they evolve, and a tiny linear controller evolved on its features. In VizDoom the world mo...

Jacquard: A Large Scale Dataset for Robotic Grasp Detection

Published:

What they do Image-based grasp detectors need lots of labeled data; the main public set, Cornell, has 885 hand-labeled images. Jacquard generates the labels in simulation instead: CAD models from ShapeNetSem are dropp...

DeepIM: Deep Iterative Matching for 6D Pose Estimation

Published:

What they do RGB-only 6D pose estimators that classify or regress the pose directly are not very accurate, because small errors turn into pose mismatches. DeepIM is a refinement network. Given an image, the object’s 3...

Sim-to-Real: Learning Agile Locomotion For Quadruped Robots

Published:

What they do Policies learned in simulation tend to fail on real legged robots, because small model errors get amplified by frequent contact changes. The authors learn trotting and galloping for the Minitaur quadruped...

Motion Planning Networks

Published:

What they do Sampling-based planners such as RRT* slow down sharply as the planning problem’s dimension grows. MPNet replaces most of the search with two networks trained by imitation on RRT* paths. One encodes the ob...

Learning Dexterous In-Hand Manipulation

Published:

What they do They train a Shadow Dexterous Hand (24 DoF, tendon-driven) to reorient an object in its palm to a sequence of goal orientations, using reinforcement learning only in simulation and no human demonstrations...

PointNetGPD: Detecting Grasp Configurations from Point Sets

Published:

What they do GPD and similar grasp evaluators project the point cloud into hand-crafted 2D images for a CNN. PointNetGPD instead gives the raw 3D points inside a candidate gripper’s closing area to a PointNet-style cl...

HG-DAgger: Interactive Imitation Learning with Human Experts

Published:

What they do DAgger collects expert labels while control switches at random between the expert and the learner (the novice). With a human expert this hurts safety and label quality: the human gets little feedback from...

Expressing Robot Incapability

Published:

What they do A robot that fails a task, such as lifting a cup that is too heavy or pushing a shelf that will not move, often does not get far enough for an observer to tell what it wanted or why it stopped. This paper...

Differentiable MPC for End-to-end Planning and Control

Published:

What they do The paper treats Model Predictive Control as a policy class whose cost and dynamics are learned end-to-end from a task loss. Earlier differentiable planners unrolled the optimizer; here the derivative is ...

Learning Latent Dynamics for Planning from Pixels

Published:

What they do Learned latent models had only planned simple pixel tasks, such as balancing a cartpole from dense rewards. PlaNet is a purely model-based agent. It learns the dynamics from images, actions and rewards, t...

Residual Reinforcement Learning for Robot Control

Published:

What they do Conventional feedback controllers struggle with contact-rich manufacturing tasks because contacts and friction are hard to model, so they need manual tuning. RL can handle contacts, but learning from scra...

Learning to Walk via Deep Reinforcement Learning

Published:

What they do Deep RL on real legged robots is held back by sample cost and by hyperparameters that cannot be tuned through trial and error without breaking hardware. The authors extend soft actor-critic (SAC) so that ...

Learning agile and dynamic motor skills for legged robots

Published:

What they do ANYmal is a 32 kg quadruped with 12 series-elastic actuators, which no analytical model captures accurately enough for sim-to-real. The authors learn the actuator dynamics from robot data and train RL pol...

Learning Latent Plans from Play

Published:

What they do Instead of collecting segmented demonstrations per task, a human teleoperates a simulated 8-DOF arm in VR and plays freely with a desk that has a sliding door, a drawer, three buttons and a block. Play-LM...

Control Barrier Functions: Theory and Applications

Published:

What they do An overview of control barrier functions (CBFs), the safety counterpart of control Lyapunov functions (CLFs). Safety means a set C = {x : h(x) ≥ 0} is forward invariant: a system that starts in C never le...

AMASS: Archive of Motion Capture as Surface Shapes

Published:

What they do Public mocap datasets are small, and each uses its own markerset and skeleton, so they are hard to combine. Earlier merges retargeted everything to one fixed skeleton, which changes the data and throws aw...

Stanford Doggo: An Open-Source, Quasi-Direct-Drive Quadruped

Published:

What they do State-of-the-art legged robots take years to develop and are usually expensive custom designs. Stanford Doggo is a small quadruped (42 cm long, 4.8 kg, 8 degrees of freedom) built around a quasi-direct-dr...

Human Motion Trajectory Prediction: A Survey

Published:

What they do A survey of human motion trajectory prediction: ground-level 2D trajectories of pedestrians, plus cyclists and vehicles, for service robots, self-driving vehicles and surveillance. It covers more than 200...

Connecting Touch and Vision via Cross-Modal Prediction

Published:

What they do A camera sees the whole scene, while a GelSight sensor feels about 1.5 cm x 1.5 cm. The paper poses two tasks across this scale gap, synthesizing GelSight images from video of a robot touching an object a...

Learning Trajectory Dependencies for Human Motion Prediction

Published:

What they do Given 10 observed poses (400 ms), the task is to forecast the next 10 (400 ms) or 25 (1 s). RNN predictors accumulate errors and often jump between the last observed and the first predicted frame. This pa...

Controlling Assistive Robots with Learned Latent Actions

Published:

What they do Assistive arms have more degrees of freedom than a user can coordinate with a joystick. The paper learns latent actions: a low-DoF embedding of demonstrated robot actions, so that a 1- or 2-DoF joystick i...

Towards Variable Assistance for Lower Body Exoskeletons

Published:

What they do Crutch-less exoskeletons normally track a fixed gait rigidly, which is not ideal for patients who are recovering muscle function and should do part of the work themselves. This paper adds a variable assis...

Deep Dynamics Models for Learning Dexterous Manipulation

Published:

What they do PDDM (online planning with deep dynamics models) applies model-based RL to dexterous manipulation. It learns a neural-network model of the hand and objects and plans through it with MPC. The tasks are val...

RLBench: The Robot Learning Benchmark & Learning Environment

Published:

What they do Manipulation methods are usually tested on tasks each group builds for itself, or on control benchmarks such as OpenAI Gym and the DeepMind Control Suite, whose tasks the authors call toy problems. RLBenc...

Preference-Based Learning for Exoskeleton Gait Optimization

Published:

What they do The Atalante exoskeleton walks without crutches using gaits generated by partial hybrid zero dynamics, but nothing in that optimization captures user comfort, and users struggle to give reliable numerical...

Cable Manipulation with a Tactile-Reactive Gripper

Published:

What they do Most cable manipulation keeps the cable quasistatic or constrains it with tables, pegs or clamps. Here a free-hanging cable is followed in real time: a gripper fixed to the table holds its start, and a se...

Solving Rubik’s Cube with a Robot Hand

Published:

What they do A Shadow Dexterous Hand solves a Rubik’s cube with control and vision models trained only in simulation. A standard solver plans the moves; the learned system senses the cube’s pose and face angles and ex...

RoboNet: Large-Scale Multi-Robot Learning

Published:

What they do Most robot-learning experiments use one robot in one lab. RoboNet is an open dataset of about 162,000 trajectories (nearly 15 million frames) from 7 robots, 4 institutions and 113 camera configurations. T...

Dream to Control: Learning Behaviors by Latent Imagination

Published:

What they do Model-based agents that maximize imagined rewards over a fixed horizon tend to be shortsighted, and many rely on derivative-free planning. Dreamer learns a world model from images and then learns behavior...

OmniTact: A Multi-Directional High Resolution Touch Sensor

Published:

What they do Optical tactile sensors like GelSight give high-resolution images but only on one flat face, while curved multi-directional sensors (capacitive, resistive, photodiode arrays, BioTac) have far lower resolu...

RAFT: Recurrent All-Pairs Field Transforms for Optical Flow

Published:

What they do Most deep optical-flow networks work coarse-to-fine, so coarse errors are hard to undo and small fast-moving objects get missed. RAFT keeps one flow field at 1/8 resolution and refines it many times with ...

Learning Agile Robotic Locomotion Skills by Imitating Animals

Published:

What they do Instead of a hand-designed controller or reward per skill, an 18-DoF Laikago quadruped gets a reference motion (dog mocap or an artist’s animation), and RL in simulation trains a policy to track it. A sho...

Shared Autonomy with Learned Latent Actions

Published:

What they do Eating with an assistive arm takes coarse reaching (which plate) and precise manipulation at that goal (cutting, stabbing, scooping), all through a low-dimensional joystick. Shared autonomy helps reach di...

3D Shape Reconstruction from Vision and Touch

Published:

What they do Vision gives global shape but is blocked by the hand; touch gives exact local geometry but cannot extrapolate. This paper fuses the two for 3D shape reconstruction. A single RGB image and four touch readi...

Object Handovers: a Review for Robotics

Published:

What they do A handover is a joint action in which a giver passes an object to a receiver. This review splits it into a pre-handover phase (communication, grasping and transport, until the receiver first touches the o...

Integrated Task and Motion Planning

Published:

What they do A manipulating robot must choose actions, their continuous parameters (which grasp, where to place an object) and collision-free motions. This review treats all three as one problem, task and motion plann...

Learning Quadrupedal Locomotion over Challenging Terrain

Published:

What they do Rough-terrain legged controllers have usually been state machines that trigger reflexes from estimated contact and slip events. This paper trains a blind controller for ANYmal quadrupeds with RL in simula...

Reactive Human-to-Robot Handovers of Arbitrary Objects

Published:

What they do Prior human-to-robot handover systems were tested on a handful of objects, or needed the object to belong to a detector’s known categories. This paper builds a vision-only system that takes unknown rigid ...

ACRONYM: A Large-Scale Grasp Dataset Based on Simulation

Published:

What they do ACRONYM is a dataset for learning 6-DOF parallel-jaw grasping. It labels 2000 grasps on each of 8872 ShapeNetSem objects with the outcome of a physics simulation instead of an analytical quality measure, ...

iNeRF: Inverting Neural Radiance Fields for Pose Estimation

Published:

What they do Render-and-compare pose estimation normally needs a watertight mesh of the object; iNeRF uses a NeRF instead. Given an RGB image, known intrinsics, an initial pose guess and a trained NeRF, it freezes the...

Generation of GelSight Tactile Images for Sim2Real Learning

Published:

What they do GelSight sensors image a soft, opaque-painted membrane from the inside under coloured LEDs, which gives high-resolution touch, but no simulator produced such images. The authors build a simulated GelSight...

iMAP: Implicit Mapping and Positioning in Real-Time

Published:

What they do iMAP is a dense SLAM system for a handheld RGB-D camera. Its only scene representation is an MLP that starts from random weights and is trained live, with no prior data. A tracking process aligns each fra...

DexYCB: A Benchmark for Capturing Hand Grasping of Objects

Published:

What they do Joint hand and object pose estimation needs real data of hands handling objects, but gloves, magnetic sensors or mocap markers can change how hands move and look. DexYCB records 10 subjects picking up 20 ...

Emerging Properties in Self-Supervised Vision Transformers

Published:

What they do The authors ask whether self-supervised pretraining gives Vision Transformers (ViTs) properties that supervised ViTs and convnets lack. With DINO, self-distillation with no labels, they find two: the [CLS...

RMA: Rapid Motor Adaptation for Legged Robots

Published:

What they do Simulation-trained walking policies break when real terrain, payload or motors differ from the simulator, and collecting real data to adapt risks falls. RMA trains, entirely in simulation, a walking polic...

Implicit Behavioral Cloning

Published:

What they do Behavioral cloning policies are usually explicit networks that map an observation straight to an action. This paper changes only the form of the policy: an energy-based model scores observation-action pai...

CLIPort: What and Where Pathways for Robotic Manipulation

Published:

What they do Transporter places objects precisely but ignores language, so every new goal needs new demonstrations. CLIP grounds words in images but cannot say where to act. CLIPort combines them in a two-stream netwo...

Vision-Only Robot Navigation in a Neural Radiance World

Published:

What they do Given a NeRF of a scene trained offline, a robot plans dynamically feasible, collision-free trajectories through it and localizes itself using only an onboard RGB camera. A trajectory optimizer and a NeRF...

Ego4D: Around the World in 3,000 Hours of Egocentric Video

Published:

What they do Most video datasets are short, curated, third-person clips. Ego4D collects long, unscripted, head-mounted video of daily life: 3,670 hours from 931 participants in 74 cities and 9 countries, gathered by 1...

ReSkin: versatile, replaceable, lasting tactile skins

Published:

What they do Soft tactile skins wear out, and every replacement responds slightly differently, so a model calibrated on one skin fails on the next. ReSkin separates the cheap, wearable part (a magnetized elastomer) fr...

A System for General In-Hand Object Re-Orientation

Published:

What they do They train a multi-finger hand to rotate an object from any orientation to any goal orientation. The hand faces up, faces down over a table, or faces down with nothing below it. Everything runs on a simul...

IKFlow: Generating Diverse Inverse Kinematics Solutions

Published:

What they do An arm with 7 or more joints can reach most poses in infinitely many ways. Analytical solvers such as IKFast return every solution but only work up to 6 DOF. Numerical solvers return one solution per call...

Rapid Locomotion via Reinforcement Learning

Published:

What they do A single neural network controller for the MIT Mini Cheetah, trained with RL in simulation and deployed zero-shot, tracks commanded forward, lateral and yaw velocity up to high speeds. Sampling fast comma...

Motion Planning around Obstacles with Convex Optimization

Published:

What they do Trajectory optimization handles dynamics but relies on local solvers that can fail in cluttered spaces; sampling-based planners struggle in high dimensions and with continuous differential constraints; mi...

A Generalist Agent

Published:

What they do Gato applies the large-language-model recipe to control. Every modality (text, image patches, proprioception, joint torques, button presses) is serialized into one flat token sequence, and a single decode...

Planning with Diffusion for Flexible Behavior Synthesis

Published:

What they do Model-based RL usually learns a dynamics model and hands it to a classical trajectory optimizer, which tends to exploit the model’s errors. Diffuser instead folds planning into the generative model: a dif...

Behavior Transformers: Cloning $k$ modes with one stone

Published:

What they do Standard behavior cloning fits a unimodal policy with an MSE loss, which averages across the different ways demonstrators solve a task. BeT learns from offline demonstrations with no reward or task labels...

DayDreamer: World Models for Physical Robot Learning

Published:

What they do Deep RL needs too much trial and error for real hardware, so most robot learning goes through a simulator. This paper tests whether Dreamer, a world-model RL algorithm shown on video games and simulated t...

ActiveNeRF: Learning where to See with Uncertainty Estimation

Published:

What they do NeRF trained on few images, or on images that miss part of the scene, generalizes poorly and can collapse to zero density in the unseen regions. ActiveNeRF works under a fixed capture budget: it adds an u...

VIMA: General Robot Manipulation with Multimodal Prompts

Published:

What they do Robot tasks are usually specified in one format per model: a language instruction, a goal image or a demonstration video. VIMA expresses all of them as one multimodal prompt, an interleaved sequence of te...

Real-World Robot Learning with Masked Visual Pre-training

Published:

What they do They test whether a single vision encoder, pre-trained with self-supervision on in-the-wild images, can serve as the perception module for many real robot tasks. The encoder is trained once as a masked au...

In-Hand Object Rotation via Rapid Motor Adaptation

Published:

What they do The task is rotating an object about the z-axis using only the fingertips of a 16-DoF Allegro hand, with no vision or touch. The policy is trained entirely in simulation on cylinders and runs on the real ...

Motion Policy Networks

Published:

What they do MπNets replaces a motion planning pipeline with a single learned policy for a Franka Panda arm. From the current joint configuration and a point cloud of the scene, it predicts the next joint configuratio...

Touch and Go: Learning from Human-Collected Vision and Touch

Published:

What they do Earlier visuo-tactile datasets were collected by robots on lab tabletops or in simulation, so they cover a small number of graspable, mostly rigid objects. In Touch and Go, people collect the data instead...

VideoDex: Learning Dexterity from Internet Videos

Published:

What they do Teleoperating a dexterous hand for every task is slow, and most work uses human video only to pretrain visual features. VideoDex also takes the motion from egocentric human videos in Epic Kitchens: it con...

RT-1: Robotics Transformer for Real-World Control at Scale

Published:

What they do RT-1 is a single language-conditioned policy for a mobile manipulator (7-DoF arm, gripper, mobile base), trained by behavior cloning on about 130k human demonstrations. The data was collected by 13 robots...

PaLM-E: An Embodied Multimodal Language Model

Published:

What they do LLM planners such as SayCan see only text, so they miss scene geometry. Off-the-shelf vision-language models cannot solve robot reasoning tasks either. PaLM-E encodes images, state vectors or object-centr...

Real-World Humanoid Locomotion with Reinforcement Learning

Published:

What they do A fully learned walking controller for Digit, a 1.6 m, 45 kg humanoid from Agility Robotics. The policy is a causal transformer over the recent history of proprioceptive observations and actions, trained ...

LERF: Language Embedded Radiance Fields

Published:

What they do A NeRF captures how a scene looks but not what is in it. LERF trains a language field alongside the NeRF, supervised by CLIP embeddings of image crops. A user can then type any query, from “yellow” and “e...

Learning Human-to-Robot Handovers from Point Clouds

Published:

What they do They learn a human-to-robot handover policy from vision alone. The robot sees segmented hand and object point clouds from its wrist camera and outputs end-effector motions. Training happens in HandoverSim...

Segment Anything

Published:

What they do Segmentation has no web-scale data source, so the authors build three linked pieces: a promptable segmentation task (return a valid mask for any point, box, mask or text prompt, even an ambiguous one), a ...

DINOv2: Learning Robust Visual Features without Supervision

Published:

What they do Self-supervised image features had mostly been trained on ImageNet-1k or on large uncurated web sets, which hurt feature quality. DINOv2 tests whether self-supervised pretraining on a large curated datase...

Perpetual Humanoid Control for Real-time Simulated Avatars

Published:

What they do Simulated humanoids driven by noisy pose estimates tend to fall. Prior imitators either reset to the (unreliable) reference pose or push the root with a non-physical residual force. PHC is a single RL pol...

CoTracker: It is Better to Track Together

Published:

What they do Point trackers such as PIPs++ and TAPIR follow each query point independently, even though points on the same object move together. CoTracker estimates many tracks at once with a transformer that attends ...

A high-performance speech neuroprosthesis

Published:

What they do Participant T12 of the BrainGate2 trial has ALS and can no longer speak intelligibly. The authors implanted four 64-electrode arrays, two in area 6v (ventral premotor cortex) and two in area 44 (part of B...

3D Gaussian Splatting for Real-Time Radiance Field Rendering

Published:

What they do Novel-view synthesis from photos of a static scene. The best NeRF, Mip-NeRF360, needs 48 h of training and 10 s per frame; faster methods lose quality. 3D Gaussian Splatting replaces the continuous field ...

BridgeData V2: A Dataset for Robot Learning at Scale

Published:

What they do BridgeData V2 is a real-robot manipulation dataset built to be reused by other labs. It has 60,096 trajectories on a low-cost WidowX 250 arm, covering 13 skills, 24 environments (mostly toy kitchens, plus...

Robot Parkour Learning

Published:

What they do Parkour on low-cost quadrupeds (Unitree A1 and Go1): climbing, leaping gaps, crawling under barriers, tilting through narrow slits and running, all from one end-to-end policy that uses only onboard depth ...

General In-Hand Object Rotation with Vision and Touch

Published:

What they do Earlier sim-to-real rotation policies on the Allegro hand, such as Hora, turned objects only about the z-axis. RotateIt rotates objects continuously about the x, y or z axis using only the fingertips, wit...

Extreme Parkour with Legged Robots

Published:

What they do Parkour on a low-cost Unitree A1 with imprecise actuation and one front-facing depth camera that is low-frequency, jittery and prone to artifacts. Instead of building an elevation map and planning, a sing...

DTC: Deep Tracking Control

Published:

What they do Trajectory optimization (TO) places feet precisely but breaks when its model assumptions fail; RL is robust but struggles where valid footholds are rare, such as gaps and stepping stones. DTC has a TO pla...

Open X-Embodiment: Robotic Learning Datasets and RT-X Models

Published:

What they do Robot learning usually trains a separate policy per robot and setup. This paper tests whether one policy trained on many robots’ data can beat those specialists. Twenty-one institutions pooled 60 existing...

A Unified, Scalable Framework for Neural Population Decoding

Published:

What they do Neural recordings share no vocabulary: each session, day and animal gives a different set of neurons with no known correspondence, so decoders are usually trained per session. POYO (Pre-training On manY n...

SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM

Published:

What they do SplaTAM does dense SLAM from a single unposed RGB-D camera, using 3D Gaussians as the map. Because Gaussians rasterize fast, both camera tracking and mapping can use a dense loss over every pixel of the r...

DUSt3R: Geometric 3D Vision Made Easy

Published:

What they do Classical reconstruction chains matching, pose estimation, triangulation and multi-view stereo, and needs camera parameters before dense reconstruction. DUSt3R replaces the chain with one network: given t...

Expressive Whole-Body Control for Humanoid Robots

Published:

What they do Human motion-capture data has far more degrees of freedom than a real humanoid (69 in physics-based animation vs 19 on a Unitree H1), and animation controllers use actuator gains about ten times higher th...

Humanoid Locomotion as Next Token Prediction

Published:

What they do They treat humanoid control like language modeling: a causal transformer learns to predict the next token of sensorimotor trajectories. Because it predicts observations as well as actions, it can also lea...

DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset

Published:

What they do DROID is a teleoperated robot manipulation dataset built for scene diversity: 76k successful demonstrations (350 hours) across 564 scenes in 52 buildings and 86 tasks. Fifty data collectors at 13 institut...

Learning Visuotactile Skills with Two Multifingered Hands

Published:

What they do Instead of the usual parallel-jaw grippers, this paper fits two UR5e arms with Psyonic Ability Hands, prosthetic hands repurposed for research with custom PCBs, whose fingertips carry six touch sensors ea...

Octo: An Open-Source Generalist Robot Policy

Published:

What they do Octo is a generalist robot policy pretrained on 800k demonstrations from 25 Open X-Embodiment datasets, released fully open source: checkpoints, finetuning and pretraining code, and data loaders. It takes...

OpenVLA: An Open-Source Vision-Language-Action Model

Published:

What they do Existing VLAs such as RT-2-X are closed, and their fine-tuning to new robots was unexplored. OpenVLA is a 7B open-source VLA trained on 970k Open X-Embodiment trajectories, evaluated out of the box on two...

HumanPlus: Humanoid Shadowing and Imitation from Humans

Published:

What they do HumanPlus is a full-stack system for a humanoid to learn from human data. An RL policy trained on human motion lets the robot shadow an operator in real time from a single RGB camera. The shadowing double...

Humanoid Parkour Learning

Published:

What they do Learning-based humanoid locomotion has mostly stayed on flat ground and relied on motion references or rewards that force the feet to lift. This paper trains one end-to-end policy for the Unitree H1 that ...

SAM 2: Segment Anything in Images and Videos

Published:

What they do SAM 2 is one model for promptable segmentation of images and videos. Clicks, a box or a mask on any frame define an object; the model returns its mask on that frame, propagates it through the video as a “...

Digitizing Touch with an Artificial Multimodal Fingertip

Published:

What they do Vision-based tactile sensors like Digit and GelSight mostly capture contact geometry, at camera rates of 30-60 Hz. Digit 360, from FAIR at Meta, is a fingertip-shaped sensor that keeps a high-resolution c...

ExBody2: Advanced Expressive Humanoid Whole-Body Control

Published:

What they do ExBody2 trains one RL controller that lets a Unitree G1 humanoid track whole-body human motions (walking, crouching, dancing) from motion capture, then transfers it to the real robot. Raw mocap contains m...

Design and Control of a Bipedal Robotic Character

Published:

What they do Legged robots are usually designed for function. This one is designed for character: a small droid-like biped whose proportions, backward-bending knees and walking style were set by animators, with the me...

Brain-to-Text Decoding: A Non-invasive Approach via Typing

Published:

What they do Brain-to-text neuroprostheses work well but need brain surgery. This paper asks how far non-invasive recordings can go. 35 healthy, skilled typists read a short Spanish sentence, waited, then typed it fro...

VGGT: Visual Geometry Grounded Transformer

Published:

What they do Classical 3D reconstruction relies on iterative optimization such as bundle adjustment (BA), and learned methods like DUSt3R and MASt3R process only two images at a time, fusing pairs in post-processing. ...

RUKA: Rethinking the Design of Humanoid Hands with Learning

Published:

What they do Robot hands trade off precision, compactness, strength and cost. RUKA is an open-source tendon-driven hand with the dimensions of an average human hand, built from 3D-printed parts and off-the-shelf compo...

TWIST: Teleoperated Whole-Body Imitation System

Published:

What they do TWIST teleoperates a humanoid by having it imitate the operator’s whole-body motion in real time. OptiTrack MoCap captures the human, an online retargeter converts the pose into robot joint positions and ...

Energy-Based Policy Orchestration

Published:

What they do Hierarchical controllers usually map a planner’s target straight to the goal of one low-level policy. On a humanoid, slips, pushes or loads can make that goal infeasible faster than the planner updates. E...

SNAP: Semantic Navigation Among Pedestrians

Published:

What they do City robots must avoid pedestrians and also follow norms such as staying on sidewalks. Crowd-navigation methods usually need explicit pedestrian positions and velocities, while semantic planners treat the...

RAIPAL: A Powerful and Agile Humanoid Arm for Human Tasks

Published:

What they do Humanoid arms tend to be either strong and slow or fast and weak. RAIPAL (KAIST) is a 7-DoF, 8.9 kg arm built to be both. Its force, speed and endurance targets come from published human upper-limb measur...

Asynchronous Robot Policies

Published:

What they do A standard VLA runs its vision-language model and action expert one after the other inside the control loop. Latency therefore adds up and grows with VLM size. ARP decouples the two into asynchronous proc...

GSBR: Geometric Support Behavior Regularization for Online RL

Published:

What they do Online RL on top of a frozen diffusion or flow-matching policy is usually regularized toward it. Behavior-cloning penalties stop the actor from dropping poor modes; hard support constraints forbid anythin...

Ctrl-F: Force as an Interface for Humanoid Loco-Manipulation

Published:

What they do Humanoid motion trackers reproduce where the body should be, not how hard it should push. In the authors’ tests a force-blind tracker cannot lift 1.5-6 kg boxes because they slide off its palms. Ctrl-F ad...

Fluidic Body: Swarm Robots as a Fluid Extension of the Hand

Published:

What they do Swarm interfaces usually let a person say where the robots should go, but not how the group should behave. Fluidic Body puts a simulated fluid between the hand and a tabletop swarm. Each robot is a partic...

Dynamic Shape Control of Soft Trunk Robots

Published:

What they do Soft trunk robots meant for confined spaces need control over the shape of their whole body, not just the tip, but their compliance makes the shape dynamics strongly nonlinear. The authors learn an adiaba...

Perceptive Humanoid Motion from Simulation

Published:

What they do Humanoid policies that map pixels and language to joint commands need paired data, and teleoperation or wearable capture costs at least an hour of human time per hour of data. HALO generates that data in ...

A Unified Fractal Impedance Control Framework

Published:

What they do Fractal Impedance Control (FIC) is a passive, asymptotically stable interaction controller whose variants have so far lived in separate implementations. This paper packages them into one C++ library with ...

GPWM: Modeling the World as 3D Particle Motion

Published:

What they do Learned 3D dynamics models are usually deterministic, short-horizon and built for one kind of material. GPWM is a generative world model that represents a scene as tracked particles and rigid bodies and g...

Latent RL Priors for Data-Efficient Dexterous Manipulation

Published:

What they do RL in simulation can learn rich dexterous control, but the resulting policies rely on privileged object and goal states. Latent RL Prior distills such a policy into a flow-matching action decoder, freezes...

CS-Pi: Constraint-Satisfying Flow Policy Sampling

Published:

What they do A flow-matching policy deployed in the field often meets a constraint its demonstrations never contained, such as an obstacle next to the target. CS-Pi enforces the constraint at inference time without re...

Autonomous Whole-body Loco-manipulation Using a Snake Robot

Published:

What they do A snake robot has no separate base and arm: the body that propels it is also what touches the object. Earlier snake loco-manipulation mostly used a gripper on one end, or ran whole-body loco-manipulation ...

On Agentic Manipulation Policies

Published:

What they do Instead of training a VLA on demonstrations, AMP1 keeps a general-purpose frontier model (Gemini 4 Argon, Claude Opus 5.5 and others) in the control loop. The model writes code that looks at the cameras, ...

Zero-Shot VLA Adaptation with Retrieved Guidance

Published:

What they do Vision-language-action models degrade under small shifts in viewpoint, background, objects or embodiment. RAGu brings retrieval-augmented generation to VLAs without retraining: it pulls demonstrations of ...

Data-Driven Precision Control of Soft Helicoid Robots

Published:

What they do Soft robots are nonlinear and hard to model from first principles. This paper learns a low-dimensional model of a soft trunk robot purely from hardware data, using the theory of adiabatic spectral submani...

Flip Anything with Agentic Robotics Self-Learning

Published:

What they do Flipping an object 180 degrees on a table needs different strategies for different shapes: a hammer can be handed between two grippers, a wide multimeter has to be pushed and toppled. FlipAnything is a sh...

GAP: Generalized Adaptation Prior for Dexterous Piano Playing

Published:

What they do Piano trajectories trained in simulation lose accuracy on a real dexterous hand, and fixing them with residual RL costs a lot of hardware time. GAP learns how to adapt instead of how to act: a Gradient Gu...

Embodied First Exam

Published:

What they do Coding agents are starting to be used to control robots, but robotics benchmarks are built for visuomotor policies. The connects unmodified coding agents (a model in a harness such as Codex or Claude Cod...

Be My Eye: Learning to See Beyond a Robot’s Own Perspective

Published:

What they do In multi-robot work, one robot often sees what its partner cannot. Instead of sending raw images or generic visual features, each robot encodes its camera views into a few task-conditioned tokens that the...

SWAM: Scaling World Action Models with Cross-Modal Alignment

Published:

What they do A world action model (WAM) starts from a pretrained video generator and adds action prediction. SWAM asks which data and supervision keep the video model’s language-video alignment intact while actions ar...

World In-Context Learning for Agentic Robot Manipulation

Published:

What they do Trained manipulation policies store skill in weights and lose it under distribution shift. In-context learning from one demonstration avoids retraining, but the demonstration carries no feedback from the ...

AHEADance: Anticipatory Humanoid Dance from Streaming Audio

Published:

What they do A humanoid dancing to live music has to commit each chunk of motion before it hears the music that chunk goes with, and it cannot revise motion it has already sent to the robot. AHEADance generates refere...

A computational framework for understanding embodied plans

Published:

What they do People read goals and plans off the bodily actions of others. Existing computational accounts treat the agent as a point in a 2D grid world. GenEA (Generative Embodied Agent) is an agent model with an art...

LEASH: Stay Safe in Agentic Robotics

Published:

What they do Agentic harnesses now let foundation models such as GPT and Claude drive robot arms directly. Classical safety checks such as joint limits and collision-free paths do not catch semantic hazards like putti...

SIGMO: Sigmoid Gated Interaction-Aware Motion Planning

Published:

What they do In a narrow street or a packed merge there is often no gap that is safe on its own, so the ego vehicle has to negotiate: push in when others will yield, back off when they won’t. SIGMO adds this to a stan...

UGG: Unified Generative Geometry Model

Published:

What they do Feedforward models predict depth and camera pose from video; generative models synthesize new views. UGG treats both as queries to one generative model of a 4D scene. RGB video, depth and camera pose are ...

Genie Sim Benchmark: Real-to-Sim Fidelity and Evaluation

Published:

What they do A simulation benchmark is only useful if its scores say something about real robots. This report tests Genie Sim Benchmark, which has 78 simulated manipulation tasks covering instruction following, spatia...

Does Your World Model Know Actions Have Consequences?

Published:

What they do World models are valued because they can imagine what happens under alternative actions, yet they are usually judged by reconstruction quality, prediction error or task success. This short paper argues th...

Elastra: A Simulation Framework for Elastic Bodies

Published:

What they do Robot learning environments usually treat the ground as rigid. Mattresses, cushions and trampolines instead sink where they are pressed and recover when the load moves on, so every push changes where the ...

Autonomous Trail Racing with a Bicycle Robot

Published:

What they do Racing a two-wheeled, statically unstable robot down a mountain-bike trail couples perception and agile control. Instead of a planner sending velocity commands to a locomotion policy, the authors encode t...

ActionCodec2: A Streaming Action Tokenizer

Published:

What they do Most action tokenizers for autoregressive VLAs encode one to two seconds of motion as a chunk that must be fully decoded before any action runs, and small per-step errors add up: in LIBERO replays, FAST r...

Generalized Inverse Kinematics via Conditional Flow

Published:

What they do Planners need many IK solutions, not one: alternative base placements, starts from which a whole path stays reachable, collision-free postures in clutter. GenGenIK samples whole-body configurations, or tr...

JustRUA: A Controlled Study of GPT-6 Robot-Use Agents

Published:

What they do GPT-6 Astra can drive a bimanual arm with no robot training: a harness turns each of its decisions into end-effector targets for a motion planner. Systems built this way keep raising scores on the RoboDoj...

Alloy: A Multi-Material Physics Engine for Robot Learning

Published:

What they do Simulators for robot learning at scale are strongest on rigid bodies, and adding cloth, cables or liquids usually means a second engine plus hand-written glue. Alloy puts all of these materials in one sce...

Inner Product Regularization for Imitation Learning

Published:

What they do Policies learned by imitation often react to state changes that do not matter, such as a fruit moved on the table. Existing regularizers penalize the norm of the policy Jacobian (or Fisher information) at...

HEIR: Harness Egocentric Intent for Human-Robot Interactions

Published:

What they do A home robot has to work out what a person means (“bring me that thing”) and then carry it out over several navigation and manipulation steps. HEIR is a dataset for this. A participant wearing eye-trackin...

Natural Swing-Up of an Underpowered Double Pendulum

Published:

What they do The direct-drive motors of this planar two-link robot deliver 1.1 Nm, about 6% of the shoulder’s gravity torque. At full torque, link 2 tilts only 16.3°, so the robot must build up energy over many swings...

TouchTwin: Scaling Human Touch Data via Real-to-Sim-to-Real

Published:

What they do Real-to-sim-to-real pipelines multiply human demonstrations by rebuilding them in simulation, but they rebuild the interaction from vision. A hand pose shows where each finger stops on the surface, not ho...

Leaning Over to Reach Far-Away Objects

Published:

What they do People reach far across a table by leaning in and bracing a forearm on it. This paper gets a Unitree H1-2 to do the same with an online whole-body controller instead of a learned policy or a precomputed c...

publications

Touch-GS: Visual-Tactile Supervised 3D Gaussian Splatting

Published in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024

Touch-GS combines the power of vision and touch to generate high-quality few-shot and challenging scenes, such as few-view object centric scenes, mirrors, and transparent objects.

SPACeR: Self-Play Anchoring with Centralized Reference Models

Published in International Conference on Learning Representations (ICLR), 2026

SPACeR uses a pretrained tokenized autoregressive motion model as a centralized reference policy to guide decentralized self-play RL, matching human driving distributions with 10x faster inference and 50x fewer parame...

UniQueR: Unified Query-based Feedforward 3D Reconstruction

Published in European Conference on Computer Vision (ECCV), 2026

UniQueR formulates feedforward reconstruction as sparse 3D query inference, learning compact 3D anchor points that infer scene structure including occluded geometry in a single forward pass.

talks

PhD Forum: 4th Workshop on Proximity Perception in Robotics

Published:

At the IROS 2021 4th Workshop on Proximity Perception in Robotics, Caleb Escobedo (my colleague in the HIRO group) and I presented a talk on Contact Anticipation for Physical Human-Robot Interaction with Robotic Manip...

teaching

Teaching experience 1

Undergraduate course, University 1, Department, 2014

This is a description of a teaching experience. You can use markdown like any other post.

Teaching experience 2

Workshop, University 1, Department, 2015

This is a description of a teaching experience. You can use markdown like any other post.