Sitemap
A list of all the posts and pages found on the site. For you robots out there, there’s an XML version available for digesting as well.
Pages
Page Not Found
Page not found. Your pixels are in another canvas.
Page not in menu
This is a page not in th emain menu
Posts
11 Lessons from 2025
Published:
10 Lessons from 2024
Published:
10 Lessons from 2023
Published:
My Love Letter to Advanta
Published:
Memories from Advanta
Running Poem
Published:
Run!
Welcome
Published:
to my personal blog! I will be posting about my research, art, photography, personal musings, and more!
digest
A New Approach to Linear Filtering and Prediction Problems
Published:
What they do Wiener’s theory specifies the optimal filter by its impulse response, which is hard to synthesize, poorly suited to machine computation, and needs a new derivation for each generalization. Kalman restates...
A Formal Basis for the Heuristic Determination of Minimum Cost Paths
Published:
What they do Minimum-cost path problems (telephone routing, mazes, circuit layout, theorem proving) had been attacked either mathematically, with guaranteed algorithms that use no knowledge of the problem, or heuristi...
Analysis of the Computed Torque Drive Method and Comparison with Conventional Position Servo for a Computer-Controlled Manipulator
Published:
What they do JPL was building a manipulator modeled on the Stanford arm and studied how to control its joints. This memo analyzes one joint under Stanford’s “computed torque” method: the planning program computes the ...
Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography
Published:
What they do Least squares fits all the data and relies on good points averaging out bad ones; even when the worst residual is deleted repeatedly, one gross error can survive (Figure 1). RANSAC reverses the order: fit...
Hybrid Position/Force Control of Manipulators
Published:
What they do Joint sensors on an imperfectly rigid, geared arm are not precise enough for assembly, but contact forces reveal small relative position errors. Building on Mason’s framework of natural and artificial con...
An Iterative Image Registration Technique with an Application to Stereo Vision
Published:
What they do Registering two images means finding the displacement h that makes F(x + h) best match G(x) over a region R. Exhaustive search over an MxM range of displacements costs O(M²N²) for an NxN image. Lucas and ...
Articulated Hands: Force Control and Kinematic Issues
Published:
What they do Salisbury and Craig work through the kinematic and control design of the Stanford-JPL hand: three fingers with three joints each. Motors sit on the forearm and drive each finger through four Teflon-coated...
Measure of Manipulability for Robot Manipulators
Published:
What they do Choosing an arm’s mechanism and link lengths, and where and in what posture it should work, had been done by intuition and experience. Yoshikawa earlier proposed the measure of manipulability, w = √det(J ...
Impedance Control: An Approach to Manipulation, Part I: Theory
Published:
What they do This is Part I of a three-part paper that sets out impedance control as one framework for manipulation. The framework covers tasks such as drilling, grinding or using a tool, where the manipulator does wo...
Obstacle Avoidance for Kinematically Redundant Manipulators in Dynamically Varying Environments
Published:
What they do Earlier obstacle-avoidance methods define a task by its start and goal configurations, which suits pick-and-place but not spray painting or arc welding, where the end-effector must follow a set trajectory...
A Robust Layered Control System for a Mobile Robot
Published:
What they do Mobile-robot control systems were usually built as a chain of functional modules: perception, modelling, planning, task execution and motor control. Brooks slices the problem by behavior instead, into “le...
Real-Time Obstacle Avoidance for Manipulators and Mobile Robots
Published:
What they do Collision avoidance was usually treated as path planning, which runs orders of magnitude slower than the robot can respond. Khatib moves it into low-level control. The goal attracts the end effector and o...
A Computational Approach to Edge Detection
Published:
What they do Canny writes the goals of an edge detector as functionals of the filter’s impulse response and optimizes them: good detection (high signal-to-noise ratio), good localization, and one response per edge. Fo...
A Unified Approach for Motion and Force Control of Robot Manipulators: The Operational Space Formulation
Published:
What they do Robot dynamics and control were mostly written in joint space, while tasks, contact forces and force sensing are all defined at the end-effector. Khatib builds the equations of motion of the end-effector ...
Adaptive Control of Mechanical Manipulators
Published:
What they do Computed torque decouples and linearizes a manipulator only if its model parameters match the real arm, yet loads and friction are often unknown and change over time. This paper adds an adaptive element: ...
A Combined Corner and Edge Detector
Published:
What they do To build a 3D model of the world from a moving monocular camera, the tracked features have to be discrete and detected the same way in every frame. Canny edges fail this test. When an edge’s response sits...
Numerical Filtering for the Operation of Robotic Manipulators through Kinematically Singular Configurations
Published:
What they do Near a kinematic singularity, the pseudoinverse solution of resolved-rate control jumps discontinuously and demands very high joint velocities. Damped least squares (DLS) bounds the solution at the cost o...
Passive Dynamic Walking
Published:
What they do Walking machines of the time generated their gait with active control. McGeer studies the other extreme: a biped whose gait comes from gravity and inertia alone. Started by hand on a shallow slope, it set...
Object Modeling by Registration of Multiple Range Images
Published:
What they do To build a complete 3D model of a physical object from several range images, the views must first be brought into one frame. Earlier systems either assumed the transformation between views was known from ...
Planning Optimal Grasps
Published:
What they do Force closure says only whether a grasp can resist every external wrench. Ferrari and Canny make it quantitative: a good grasp resists every wrench without large finger forces. They define two quality cri...
Good Features to Track
Published:
What they do A feature tracker only works if its windows can be tracked well and stay attached to one point in the world. Shi and Tomasi derive the selection rule from the tracker itself instead of from a separate “in...
Probabilistic Roadmaps for Path Planning in High-Dimensional Configuration Spaces
Published:
What they do Robots with five or more dof are hard to plan for: complete planners are too expensive, and potential-field planners such as the Randomized Path Planner (RPP) get stuck in local minima behind narrow passa...
A New Extension of the Kalman Filter to Nonlinear Systems
Published:
What they do The extended Kalman filter (EKF) linearises the process and observation models so that the linear Kalman filter equations apply. The authors point out two drawbacks: linearisation can make the filter unst...
Rapidly-Exploring Random Trees: A New Tool for Path Planning
Published:
What they do Probabilistic roadmaps need a local planner that connects pairs of configurations, often thousands of them. For nonholonomic and dynamical systems each connection can be as hard as designing a nonlinear c...
Monte Carlo Localization for Mobile Robots
Published:
What they do Kalman filters track a robot’s pose well but cannot represent ambiguity or relocalize after a failure. The authors’ earlier grid-based Markov localization handles arbitrary densities, but costs a lot of m...
Bundle Adjustment: A Modern Synthesis
Published:
What they do A survey of photogrammetric bundle adjustment written for computer vision implementors. Bundle adjustment refines a reconstruction so that 3D structure and camera pose and calibration are jointly optimal ...
Templates and Anchors: Neuromechanical Hypotheses of Legged Locomotion on Land
Published:
What they do A biologist and a roboticist ask how to model legged locomotion when animals have far more joints, muscles and neurons than the six “payload” dimensions of the body need. A template is the simplest model ...
Robotic Grasping and Contact: A Review
Published:
What they do A survey of robot grasping research in the two decades before 2000, focused on the mechanics of grasping and finger-object contact. It covers both restraining objects (fixturing) and manipulating them wit...
Robot Dynamics: Equations and Algorithms
Published:
What they do A review of robot dynamics from the recursive Newton-Euler algorithm up to 2000. It covers inverse dynamics (torques from a trajectory, for feedforward control), forward dynamics (accelerations from torqu...
RRT-Connect: An Efficient Approach to Single-Query Path Planning
Published:
What they do Randomized potential fields answer single queries fast but get stuck in local minima; probabilistic roadmaps are reliable but built to preprocess an environment for many queries. RRT-Connect handles singl...
A Flexible New Technique for Camera Calibration
Published:
What they do Classical calibration needs an object of known 3D geometry, usually two or three orthogonal planes. Self-calibration needs no target but is not always reliable. Zhang’s method sits in between: the camera ...
Rapid Object Detection using a Boosted Cascade of Simple Features
Published:
What they do A frontal face detector built to be fast on a single greyscale image, with no frame differencing or skin color. It combines an integral image that makes rectangle features cheap, AdaBoost to pick a few fe...
FastSLAM: A Factored Solution to the Simultaneous Localization and Mapping Problem
Published:
What they do In EKF-based SLAM, each sensor update takes time quadratic in the number of landmarks K, which limits maps to a few hundred landmarks. FastSLAM relies on a conditional independence: once the robot’s path ...
Hybrid Zero Dynamics of Planar Biped Walkers
Published:
What they do A planar biped with no ankle actuation has one more degree of freedom than actuators, and its model is hybrid: swing dynamics plus an impact map when the swing leg lands. The paper defines the hybrid zero...
Biped Walking Pattern Generation by using Preview Control of Zero-Moment Point
Published:
What they do A walking pattern generator must find a center-of-mass (CoM) motion whose ZMP follows a reference set by the footholds and step period. Earlier solvers (FFT, an O(N) trinomial method) work in batch over f...
Zero-Moment Point: Thirty Five Years of its Life
Published:
What they do Thirty-five years after the Zero-Moment Point was first used implicitly, its originators restate what it means. They had found imprecise or incomplete ZMP definitions in humanoid conference papers, often ...
Distinctive Image Features from Scale-Invariant Keypoints
Published:
What they do Image matching needs local features that survive changes in scale, rotation, viewpoint, illumination and noise, and are distinctive enough to find their match in a large database. This paper fully develop...
GraspIt! A Versatile Simulator for Robotic Grasping
Published:
What they do GraspIt! is a public, interactive grasping simulator. A user imports hands, arms, objects and obstacles, closes a hand on an object, and sees the grasp’s contacts, quality and weakest direction update as ...
Efficient Bipedal Robots Based on Passive-Dynamic Walkers
Published:
What they do Passive-dynamic walkers have no motors or controllers. Powered only by gravity, they walk stably down shallow slopes with humanlike gaits, but they cannot walk on level ground. The authors built three lev...
Histograms of Oriented Gradients for Human Detection
Published:
What they do Dalal and Triggs ask which image features best support detecting people, with a sliding-window linear SVM as a fixed test bed. They propose dense grids of locally normalized histograms of gradient orienta...
The GraphSLAM Algorithm with Applications to Large-Scale Mapping of Urban Structures
Published:
What they do GraphSLAM solves the offline (full) SLAM problem: recover the robot’s whole path and the map together after all data is collected. Each control and each feature measurement becomes a soft constraint, an e...
Compliant Leg Behaviour Explains Basic Dynamics of Walking and Running
Published:
What they do Walking has been modelled as vaulting over stiff legs (the inverted pendulum) and running as bouncing on a compliant leg (the spring-mass model). The stiff-leg model cannot produce walking’s M-shaped vert...
Visual Servo Control, Part I: Basic Approaches
Published:
What they do Visual servo control puts camera data in the loop that moves a robot. This tutorial, Part I of two, covers the basic schemes for a 6-DOF eye-in-hand camera, a fixed goal and a motionless target. Every sch...
Capture Point: A Step toward Humanoid Push Recovery
Published:
What they do A pushed humanoid has to decide whether to step and where. The paper defines a Capture Point as a ground point where, if the biped puts its base of support there and keeps its center of pressure on it, it...
Parallel Tracking and Mapping for Small AR Workspaces
Published:
What they do PTAM tracks a calibrated hand-held camera in a small, mostly static, unknown scene (such as a desk) for augmented reality, with no prior map or initialisation target. Incremental systems such as EKF-SLAM ...
Robotic Grasping of Novel Objects using Vision
Published:
What they do Grasping methods of the time needed a 3-d model of the object, which stereo rarely delivers for textureless, transparent or reflective objects. This paper skips the model: a learned classifier marks where...
iSAM: Incremental Smoothing and Mapping
Published:
What they do iSAM treats SLAM as smoothing: the full trajectory and all landmarks form one least-squares problem whose information matrix stays sparse. Rather than refactoring it at every step, as their batch square r...
CHOMP: Gradient Optimization Techniques for Efficient Motion Planning
Published:
What they do Sampling-based planners such as PRM and RRT find a feasible path, then need a separate smoothing step. CHOMP (Covariant Hamiltonian Optimization for Motion Planning) optimizes the trajectory directly for ...
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Published:
What they do Supervised imitation trains only on states the expert visits; once the learned policy errs it reaches states the expert never showed, and mistakes compound to as much as T²ε extra cost over T steps for pe...
g2o: A General Framework for Graph Optimization
Published:
What they do Graph-based SLAM and bundle adjustment both minimize a sum of squared errors in which each term links two state variables. That makes the problem a graph: nodes are poses or landmarks, edges are measureme...
Sampling-based Algorithms for Optimal Motion Planning
Published:
What they do PRM and RRT find a feasible path with probability approaching one, but little was known about how good that path is as samples grow. This paper proves that PRM, RRT and k-nearest PRM with fixed k do not c...
Cutaneous Force Feedback as a Sensory Subtraction Technique in Haptics
Published:
What they do Communication delay can destabilize kinesthetic force feedback in teleoperation. This paper drops the kinesthetic part and keeps only the cutaneous part: wearable devices press on the thumb and index fing...
KinectFusion: Real-Time Dense Surface Mapping and Tracking
Published:
What they do KinectFusion builds a dense 3D surface model of a room in real time from a handheld Kinect, using only the depth stream and a commodity GPU. Every depth frame is fused into one global volumetric model, an...
ORB: An Efficient Alternative to SIFT or SURF
Published:
What they do SIFT is too slow for real-time systems such as visual odometry and for low-power devices such as phones. ORB (Oriented FAST and Rotated BRIEF) is a cheaper replacement built from the FAST keypoint detecto...
Reach and grasp by people with tetraplegia using a neurally controlled robotic arm
Published:
What they do Two people with tetraplegia after brainstem stroke (S3 and T2) controlled a robot arm and hand in 3D with neural activity from a 96-channel microelectrode array in motor cortex. S3 used the DLR Light-Weig...
Brain-Computer Interface Controlled Robotic Gait Orthosis
Published:
What they do People with paraplegia from spinal cord injury (SCI) cannot walk, and the authors’ earlier EEG work only moved an avatar in virtual reality. Here a noninvasive EEG brain-computer interface (BCI) drives a ...
ImageNet Classification with Deep Convolutional Neural Networks
Published:
What they do Train a large, deep CNN, purely supervised, on the 1.2 million images of ImageNet LSVRC-2010 (1000 classes). An optimized GPU implementation of convolution and a split across two GPUs make a network with ...
Deep Learning for Detecting Robotic Grasps
Published:
What they do Grasp detection from a single RGB-D image: find the oriented rectangle where a parallel-plate gripper should close. Earlier detectors ranked candidate rectangles with hand-designed features. This paper le...
Occupancy Grids: A Stochastic Spatial Representation for Active Robot Perception
Published:
What they do Geometric-feature maps built with prior models and heuristics are, Elfes argues, sparse, brittle and blind to sensor uncertainty. The Occupancy Grid instead divides space into cells, each storing the prob...
Estimating Uncertain Spatial Relationships in Robotics
Published:
What they do Robots get spatial information as uncertain relative measurements in many frames. Earlier methods kept worst-case min-max error bounds, which become very conservative when many pieces are combined. The st...
Finding Locally Optimal, Collision-Free Trajectories with Sequential Convex Optimization
Published:
What they do Trajopt treats motion planning as trajectory optimization from scratch. It starts from a trajectory that may pass deep through obstacles and optimizes it into a collision-free, locally optimal path. Colli...
Fast Marching Tree: a Fast Marching Sampling-Based Method for Optimal Motion Planning in Many Dimensions
Published:
What they do FMT* is an asymptotically optimal sampling-based motion planner for high-dimensional configuration spaces. It runs a “lazy” dynamic programming recursion over a fixed set of samples, growing a tree of pat...
Data-Driven Grasp Synthesis - A Survey
Published:
What they do A survey of data-driven grasp synthesis: methods that sample candidate grasps and rank them using experience (heuristics, simulation, human demonstration or real robot trials), as opposed to analytic meth...
Informed RRT*: Optimal Sampling-based Path Planning Focused via Direct Sampling of an Admissible Ellipsoidal Heuristic
Published:
What they do With uniform sampling, RRT* ends up optimizing the path to every state in the space, which is wasted effort in a single-query problem. When the cost is path length, only states inside an ellipsoid with th...
Batch Informed Trees (BIT*): Sampling-based Optimal Planning via the Heuristically Guided Search of Implicit Random Geometric Graphs
Published:
What they do A* searches in order of solution quality but needs a fixed discretization; RRT* scales to high dimensions but searches in random order. BIT* treats a batch of random samples as an implicit random geometri...
Efficient Model Learning for Human-Robot Collaborative Tasks
Published:
What they do A robot working next to a person should adapt to their style, but learning a new model for every user is slow. Observing that human teams follow only a few high-level strategies, the authors learn a small...
LSD-SLAM: Large-Scale Direct Monocular SLAM
Published:
What they do Direct monocular methods align images on raw intensities rather than keypoints, but earlier ones were pure visual odometry with no global map or loop closure. LSD-SLAM builds a consistent large-scale map ...
Real-Time Grasp Detection Using Convolutional Neural Networks
Published:
What they do Grasp detection on the Cornell dataset was done with sliding-window classifiers that score many small patches, taking 13.5 s per image. This paper applies one large CNN once to the whole RGB-D image and p...
Autonomy Infused Teleoperation with Application to BCI Manipulation
Published:
What they do Commands decoded from an intracortical brain-computer interface are noisy and low-dimensional: here 3D hand translation plus a grasp signal. The system recognizes objects, infers which grasp the user is r...
Shared Autonomy via Hindsight Optimization
Published:
What they do In shared autonomy the robot often does not know which object the user wants. Instead of predicting the most likely goal and assisting toward it (predict-then-blend), this paper treats assistance as a POM...
End-to-End Training of Deep Visuomotor Policies
Published:
What they do Robot policy search usually sits on top of a hand-built vision pipeline that is trained separately and never adapted to the task. This paper asks whether training perception and control jointly works bett...
U-Net: Convolutional Networks for Biomedical Image Segmentation
Published:
What they do Biomedical segmentation needs a label for every pixel, and thousands of annotated training images are usually out of reach. The ISBI 2012 EM winner classified each pixel from a sliding-window patch, which...
Reducing the energy cost of human walking using an unpowered exoskeleton
Published:
What they do Earlier devices lowered the energy cost of walking by adding power from pumps or motors. This paper asks whether a device with no energy source can. The authors built an ankle exoskeleton whose spring, he...
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Published:
What they do By 2015, region-based detectors like Fast R-CNN ran in a fraction of a second on a GPU, but the proposals they relied on came from CPU methods: Selective Search at about 2 s per image, EdgeBoxes at 0.2 s....
You Only Look Once: Unified, Real-Time Object Detection
Published:
What they do Detectors such as DPM and R-CNN run a classifier over sliding windows or region proposals, with each stage trained separately, and are slow. YOLO replaces the pipeline with one convolutional network that ...
Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images
Published:
What they do Locally optimal controllers like iLQG need a known model or a hand-designed low-dimensional state. Embed to Control (E2C) learns that state from images. Instead of fitting a latent space and then lineariz...
Recurrent Network Models for Human Dynamics
Published:
What they do Predicting human motion calls for a learned model of human dynamics. The Encoder-Recurrent-Decoder (ERD) puts nonlinear encoder and decoder networks before and after stacked LSTM layers and trains all thr...
Continuous control with deep reinforcement learning
Published:
What they do DQN needs a max over discrete actions; discretizing a 7-DoF arm with three levels per joint already gives 2187 actions. DDPG builds instead on the deterministic policy gradient (DPG): an actor network map...
Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours
Published:
What they do Human-labeled grasp datasets are small and biased: objects can be grasped many ways, and annotators favor parts like handles. Here a Baxter robot instead made random planar grasps on cluttered tables for ...
Deep Kalman Filters
Published:
What they do A classical Kalman filter is linear with Gaussian noise. replace its transition, action and emission functions with neural networks. Every variant is learned the same way: stochastic backpropagation on a...
Structural-RNN: Deep Learning on Spatio-Temporal Graphs
Published:
What they do Many problems are posed as spatio-temporal graphs (st-graphs): nodes for components such as body parts, humans or objects, edges for their interactions in space and over time. Structural-RNN (S-RNN) is a ...
Deep Learning for Tactile Understanding From Visual and Haptic Data
Published:
What they do They classify objects with binary haptic adjectives (24 of them, e.g. squishy, metallic, thin) using deep networks instead of hand-designed features. Three models are trained on the PHAC-2 dataset: one on...
Deep Residual Learning for Image Recognition
Published:
What they do Once networks with batch normalization start converging, adding layers past some depth makes them worse: on CIFAR-10 a 56-layer plain network has higher training error than a 20-layer one, so this is not ...
Value Iteration Networks
Published:
What they do Deep RL policies are usually reactive: CNN features map straight to actions, with no planning step. A value iteration network (VIN) puts a differentiable form of value iteration inside the policy. It lear...
Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection
Published:
What they do Grasping usually relies on careful camera-to-robot calibration and open-loop planning. This paper learns hand-eye coordination instead: a CNN predicts how likely a task-space gripper motion is to end in a...
Revisiting Active Perception
Published:
What they do A historical review and position paper on active perception. It focuses on vision but is meant to apply to other senses. The authors define an active perceiver as an agent that “knows why it wishes to sen...
Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstration
Published:
What they do The RBO Hand 2 is a cheap, compliant hand driven by inflating air chambers, with no finger position sensing and only on/off valves. Kinesthetic teaching and teleoperation do not work for it. Instead, a hu...
Structure-from-Motion Revisited
Published:
What they do Incremental SfM rebuilds 3D scenes from unordered photos by registering one image at a time, triangulating points and running bundle adjustment (BA). Existing systems (Bundler, VisualSFM) often leave many...
Cooperative Inverse Reinforcement Learning
Published:
What they do Inverse reinforcement learning (IRL) assumes the human acts optimally as if no robot were watching, which rules out teaching. Cooperative IRL (CIRL) is instead a two-player Markov game: the human observes...
Generative Adversarial Imitation Learning
Published:
What they do The setting is imitation from expert trajectories alone, with no expert queries and no reward. Behavioral cloning suffers from compounding errors, and inverse RL runs RL in an inner loop only to return a ...
Control Barrier Function Based Quadratic Programs for Safety Critical Systems
Published:
What they do The paper defines two barrier-function conditions that certify forward invariance of a safe set, turns them into control barrier functions (CBFs) that are affine constraints on the input, and combines the...
Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates
Published:
What they do Deep RL had mostly been limited to simulation because of its sample cost. This paper shows that off-policy deep Q-learning (NAF, plus DDPG in simulation) is efficient enough to learn manipulation on real ...
Deep Visual Foresight for Planning Robot Motion
Published:
What they do Visual foresight replaces the usual segment-model-simulate pushing pipeline with one learned model of future camera images. The user clicks a pixel on an object and a goal location for it; the robot picks...
ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras
Published:
What they do ORB-SLAM2 extends the monocular, feature-based ORB-SLAM to stereo and RGB-D cameras, keeping loop closing, relocalization and map reuse, and releases it as open source. Depth from a stereo pair or depth s...
CAD2RL: Real Single-Image Flight without a Single Real Image
Published:
What they do Learning collision avoidance by trial and error is impractical on a fragile quadrotor. CAD2RL trains a vision-based policy entirely in simulated 3D hallways and flies it on real drones without ever seeing...
EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces
Published:
What they do EEG-based BCIs usually rely on feature extractors and classifiers hand-tailored to one signal type. EEGNet is a single compact CNN meant to work across paradigms, train on limited data, and keep its learn...
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Published:
What they do Deep networks for 3D data usually converted point clouds to voxel grids or rendered images first. PointNet takes the point set itself as input and stays invariant to point order by max-pooling over featur...
Game-Theoretic Modeling of Human Adaptation in Human-Robot Collaboration
Published:
What they do People often misjudge what a robot can do. This paper models how that belief changes as the person watches the robot act. The robot uses the model to choose each round between revealing one of its limits ...
Human-Robot Mutual Adaptation in Shared Autonomy
Published:
What they do Shared-autonomy assistance usually assumes the user’s goal is right. Here the robot knows a better goal and steers the user there only if they seem willing to follow; otherwise it complies to keep their t...
Enabling Robots to Communicate their Objectives
Published:
What they do People anticipate a robot better if they understand the objective it optimizes. The robot here picks which environments to show its optimal behavior in, so that a person watching infers the right objectiv...
Deep learning with convolutional neural networks for EEG decoding and visualization
Published:
What they do EEG movement decoding has relied on hand-designed features, above all filter bank common spatial patterns (FBCSP), which extracts band power. This study asks whether ConvNets trained end-to-end on raw EEG...
Mask R-CNN
Published:
What they do extends the two-stage Faster R-CNN detector to instance segmentation. Next to the existing class and box branches, each region of interest (RoI) gets a third branch: a small fully convolutional network t...
Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
Published:
What they do Instead of making a simulator look realistic, they randomize its rendering so heavily that the real world looks like one more variation. They test this on object localization: a CNN, trained only on MuJoC...
One-Shot Imitation Learning
Published:
What they do Instead of one policy per task, this paper trains a single network that takes one demonstration of a block-stacking task plus the current state and outputs an action. A demonstration of a new layout (e.g....
Socially Aware Motion Planning with Deep Reinforcement Learning
Published:
What they do Socially compliant navigation is hard to specify directly, because people’s paths vary from person to person and from run to run. The paper argues that it is easier to specify what not to do. It builds on...
Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics
Published:
What they do Learning grasps from human labels or physical trials takes tedious labeling or months of robot time. Dex-Net 2.0 instead renders 6.7 million synthetic depth images of 1,500 3D models, labels parallel-jaw ...
Shape-independent Hardness Estimation Using Deep Learning and a GelSight Tactile Sensor
Published:
What they do Robots usually measure hardness by pressing with strictly controlled motion on objects of known geometry. The authors’ earlier GelSight model handled only hemispheres and needed the radius as input. Here ...
DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents
Published:
What they do DESIRE predicts where every agent in a scene will be up to 4 s ahead, from 2 s of past trajectory and a scene image or top-down map. Because a car at an intersection may turn either way, it samples many h...
Time-Contrastive Networks: Self-Supervised Learning from Video
Published:
What they do Imitating a human from video needs features that ignore viewpoint and embodiment but track what changes during a task, such as hand contact, container tilt, liquid level or body pose. Time-Contrastive Net...
On human motion prediction using recurrent neural networks
Published:
What they do Deep RNN models of human motion (ERD, LSTM-3LR, SRNN) were judged on short-term angle error and on long-term plausibility. The authors show that on Human3.6M a zero-velocity baseline, which just repeats t...
Safe Model-based Reinforcement Learning with Stability Guarantees
Published:
What they do Most RL algorithms explore freely, which can break a real system. Here safety means stability: the state must stay in a region of attraction, from which the policy returns it to the goal. Starting from a ...
PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
Published:
What they do PointNet encodes each point on its own and max-pools everything into one global feature, so it never captures local structure. PointNet++ adds the hierarchy that CNNs get from growing receptive fields. It...
Deep reinforcement learning from human preferences
Published:
What they do Many goals are hard to write as a reward or to demonstrate. Here a non-expert human instead watches pairs of short clips of the agent and picks the better one. A reward model is fit to these comparisons a...
Grasp Pose Detection in Point Clouds
Published:
What they do Grasp pose detection finds 6-DOF parallel-jaw grasps directly in a point cloud, with no object segmentation, CAD model or pose estimate. The paper combines a candidate sampler, a new grasp descriptor and ...
Hindsight Experience Replay
Published:
What they do Robot RL usually needs a carefully shaped reward. (HER) learns from a binary success signal instead. A failed attempt to reach goal g still shows how to reach the state the agent actually ended up in, so...
Emergence of Locomotion Behaviours in Rich Environments
Published:
What they do Locomotion rewards are usually hand-shaped and brittle. This paper asks whether a rich environment can replace the shaping: train simulated bodies on varied obstacle courses with a reward that is mostly f...
Information Theoretic Model Predictive Control: Theory and Applications to Autonomous Driving
Published:
What they do The paper derives a sampling-based model predictive controller (IT-MPC) from information theory and uses it to race a 1:5 scale AutoRally car around a dirt track. The free energy gives a lower bound on th...
Proximal Policy Optimization Algorithms
Published:
What they do Reusing a batch for several policy gradient steps tends to cause destructively large updates. TRPO limits each update with a KL constraint, but it is relatively complicated and does not fit architectures ...
One-Shot Visual Imitation Learning via Meta-Learning
Published:
What they do Learning a visuomotor skill from raw pixels usually takes many demonstrations per task. Meta-imitation learning (MIL) meta-trains a policy on demonstrations of many other tasks so that a gradient update o...
Hamilton-Jacobi Reachability: A Brief Overview and Recent Advances
Published:
What they do A tutorial on Hamilton-Jacobi (HJ) reachability, a verification method that finds the states from which a nonlinear system with bounded disturbances can be driven into a target set, or kept out of an unsa...
Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping
Published:
What they do Learned grasping from monocular RGB needs many labeled real grasps. This paper keeps the grasping system of Levine et al. unchanged (a CNN predicting grasp success from images and a motion command, plus a...
Deep Haptic Model Predictive Control for Robot-Assisted Dressing
Published:
What they do When a robot dresses a person, the garment, not the gripper, presses on the body. This paper predicts those forces from the robot’s end-effector force/torque and motion, and uses model predictive control ...
Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations
Published:
What they do The paper defines four MuJoCo tasks for the 24-DoF ADROIT hand: move a ball to a target, reorient a pen in the hand, unlatch and open a door, and hammer in a nail. Natural policy gradient (NPG) can solve ...
Overcoming Exploration in Reinforcement Learning with Demonstrations
Published:
What they do With a sparse reward, RL rarely finds any reward on long multi-step manipulation, and behavior cloning drifts off the demonstrations. The authors add 100 human demonstrations, teleoperated in VR with an H...
Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching
Published:
What they do A pick-and-place system for objects it has never seen, given only their product images. It reverses the usual recognize-then-grasp order: it picks an object without knowing what it is, lifts it away from ...
Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation
Published:
What they do Collecting demonstrations for learning visuomotor policies is hard: in kinesthetic teaching the human’s arms end up in the camera image, and other teleoperation setups can be expensive. The authors build ...
Social Attention: Modeling Attention in Human Crowds
Published:
What they do A robot moving through a crowd has to predict where the people around it will walk. Earlier models such as Social LSTM assume only people in a local neighbourhood affect each other. The authors argue that...
The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes?
Published:
What they do Does touch help a robot tell whether a grasp will hold? The authors put a GelSight tactile sensor on each finger of a parallel gripper on a Sawyer arm and autonomously collected 9,269 grasp trials on 106 ...
Sim-to-Real Transfer of Robotic Control with Dynamics Randomization
Published:
What they do Policies trained in simulation tend to exploit the simulator’s quirks and fail on the real robot. Instead of calibrating the simulator carefully, the authors randomize its dynamics during training and giv...
Asymmetric Actor Critic for Image-Based Robot Learning
Published:
What they do Policies trained in simulation must run on camera images in the real world, but learning from images alone is slow because of high dimensionality and partial observability. The simulator, however, knows t...
Trajectory Deformations from Physical Human-Robot Interaction
Published:
What they do Under impedance control a human can push a robot off its path, but the robot’s desired trajectory stays fixed, so the human has to keep pushing to change its behavior. This paper lets forces applied durin...
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
Published:
What they do PoseCNN estimates the 6D pose of known objects in cluttered scenes, where occlusion and symmetric objects break template matching and 3D-coordinate regression. It splits the problem into semantic labeling...
A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencoder
Published:
What they do A feeding robot should stop when something unusual happens, such as the user bumping the utensil, the face being occluded or a collision with the environment. The authors train a detector only on successf...
Future Person Localization in First-Person Videos
Published:
What they do The paper introduces a new task, future person localization: from a short wearable-camera clip of a person, predict where they will appear in the next frames, as a step toward assistive navigation in crow...
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
Published:
What they do Off-policy deep RL methods such as DDPG reuse past data but are brittle and sensitive to hyperparameters. On-policy methods such as PPO are more stable but need new samples for every gradient step. Soft a...
Riemannian Motion Policies
Published:
What they do Reactive obstacle avoidance often just adds many local policies together, and they fight: symmetric obstacles cancel out, heavy obstacle weights make the arm sluggish, and nullspaces need hand engineering...
Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning
Published:
What they do Deep networks need tens of thousands of examples, but one person can record only a few minutes of surface EMG per sitting. The authors pool recordings from many people instead. A ConvNet is pretrained on ...
Trust-Aware Decision Making for Human-Robot Collaboration: Model Learning and Planning
Published:
What they do The authors make human trust a latent variable in a POMDP, so the robot infers it from the human’s actions, predicts how its own actions change it, and plans for long-term team reward. The test is clearin...
Open3D: A Modern Library for 3D Data Processing
Published:
What they do An image can be loaded and shown with a few lines of OpenCV, but 3D data has no equally easy library; the authors describe PCL as bloated and largely dormant. Open3D is written from a clean slate around t...
Shared Autonomy via Deep Reinforcement Learning
Published:
What they do Earlier shared-autonomy methods infer the user’s goal, but they assume known dynamics, a known set of goals and a known model of how the user acts. This paper drops all three. A model-free deep Q-learning...
Feedback Control of an Exoskeleton for Paraplegics: Toward Robustly Stable Hands-free Dynamic Walking
Published:
What they do Exoskeletons such as EksoGT, ReWalk and HAL need crutches for balance; REX is hands-free but walks statically at about 0.05 m/s. This paper brings bipedal-robot control (virtual constraints, hybrid zero d...
PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planning
Published:
What they do Task and motion planning needs continuous values (poses, grasps, trajectories) that only procedures such as IK solvers and motion planners can produce. PDDLStream adds streams to PDDL: blackbox conditiona...
Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research
Published:
What they do This technical report has two parts. The first is a suite of simulated robotics tasks released in OpenAI Gym. A 7-DoF Fetch arm pushes, slides and picks and places objects, and a 24-DoF Shadow Dexterous H...
Addressing Function Approximation Error in Actor-Critic Methods
Published:
What they do Q-learning overestimates values when its targets are noisy. The paper proves that, under basic assumptions, deterministic policy gradient updates overestimate too, and measures a clear bias in DDPG’s valu...
GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger
Published:
What they do Earlier GelSight sensors point a camera straight at the gel and light it with several colors for photometric 3D reconstruction, which makes them bulky cuboids. GelSlim drops precise 3D reconstruction and ...
Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning
Published:
What they do Grasping fails when objects are packed tightly together, and pushing goals such as “separate these two” are hard to write down as an objective. VPG (Visual Pushing for Grasping) learns pushing and graspin...
World Models
Published:
What they do The agent is split into a large world model, which learns without rewards to compress game frames and predict how they evolve, and a tiny linear controller evolved on its features. In VizDoom the world mo...
Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks
Published:
What they do Social GAN takes 3.2 s of observed paths for everyone in a scene and predicts all their futures jointly. There are many socially acceptable ways to avoid a collision, and models trained with an L2 loss le...
Jacquard: A Large Scale Dataset for Robotic Grasp Detection
Published:
What they do Image-based grasp detectors need lots of labeled data; the main public set, Cornell, has 885 hand-labeled images. Jacquard generates the labels in simulation instead: CAD models from ShapeNetSem are dropp...
DeepIM: Deep Iterative Matching for 6D Pose Estimation
Published:
What they do RGB-only 6D pose estimators that classify or regress the pose directly are not very accurate, because small errors turn into pose mismatches. DeepIM is a refinement network. Given an image, the object’s 3...
DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
Published:
What they do Deep RL controllers for simulated characters tend to move unnaturally. DeepMimic uses PPO to train a policy that reproduces a reference clip (motion capture or keyframed animation) in physics simulation, ...
Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach
Published:
What they do Deep grasp planners of the time sampled and ranked grasp candidates one at a time. That took from about a second to tens of seconds, so the grasp was executed open-loop and needed a static scene and preci...
Towards Robotic Feeding: Role of Haptics in Fork-based Food Manipulation
Published:
What they do Feeding someone with a fork means handling food of very different compliance, size and shape. The authors instrument a fork, record 12 people feeding 12 food items (plus noodles and potato salad) to a man...
Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
Published:
What they do Policies learned in simulation tend to fail on real legged robots, because small model errors get amplified by frequent contact changes. The authors learn trotting and galloping for the Minitaur quadruped...
More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch
Published:
What they do Most learned grasping picks a grasp before touching the object and cannot react to contact. Here a Sawyer arm with a parallel gripper carries a GelSight optical tactile sensor on each finger. A deep netwo...
Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models
Published:
What they do Model-based RL with neural-network dynamics models is sample-efficient but usually converges to worse policies than model-free RL, partly because the networks overfit the little data available early on. P...
Probabilistically Safe Robot Planning with Confidence-Based Human Predictions
Published:
What they do A robot that predicts human motion with a goal-directed model can get dangerously close when the person does something the model does not explain, such as stepping around a spill the robot does not know a...
SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints
Published:
What they do SoPhie predicts the next 4.8 s of every pedestrian’s path from 3.2 s of history and one top-view image of the scene. Earlier methods use either the scene or the other agents, and many predict a single, av...
Motion Planning Networks
Published:
What they do Sampling-based planners such as RRT* slow down sharply as the planning problem’s dimension grows. MPNet replaces most of the search with two networks trained by imitation on RRT* paths. One encodes the ob...
Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation
Published:
What they do Dense Object Nets map an RGB image to a D-dimensional descriptor at every pixel. The same physical point on an object should get the same descriptor across viewpoints and deformations, and optionally acro...
Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervision
Published:
What they do A grasp that is stable is not always a grasp that lets the robot use the tool. Dex-Net-style grasping picks the most robust grasp, often near the center of mass, but hammering wants the end of the handle ...
QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
Published:
What they do Most learned grasping systems choose a grasp pose and then execute it open-loop. QT-Opt treats grasping as closed-loop reinforcement learning instead: at every step a Q-function scores gripper motions fro...
Learning Dexterous In-Hand Manipulation
Published:
What they do They train a Shadow Dexterous Hand (24 DoF, tendon-driven) to reorient an object in its palm to a sequence of goal orientations, using reinforcement learning only in simulation and no human demonstrations...
PointNetGPD: Detecting Grasp Configurations from Point Sets
Published:
What they do GPD and similar grasp evaluators project the point cloud into hand-crafted 2D images for a CNN. PointNetGPD instead gives the raw 3D points inside a candidate gripper’s closing area to a PointNet-style cl...
Gaze-based, Context-aware Robotic System for Assisted Reaching and Grasping
Published:
What they do People with upper-limb impairments usually keep control of their eyes. This system lets a user drive reach-and-grasp assistance with gaze alone: they look at an object, and a UR10 arm attached to their wr...
Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learning
Published:
What they do Earlier RL crowd-navigation policies aggregate the crowd crudely (CADRL keeps one worst-case human via maximin; LSTM-RL feeds humans in order of distance) and model only how humans affect the robot. This ...
HG-DAgger: Interactive Imitation Learning with Human Experts
Published:
What they do DAgger collects expert labels while control switches at random between the expert and the learner (the novice). With a human expert this hurts safety and label quality: the human gets little feedback from...
Deep Inertial Poser: Learning to Reconstruct Human Pose from Sparse Inertial Measurements in Real Time
Published:
What they do Commercial inertial motion capture uses about 17 IMUs. The earlier 6-IMU method SIP needs offline optimization over the whole sequence, which takes minutes to hours. DIP (Deep Inertial Poser) learns a dir...
Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience
Published:
What they do Domain randomization needs hand-tuned parameter ranges, and ranges that are too wide include infeasible scenes or yield overly conservative policies. SimOpt starts from a conservative distribution over si...
Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost
Published:
What they do Multi-fingered hands are hard to model and control because of their many joints and intermittent contacts. This paper trains neural-network policies for them with model-free deep RL directly in the real w...
Expressing Robot Incapability
Published:
What they do A robot that fails a task, such as lifting a cup that is too heavy or pushing a shelf that will not move, often does not get far enough for an observer to tell what it wanted or why it stopped. This paper...
Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks
Published:
What they do Peg insertion needs vision to find the hole and touch to align and insert, but deep RL on raw images and forces needs too many samples for a real robot. The authors first learn a compact representation of...
Differentiable MPC for End-to-end Planning and Control
Published:
What they do The paper treats Model Predictive Control as a policy class whose cost and dynamics are learned end-to-end from a task loss. Earlier differentiable planners unrolled the optimizer; here the derivative is ...
RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation
Published:
What they do Imitation learning needs hundreds to thousands of demonstrations. Kinesthetic teaching is tedious, keyboard and 3D-mouse teleoperation give unnatural axis-aligned trajectories, and VR needs hardware that ...
Learning Latent Dynamics for Planning from Pixels
Published:
What they do Learned latent models had only planned simple pixel tasks, such as balancing a cartpole from dense rewards. PlaNet is a purely model-based agent. It learns the dynamics from images, actions and rewards, t...
Residual Reinforcement Learning for Robot Control
Published:
What they do Conventional feedback controllers struggle with contact-rich manufacturing tasks because contacts and friction are hard to model, so they need manual tuning. RL can handle contacts, but learning from scra...
Learning to Walk via Deep Reinforcement Learning
Published:
What they do Deep RL on real legged robots is held back by sample cost and by hyperparameters that cannot be tuned through trial and error without breaking hardware. The authors extend soft actor-critic (SAC) so that ...
Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation
Published:
What they do Instance-level 6D pose methods need the exact CAD model of each object; category-level 3D detectors give boxes but not precise orientation. This paper estimates the full 6D pose and 3D size of unseen obje...
DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion
Published:
What they do RGB-D pose estimators either process color and depth separately or rely on slow post-processing such as ICP. DenseFusion encodes the image with a CNN and the depth points with a PointNet, fuses them pixel...
Learning agile and dynamic motor skills for legged robots
Published:
What they do ANYmal is a 32 kg quadruped with 12 series-elastic actuators, which no analytical model captures accurately enough for sim-to-real. The authors learn the actuator dynamics from robot data and train RL pol...
Peeking into the Future: Predicting Future Person Activities and Locations in Videos
Published:
What they do Most trajectory predictors treat a pedestrian as a point. Next encodes the person’s appearance, body motion and surroundings. From these it predicts the person’s future path and their future activity, one...
Learning Latent Plans from Play
Published:
What they do Instead of collecting segmented demonstrations per task, a human teleoperates a simulated 8-DOF arm in VR and plays freely with a desk that has a sliding door, a drawer, three buttons and a block. Play-LM...
Manipulation by Feel: Touch-Based Control with Deep Predictive Models
Published:
What they do Deep tactile MPC controls a robot by touch alone. A 4 cm GelSight-style sensor on a 3-axis CNC stage must move an object until the sensor reads a user-given goal tactile image. The three tasks are rolling...
Control Barrier Functions: Theory and Applications
Published:
What they do An overview of control barrier functions (CBFs), the safety counterpart of control Lyapunov functions (CLFs). Safety means a set C = {x : h(x) ≥ 0} is forward invariant: a system that starts in C never le...
TossingBot: Learning to Throw Arbitrary Objects with Residual Physics
Published:
What they do Throwing lets a picking arm place objects into boxes outside its kinematic reach without carrying them there. But where an object lands depends on how it was grasped, its mass distribution and aerodynamic...
AMASS: Archive of Motion Capture as Surface Shapes
Published:
What they do Public mocap datasets are small, and each uses its own markerset and skeleton, so they are hard to combine. Earlier merges retargeted everything to one fixed skeleton, which changes the data and throws aw...
Quasi-Direct Drive for Low-Cost Compliant Robotic Manipulation
Published:
What they do Force-controlled arms that are safe around people cost tens of thousands of dollars. This paper argues that quasi-direct-drive (QDD) actuation, low-ratio transmissions that stay backdrivable, can give a c...
Stanford Doggo: An Open-Source, Quasi-Direct-Drive Quadruped
Published:
What they do State-of-the-art legged robots take years to develop and are usually expensive custom designs. Stanford Doggo is a small quadruped (42 cm long, 4.8 kg, 8 degrees of freedom) built around a quasi-direct-dr...
Human Motion Trajectory Prediction: A Survey
Published:
What they do A survey of human motion trajectory prediction: ground-level 2D trajectories of pedestrians, plus cyclists and vehicles, for service robots, self-driving vehicles and surveillance. It covers more than 200...
PoseRBPF: A Rao-Blackwellized Particle Filter for 6D Object Pose Tracking
Published:
What they do Single-image 6D pose estimators return one pose with no uncertainty, which is not enough for symmetric or occluded objects. PoseRBPF tracks the full pose posterior over a video with a Rao-Blackwellized pa...
6-DOF GraspNet: Variational Grasp Generation for Object Manipulation
Published:
What they do To pick up an unknown object seen from a single depth view, a robot has to choose a full 6-DOF gripper pose, and it needs many candidates because some will be unreachable or in collision. 6-DOF GraspNet g...
Connecting Touch and Vision via Cross-Modal Prediction
Published:
What they do A camera sees the whole scene, while a GelSight sensor feels about 1.5 cm x 1.5 cm. The paper poses two tasks across this scale gap, synthesizing GelSight images from video of a robot touching an object a...
Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Tasks
Published:
What they do An empirical study of how the action space affects RL for manipulation. It compares joint torques, velocities and positions, variable joint impedance, and end-effector impedance with low, medium or high f...
Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer
Published:
What they do Depth datasets disagree in form: metric depth, depth up to scale (SfM), or disparity up to scale and shift (stereo with unknown calibration). This paper trains one monocular depth network on five such dat...
Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning
Published:
What they do Learned inverse-dynamics models extrapolate poorly; analytic models need precise physical parameters. Deep Lagrangian Networks (DeLaN) learn the inertia matrix H(q) and gravity g(q) with deep networks and...
Learning Trajectory Dependencies for Human Motion Prediction
Published:
What they do Given 10 observed poses (400 ms), the task is to forecast the next 10 (400 ms) or 25 (1 s). RNN predictors accumulate errors and often jump between the last observed and the first predicted frame. This pa...
Antipodal Robotic Grasping using Generative Residual Convolutional Neural Network
Published:
What they do GR-ConvNet predicts antipodal parallel-jaw grasps for unknown objects from an n-channel image (depth, RGB or RGB-D). Instead of scoring sampled candidates, it outputs a grasp at every pixel in one forward...
Crocoddyl: An Efficient and Versatile Framework for Multi-Contact Optimal Control
Published:
What they do Crocoddyl is an open-source library for multi-contact optimal control: given a predefined sequence of contacts and timings, it computes a whole-body state trajectory and a feedback control policy for a le...
Highly Dynamic Quadruped Locomotion via Whole-Body Impulse Control and Model Predictive Control
Published:
What they do Whole-body controllers usually track a commanded body trajectory, which fails for gaits like galloping or bounding where the center of mass cannot be controlled during flight. This paper pairs a low-rate ...
Controlling Assistive Robots with Learned Latent Actions
Published:
What they do Assistive arms have more degrees of freedom than a user can coordinate with a joystick. The paper learns latent actions: a low-DoF embedding of demonstrated robot actions, so that a 1- or 2-DoF joystick i...
Towards Variable Assistance for Lower Body Exoskeletons
Published:
What they do Crutch-less exoskeletons normally track a fixed gait rigidly, which is not ideal for patients who are recovering muscle function and should do part of the work themselves. This paper adds a variable assis...
Deep Dynamics Models for Learning Dexterous Manipulation
Published:
What they do PDDM (online planning with deep dynamics models) applies model-based RL to dexterous manipulation. It learns a neural-network model of the hand and objects and plans through it with MPC. The tasks are val...
RLBench: The Robot Learning Benchmark & Learning Environment
Published:
What they do Manipulation methods are usually tested on tasks each group builds for itself, or on control benchmarks such as OpenAI Gym and the DeepMind Control Suite, whose tasks the authors call toy problems. RLBenc...
Preference-Based Learning for Exoskeleton Gait Optimization
Published:
What they do The Atalante exoskeleton walks without crutches using gaits generated by partial hybrid zero dynamics, but nothing in that optimization captures user comfort, and users struggle to give reliable numerical...
An Open Torque-Controlled Modular Robot Architecture for Legged Locomotion Research
Published:
What they do Legged robots with good force control are usually expensive, hard to machine and not available to buy. This paper presents an open-source alternative: one low-cost torque-controlled actuator module, a foo...
Cable Manipulation with a Tactile-Reactive Gripper
Published:
What they do Most cable manipulation keeps the cable quasistatic or constrains it with tables, pegs or clamps. Here a free-hanging cable is followed in real time: a gripper fixed to the table holds its start, and a se...
DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System
Published:
What they do DexPilot teleoperates a KUKA iiwa7 arm and Allegro hand (23 actuated joints) by watching the operator’s bare hand with four RealSense depth cameras, with no glove or markers. The aim is a low-cost way to ...
Assistive Gym: A Physics Simulation Framework for Assistive Robotics
Published:
What they do Research on robots that physically help people is costly and risky to run with real people. Assistive Gym is an open-source simulation framework, built on PyBullet behind the OpenAI Gym interface, with si...
On the Utility of Learning about Humans for Human-AI Coordination
Published:
What they do Self-play works in competitive games because a human’s suboptimal move only helps the agent. In a common-payoff game the same mistake hurts the team, and co-trained agents can settle on conventions humans...
Solving Rubik’s Cube with a Robot Hand
Published:
What they do A Shadow Dexterous Hand solves a Rubik’s cube with control and vision models trained only in simulation. A standard solver plans the moves; the learned system senses the cube’s pose and face angles and ex...
Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
Published:
What they do Meta-RL methods have mostly been tested on narrow task families, such as different running directions for one simulated robot, which cannot show whether they adapt to genuinely new behaviors. Meta-World i...
RoboNet: Large-Scale Multi-Robot Learning
Published:
What they do Most robot-learning experiments use one robot in one lab. RoboNet is an open dataset of about 162,000 trajectories (nearly 15 million frames) from 7 robots, 4 institutions and 113 camera configurations. T...
Dream to Control: Learning Behaviors by Latent Imagination
Published:
What they do Model-based agents that maximize imagined rewards over a fixed horizon tend to be shortsighted, and many rely on derivative-free planning. Dreamer learns a world model from images and then learns behavior...
Grasping in the Wild: Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrations
Published:
What they do Most learned grasping is top-down or open-loop, because a 6DoF closed-loop policy needs too much trial-and-error data on the robot. The authors had people do everyday pick-and-place tasks with a handheld ...
Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data
Published:
What they do Trajectron++ forecasts trajectories for a varying number of interacting agents of different types (pedestrians, cars). Most forecasters ignore agent dynamics, maps and the robot’s own plan. This model pre...
The exoskeleton expansion: improving walking and running economy
Published:
What they do A review of lower-limb exoskeletons that break the “metabolic cost barrier”: walking or running costs the user less energy with the device than without it. The authors searched Web of Science through Dece...
Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction
Published:
What they do Earlier pedestrian predictors model each person with a recurrent network and combine those hidden states with pooling or attention layers. Social-STGCNN instead treats the scene as a graph from the start:...
Human Preference-Based Learning for High-dimensional Optimization of Exoskeleton Walking Gaits
Published:
What they do Exoskeleton gaits should be comfortable, but no numeric metric is known to track comfort, so the authors learn from which of two gaits a user prefers. Earlier preference-learning methods handled at most t...
OmniTact: A Multi-Directional High Resolution Touch Sensor
Published:
What they do Optical tactile sensors like GelSight give high-resolution images but only on one flat face, while curved multi-directional sensors (capacitive, resistive, photodiode arrays, BioTac) have far lower resolu...
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Published:
What they do NeRF represents a static scene as a continuous 5D function: for any 3D location and viewing direction, an MLP returns a volume density and an emitted RGB color. A pixel is rendered by sampling points alon...
RAFT: Recurrent All-Pairs Field Transforms for Optical Flow
Published:
What they do Most deep optical-flow networks work coarse-to-fine, so coarse errors are hard to undo and small fast-moving objects get missed. RAFT keeps one flow field at 1/8 resolution and refines it many times with ...
Learning Agile Robotic Locomotion Skills by Imitating Animals
Published:
What they do Instead of a hand-designed controller or reward per skill, an 18-DoF Laikago quadruped gets a reference motion (dog mocap or an artist’s animation), and RL in simulation trains a policy to track it. A sho...
It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction
Published:
What they do PECNet forecasts pedestrian trajectories in two steps: infer where each pedestrian will be at the end of the 4.8 s prediction window, then predict the path to a sampled endpoint jointly for everyone in th...
Shared Autonomy with Learned Latent Actions
Published:
What they do Eating with an assistive arm takes coarse reaching (which plate) and precise manipulation at that goal (cutting, stabbing, scooping), all through a low-dimensional joystick. Shared autonomy helps reach di...
DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
Published:
What they do Vision-based tactile sensors (GelSight and its descendants) image a soft elastomer from behind, so they give very high-resolution contact images. They were bulky, hand-built and fragile. DIGIT shrinks the...
Towards Expedited Impedance Tuning of a Robotic Prosthesis for Personalized Gait Assistance by Reinforcement Learning Control
Published:
What they do Powered knee prostheses usually run a finite-state impedance controller: four gait phases, each with a stiffness, equilibrium angle and damping, so 12 parameters that prosthetists hand-tune for hours per ...
Learning Reward Functions from Diverse Sources of Human Feedback: Optimally Integrating Demonstrations and Preferences
Published:
What they do Demonstrations are informative but noisy and hard to give on high-DoF robots; pairwise preferences are precise but carry at most one bit each. DemPref uses both: a few demonstrations set a prior over the ...
3D Shape Reconstruction from Vision and Touch
Published:
What they do Vision gives global shape but is blocked by the hand; touch gives exact local geometry but cannot extrapolate. This paper fuses the two for 3D shape reconstruction. A single RGB image and four touch readi...
Object Handovers: a Review for Robotics
Published:
What they do A handover is a joint action in which a giver passes an object to a receiver. This review splits it into a pre-handover phase (communication, grasping and transport, until the receiver first touches the o...
robosuite: A Modular Simulation Framework and Benchmark for Robot Learning
Published:
What they do robosuite is a simulation framework and benchmark for robot learning built on MuJoCo. It grew out of the SURREAL distributed-RL project and is now part of the ARISE Initiative. This version of the paper d...
Integrated Task and Motion Planning
Published:
What they do A manipulating robot must choose actions, their continuous parameters (which grasp, where to place an object) and collision-free motions. This review treats all three as one problem, task and motion plann...
Learning Quadrupedal Locomotion over Challenging Terrain
Published:
What they do Rough-terrain legged controllers have usually been state machines that trigger reflexes from estimated contact and slip events. This paper trains a blind controller for ANYmal quadrupeds with RL in simula...
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Published:
What they do They apply a standard NLP Transformer to image classification with as few image-specific changes as possible. An image becomes a sequence of patches, each treated like a word token. The model is pre-train...
Transporter Networks: Rearranging the Visual World for Robotic Manipulation
Published:
What they do Transporter Networks treat manipulation as a sequence of spatial displacements: choose a region to pick, then decide where that region should go. The model is learned from demonstrations and uses no objec...
Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Composition
Published:
What they do Reference trajectories are hard to obtain and over-constrain a gait; reference-free rewards like “move forward” are underspecified or need gait-specific shaping. This paper uses the fact that every common...
Learning Latent Representations to Influence Multi-Agent Interaction
Published:
What they do The other agent in a repeated interaction (a driver, an air hockey opponent) changes its policy in response to the robot. LILI (Learning and Influencing Latent Intent) skips modeling the other agent’s act...
Reactive Human-to-Robot Handovers of Arbitrary Objects
Published:
What they do Prior human-to-robot handover systems were tested on a handful of objects, or needed the object to belong to a detector’s known categories. This paper builds a vision-only system that takes unknown rigid ...
ACRONYM: A Large-Scale Grasp Dataset Based on Simulation
Published:
What they do ACRONYM is a dataset for learning 6-DOF parallel-jaw grasping. It labels 2000 grasps on each of 8872 ShapeNetSem objects with the outcome of a physics simulation instead of an analytical quality measure, ...
iNeRF: Inverting Neural Radiance Fields for Pose Estimation
Published:
What they do Render-and-compare pose estimation normally needs a watertight mesh of the object; iNeRF uses a NeRF instead. Given an RGB image, known intrinsics, an initial pose guess and a trained NeRF, it freezes the...
TACTO: A Fast, Flexible, and Open-source Simulator for High-Resolution Vision-based Tactile Sensors
Published:
What they do Vision-based tactile sensors such as DIGIT and OmniTact are cameras looking at a gel, so simulating them needs both contact and optical rendering, fast enough to be useful for learning. TACTO is an open-s...
Volumetric Grasping Network: Real-time 6 DOF Grasp Detection in Clutter
Published:
What they do Earlier 6 DOF grasp detectors often reasoned on isolated objects, needing separate collision checks in clutter, or took seconds per plan. VGN maps a TSDF of the whole workspace to a dense grasp map: for e...
Generation of GelSight Tactile Images for Sim2Real Learning
Published:
What they do GelSight sensors image a soft, opaque-painted membrane from the inside under coloured LEDs, which gives high-resolution touch, but no simulator produced such images. The authors build a simulated GelSight...
BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data
Published:
What they do EEG classifiers generalize poorly across subjects, sessions and tasks, and labelled EEG is scarce. BENDR pretrains one network on a large unlabelled clinical EEG corpus with a self-supervised task taken f...
Learning Transferable Visual Models From Natural Language Supervision
Published:
What they do Standard vision models predict a fixed set of labeled categories. CLIP instead learns from 400 million (image, text) pairs collected from the internet, trained only to tell which caption goes with which i...
GelSlim3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Finger
Published:
What they do Earlier slim GelSlim fingers dropped photometric stereo, because it needs three light colors hitting the gel evenly from different directions, which is hard to fit in a thin finger. GelSlim 3.0 brings it ...
iMAP: Implicit Mapping and Positioning in Real-Time
Published:
What they do iMAP is a dense SLAM system for a handheld RGB-D camera. Its only scene representation is an MLP that starts from random weights and is trained live, with no prior data. A tracking process aligns each fra...
AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting
Published:
What they do Multi-agent trajectory forecasters usually model time and social interaction in separate stages, for example an LSTM per agent followed by a graph network across agents. AgentFormer instead flattens the p...
Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes
Published:
What they do Earlier 6-DoF grasping pipelines generate grasps from a segmented object point cloud and then filter collisions with a second network, so they depend on accurate masks and ignore the surrounding scene. Co...
Robotic Guide Dog: Leading a Human with Leash-Guided Hybrid Physical Interaction
Published:
What they do Most robotic guides for visually impaired people are wheeled platforms with large bases or rigid actuated canes, which cannot fit through narrow spaces. This paper uses a small quadruped (a Mini Cheetah) ...
Sim-to-Real for Robotic Tactile Sensing via Physics-Based Simulation and Learned Latent Projections
Published:
What they do The BioTac turns skin deformation into fluidic impedances read at 19 electrodes, which is hard to simulate. The authors rebuild their experimentally validated ANSYS finite-element (FEM) model of the BioTa...
Tactile-RL for Insertion: Generalization to Objects of Unknown Geometry
Published:
What they do Peg-in-hole insertion of objects whose geometry is unknown. The robot holds the object between two GelSlim tactile sensors and tries to insert it. When the insertion is blocked, it reads the tactile image...
AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control
Published:
What they do Tracking-based character control needs a hand-designed pose-error reward and, for large datasets, a motion planner to pick which clip to track. AMP replaces both with an adversarial motion prior: a discri...
Multimodal Fusion of EMG and Vision for Human Grasp Intent Inference in Prosthetic Hand Control
Published:
What they do EMG-controlled prosthetic hands suffer from electrode shift, fatigue and impedance changes, while camera-based grasp classification suffers from occlusion and lighting. This paper fuses the two to infer t...
DexYCB: A Benchmark for Capturing Hand Grasping of Objects
Published:
What they do Joint hand and object pose estimation needs real data of hands handling objects, but gloves, magnetic sensors or mocap markers can change how hands move and look. DexYCB records 10 subjects picking up 20 ...
The MIT Humanoid Robot: Design, Motion Planning, and Control For Acrobatic Behaviors
Published:
What they do The paper presents the MIT Humanoid, a small humanoid designed for acrobatics, together with the planner and controller meant to drive it. The robot applies the MIT Cheetah design principles (torque-dense...
Emerging Properties in Self-Supervised Vision Transformers
Published:
What they do The authors ask whether self-supervised pretraining gives Vision Transformers (ViTs) properties that supervised ViTs and convnets lack. With DINO, self-distillation with no labels, they find two: the [CLS...
Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning
Published:
What they do The authors ask how far a blind biped can go on stairs. They take an existing sim-to-real RL framework for flat-ground Cassie gaits and change only the training terrain to randomized staircases, with no s...
Soft Biomimetic Optical Tactile Sensing with the TacTip: A Review
Published:
What they do A review of the BRL TacTip as the prototype “SoftBOT” (Soft Biomimetic Optical Tactile) sensor. It relates the design to human fingertip skin, argues that it embodies the shear-sensing hypothesis, and sur...
Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translation
Published:
What they do An open-source suite of PyBullet RL environments for an optical tactile sensor (the TacTip), with five tasks: edge following, surface following, object rolling, object pushing and object balancing. Polici...
GelSight Wedge: Measuring High-Resolution 3D Contact Geometry with a Compact Robot Finger
Published:
What they do Earlier GelSight sensors measure high-resolution 3D contact geometry but are too bulky for robot fingers, and the slimmer GelSlim is not optically designed for photometric stereo. GelSight Wedge is a comp...
Habitat 2.0: Training Home Assistants to Rearrange their Habitat
Published:
What they do Habitat 2.0 is a simulation stack for training mobile manipulators on household rearrangement. It has three parts: ReplicaCAD, an artist-built interactive recreation of the Replica ‘FRL-apartment’ (92 obj...
Physical Interaction as Communication: Learning Robot Objectives Online from Human Corrections
Published:
What they do Standard physical human-robot interaction (pHRI) controllers treat a push as a disturbance: the robot complies while touched, then returns to its original plan. This paper treats intentional pushes as cor...
RMA: Rapid Motor Adaptation for Legged Robots
Published:
What they do Simulation-trained walking policies break when real terrain, payload or motors differ from the simulator, and collecting real data to adapt risks falls. RMA trains, entirely in simulation, a walking polic...
Representation learning for neural population activity with Neural Data Transformers
Published:
What they do Models that turn noisy spike trains into single-trial firing rates, such as LFADS, rely on recurrent networks. These process a window of data step by step, which slows real-time uses like brain-computer i...
What Matters in Learning from Offline Human Demonstrations for Robot Manipulation
Published:
What they do Learning manipulation offline from human demonstrations lacked open human datasets and reproducible baselines. The authors collect teleoperated datasets for five simulated tasks (Lift, Can, Square, two-ar...
DexMV: Imitation Learning for Dexterous Manipulation from Human Videos
Published:
What they do Reinforcement learning for a multi-finger hand needs a lot of data, and earlier VR-glove demonstrations are costly: there were only 25 per task. DexMV records ordinary human videos instead, about 100 demo...
Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning
Published:
What they do Robot RL usually simulates physics on CPUs and runs the networks on GPUs, so state has to be copied back and forth at every step. Isaac Gym keeps the whole loop on one GPU. PhysX steps tens of thousands o...
Implicit Behavioral Cloning
Published:
What they do Behavioral cloning policies are usually explicit networks that map an observation straight to an action. This paper changes only the form of the policy: an energy-based model scores observation-action pai...
Taxim: An Example-based Simulation Model for GelSight Tactile Sensors
Published:
What they do To simulate a GelSight sensor you must model how the soft gelpad deforms and how the internal LEDs and camera see it. Physics-based renderers for this are slow, sensor-specific and miss the real sensor’s ...
The Design of Stretch: A Compact, Lightweight Mobile Manipulator for Indoor Human Environments
Published:
What they do The authors argue that smaller, lighter and cheaper mobile manipulators would see more use in homes. They present the design of Stretch (the Stretch RE1 from Hello Robot) and justify it with anthropometry...
Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning
Published:
What they do In prior work, training quadruped locomotion policies took between 12 and 120 hours. Here, Isaac Gym runs both simulation and PPO on one workstation GPU, with thousands of ANYmal robots stepping at once. ...
CLIPort: What and Where Pathways for Robotic Manipulation
Published:
What they do Transporter places objects precisely but ignores language, so every new goal needs new demonstrations. CLIP grounds words in images but cannot say where to act. CLIPort combines them in a two-stream netwo...
Vision-Only Robot Navigation in a Neural Radiance World
Published:
What they do Given a NeRF of a scene trained offline, a robot plans dynamically feasible, collision-free trajectories through it and localizes itself using only an onboard RGB camera. A trajectory optimizer and a NeRF...
Ego4D: Around the World in 3,000 Hours of Egocentric Video
Published:
What they do Most video datasets are short, curated, third-person clips. Ego4D collects long, unscripted, head-mounted video of daily life: 3,670 hours from 931 participants in 74 cities and 9 countries, gathered by 1...
ReSkin: versatile, replaceable, lasting tactile skins
Published:
What they do Soft tactile skins wear out, and every replacement responds slightly differently, so a model calibrated on one skin fails on the next. ReSkin separates the cheap, wearable part (a magnetized elastomer) fr...
A System for General In-Hand Object Re-Orientation
Published:
What they do They train a multi-finger hand to rotate an object from any orientation to any goal orientation. The hand faces up, faces down over a table, or faces down with nothing below it. Everything runs on a simul...
IKFlow: Generating Diverse Inverse Kinematics Solutions
Published:
What they do An arm with 7 or more joints can reach most poses in infinitely many ways. Analytical solvers such as IKFast return every solution but only work up to 6 DOF. Numerical solvers return one solution per call...
CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks
Published:
What they do CALVIN (Composing Actions from Language and Vision) is a simulated benchmark for long-horizon, language-conditioned manipulation. A single agent must carry out a series of free-form instructions in a row,...
Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation
Published:
What they do Learning pick-and-place from demonstrations usually needs demos that cover many object instances and poses. Neural Descriptor Fields (NDFs) let a Franka Panda repeat a task (hang a mug on a rack, place a ...
DenseTact: Optical Tactile Sensor for Dense Shape Reconstruction
Published:
What they do High-resolution camera-based tactile sensors such as GelSight, GelSlim and DIGIT are flat, which limits in-hand manipulation, and curved alternatives are costly or low-resolution. DenseTact is a hemispher...
Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents
Published:
What they do The paper asks whether large pre-trained language models already know how to break a high-level household task (“make breakfast”) into executable steps (“open fridge”), without any training. In VirtualHom...
Learning robust perceptive locomotion for quadrupedal robots in the wild
Published:
What they do Seeing terrain before stepping on it lets a legged robot move faster, but elevation maps built from depth sensors fail on snow, vegetation, reflective or transparent surfaces, occlusion and pose-estimatio...
BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning
Published:
What they do BC-Z asks whether plain behavior cloning, scaled to many tasks, can do new manipulation tasks without any robot demos of them. Seven operators on 12 robots collected 25,877 demonstrations (125 hours) of 1...
Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion
Published:
What they do RL locomotion controllers usually need an accurate state estimate, but analytical estimators become unreliable on ice or sand, and many assume a fixed gait schedule that learned policies don’t follow. Thi...
R3M: A Universal Visual Representation for Robot Manipulation
Published:
What they do Robot datasets cover few tasks and scenes, so R3M pretrains a visual representation on human video instead: Ego4D, over 3500 hours from more than 70 locations, each clip paired with a language description...
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
Published:
What they do A language model can break “I spilled my drink, can you help?” into steps, but it does not know what the robot can do or what is in front of it. SayCan restricts the LLM to scoring a fixed set of skill de...
Rapid Locomotion via Reinforcement Learning
Published:
What they do A single neural network controller for the MIT Mini Cheetah, trained with RL in simulation and deployed zero-shot, tracks commanded forward, lateral and yaw velocity up to high speeds. Sampling fast comma...
Motion Planning around Obstacles with Convex Optimization
Published:
What they do Trajectory optimization handles dynamics but relies on local solvers that can fail in cluttered spaces; sampling-based planners struggle in high dimensions and with continuous differential constraints; mi...
A Generalist Agent
Published:
What they do Gato applies the large-language-model recipe to control. Every modality (text, image patches, proprioception, joint torques, button presses) is serialized into one flat token sequence, and a single decode...
HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers
Published:
What they do Testing human-to-robot handovers usually needs a person in the loop, and every study picks its own objects and metrics. HandoverSim is a simulation benchmark where a Franka Panda takes objects from a simu...
Planning with Diffusion for Flexible Behavior Synthesis
Published:
What they do Model-based RL usually learns a dynamics model and hands it to a classical trajectory optimizer, which tends to exploit the model’s errors. Diffuser instead folds planning into the generative model: a dif...
MyoSuite – A contact-rich simulation suite for musculoskeletal motor control
Published:
What they do Muscle-driven control is pull-only, redundant and non-stationary, but RL benchmarks use joint-level control, and musculoskeletal simulators such as OpenSim are slow and offer limited contact support. MyoS...
Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning
Published:
What they do Bi-DexHands is a simulated benchmark for bimanual dexterous manipulation with RL. Two Shadow Hands perform 20 tasks such as handing over, catching, opening bottle caps, lifting pots and stacking blocks. E...
Behavior Transformers: Cloning $k$ modes with one stone
Published:
What they do Standard behavior cloning fits a unimodal policy with an MSE loss, which averages across the different ways demonstrators solve a task. BeT learns from offline demonstrations with no reward or task labels...
DayDreamer: World Models for Physical Robot Learning
Published:
What they do Deep RL needs too much trial and error for real hardware, so most robot learning goes through a simulator. This paper tests whether Dreamer, a world-model RL algorithm shown on video games and simulated t...
Inner Monologue: Embodied Reasoning through Planning with Language Models
Published:
What they do LLM planners such as SayCan pick skills assuming each one succeeds. Inner Monologue closes the loop: after every skill, textual feedback from the environment is appended to the LLM’s prompt, so it can ret...
A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learning
Published:
What they do The paper asks how fast fully model-free deep RL can learn quadruped locomotion directly on hardware. Using no new algorithm, only existing components implemented carefully and a well-chosen task setup, a...
Perceptive Locomotion through Nonlinear Model Predictive Control
Published:
What they do Dynamic locomotion over gaps, stepping stones and stairs needs precise footholds, leg-terrain collision avoidance and planning of the underactuated torso at the same time. Instead of choosing footholds fi...
Decoding speech perception from non-invasive brain recordings
Published:
What they do Speech decoders that work well so far rely on electrodes implanted in the brain. This paper decodes perceived speech from non-invasive recordings instead. Given 3 s of MEG or EEG recorded while a healthy ...
Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation
Published:
What they do Behavior cloning that maps 2D images straight to 6-DoF actions needs a lot of data. PerAct (Perceiver-Actor) puts both observation and action in a 3D voxel grid and treats each step as classification: giv...
Code as Policies: Language Model Programs for Embodied Control
Published:
What they do In Code as Policies (CaP), a code-writing LLM writes the robot policy itself rather than choosing from a fixed set of learned skills. Prompted with example commands (as Python comments) and their code, Co...
ActiveNeRF: Learning where to See with Uncertainty Estimation
Published:
What they do NeRF trained on few images, or on images that miss part of the scene, generalizes poorly and can collapse to zero density in the unseen regions. ActiveNeRF works under a fixed capture budget: it adds an u...
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models
Published:
What they do LLM task planners either score every possible next action, which scales badly, or generate free text that may name actions the robot lacks or objects that are not there. ProgPrompt writes the prompt as a ...
VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training
Published:
What they do Pre-trained visual encoders (R3M, MoCo, CLIP, ImageNet ResNet) work well as policy inputs. But the distance to a goal image in their embedding space makes a poor reward. VIP treats Ego4D human videos as d...
Personalizing exoskeleton assistance while walking in the real world
Published:
What they do Human-in-the-loop optimization gives the largest exoskeleton benefits, but it ranks torque patterns with respirometry during long treadmill bouts in a lab. The authors replace respirometry with a model th...
DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation
Published:
What they do Dexterous grasping lacks large training datasets: the only robotic one before this, DDGdata, has 6.9K ShadowHand grasps on 565 objects, generated with GraspIt!. The authors speed up and harden an existing...
VIMA: General Robot Manipulation with Multimodal Prompts
Published:
What they do Robot tasks are usually specified in one format per model: a language instruction, a goal image or a demonstration video. VIMA expresses all of them as one multimodal prompt, an interleaved sequence of te...
Real-World Robot Learning with Masked Visual Pre-training
Published:
What they do They test whether a single vision encoder, pre-trained with self-supervision on in-the-wild images, can serve as the perception module for many real robot tasks. The encoder is trained once as a masked au...
In-Hand Object Rotation via Rapid Motor Adaptation
Published:
What they do The task is rotating an object about the z-axis using only the fingertips of a 16-DoF Allegro hand, with no vision or touch. The policy is trained entirely in simulation on cylinders and runs on the real ...
Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion
Published:
What they do Legged manipulators are usually controlled hierarchically, with separate controllers for the arm and the legs. This paper trains one RL policy that outputs joint targets for all 12 leg joints and 6 arm jo...
Motion Policy Networks
Published:
What they do MπNets replaces a motion planning pipeline with a single learned policy for a Franka Panda arm. From the current joint configuration and a point cloud of the scene, it predicts the next joint configuratio...
DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Reality
Published:
What they do DeXtreme revisits OpenAI’s in-hand cube reorientation with cheaper parts: a four-fingered Allegro Hand with a locked wrist, three off-the-shelf RGB cameras, no markers, no cage, and GPU simulation in Isaa...
MidasTouch: Monte-Carlo inference over distributions across sliding touch
Published:
What they do A vision-based touch sensor sliding over an object sees only a small patch, so a single touch is ambiguous: a sharp edge could lie anywhere along a mug’s lip. MidasTouch keeps a full distribution over the...
Legged Locomotion in Challenging Terrains using Egocentric Vision
Published:
What they do Vision-based legged locomotion usually fuses depth images into a metric elevation map, which needs odometry and comes out noisy. Here a Unitree A1 with one front-facing RealSense camera maps egocentric de...
Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment
Published:
What they do Learned manipulation policies are not reliable enough to deploy on their own. Sirius deploys one anyway, with a human supervisor. The robot acts by default; the human watches, takes over with a 6-DoF Spac...
Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes
Published:
What they do In-hand reorientation turns a held object from any start orientation to any target in the full rotation space. Here the hand faces downward, so the fingers must also counter gravity, and the only sensors ...
Touch and Go: Learning from Human-Collected Vision and Touch
Published:
What they do Earlier visuo-tactile datasets were collected by robots on lab tabletops or in simulation, so they cover a small number of graspable, mostly rigid objects. In Touch and Go, people collect the data instead...
Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior
Published:
What they do Locomotion policies trained in simulation handle environments like their training set, but when they fail out of distribution the usual fix is to redesign rewards or environments and retrain. Walk These W...
See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation
Published:
What they do A Franka Panda arm sees with a third-person camera, hears with a piezo contact microphone attached to the object of interest, and feels with GelSight sensors on its fingertips. A self-attention model fuse...
VideoDex: Learning Dexterity from Internet Videos
Published:
What they do Teleoperating a dexterous hand for every task is slow, and most work uses human video only to pretrain visual features. VideoDex also takes the motion from egocentric human videos in Epic Kitchens: it con...
RT-1: Robotics Transformer for Real-World Control at Scale
Published:
What they do RT-1 is a single language-conditioned policy for a mobile manipulator (7-DoF arm, gripper, mobile base), trained by behavior cloning on about 130k human demonstrations. The data was collected by 13 robots...
MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare
Published:
What they do MegaPose estimates the 6D pose of objects never seen during training. At test time it needs only the object’s CAD model and a 2D region of interest, and works from RGB or RGB-D. It keeps the detect / coar...
AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domains
Published:
What they do AnyGrasp is a grasp perception system for a parallel gripper. From a single partial point cloud it predicts dense 7-DoF grasps (rotation, center, width), and it keeps those grasps consistent across frames...
Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments
Published:
What they do Orbit is an open-source robot learning framework built on NVIDIA Isaac Sim. It combines GPU-accelerated rigid and FEM-based deformable physics (PhysX 5) with ray-traced rendering and sensors such as RGB-D...
MimicPlay: Long-Horizon Imitation Learning by Watching Human Play
Published:
What they do Long-horizon imitation learning usually needs hours of teleoperated robot data. MimicPlay splits the problem in two. A high-level planner learns where and what to interact with from human play data: a per...
BrainBERT: Self-supervised representation learning for intracranial recordings
Published:
What they do Linear decoders of neural recordings are explainable but weak; deep decoders are stronger but may solve the task themselves instead of reading it from the brain. BrainBERT sits in between: a Transformer i...
UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policy
Published:
What they do UniDexGrasp grasps and lifts arbitrary tabletop objects with a ShadowHand, using only a point cloud and proprioception, in Isaac Gym. Like parallel-gripper pipelines, it works in two stages. First it samp...
PaLM-E: An Embodied Multimodal Language Model
Published:
What they do LLM planners such as SayCan see only text, so they miss scene geometry. Off-the-shelf vision-language models cannot solve robot reasoning tasks either. PaLM-E encodes images, state vectors or object-centr...
Real-World Humanoid Locomotion with Reinforcement Learning
Published:
What they do A fully learned walking controller for Digit, a 1.6 m, 45 kg humanoid from Agility Robotics. The policy is a causal transformer over the recent history of proprioceptive observations and actions, trained ...
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
Published:
What they do Demonstration data is multimodal, temporally correlated and needs precision, which strains standard behavior cloning. Diffusion Policy models the policy as a conditional denoising diffusion process over r...
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
Published:
What they do Grounding DINO detects arbitrary objects named in text, either a list of category names or a referring expression with attributes. It adds language to the closed-set Transformer detector DINO and pretrain...
LERF: Language Embedded Radiance Fields
Published:
What they do A NeRF captures how a scene looks but not what is in it. LERF trains a language field alongside the NeRF, supervised by CLIP embeddings of image crops. A user can then type any query, from “yellow” and “e...
Rotating without Seeing: Towards In-hand Dexterity through Touch
Published:
What they do Touch Dexterity rotates an object in the palm of a multi-finger hand without vision. Instead of a few high-resolution fingertip sensors, it covers the palm, finger links and fingertips of an Allegro hand ...
BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects
Published:
What they do BundleSDF tracks an unknown rigid object through a monocular RGBD video and reconstructs its shape and texture at the same time. The only input besides the video is a 2D object mask in the first frame: no...
Learning Human-to-Robot Handovers from Point Clouds
Published:
What they do They learn a human-to-robot handover policy from vision alone. The robot sees segmented hand and object point clouds from its wrist camera and outputs end-effector motions. Training happens in HandoverSim...
Segment Anything
Published:
What they do Segmentation has no web-scale data source, so the authors build three linked pieces: a promptable segmentation task (return a valid mask for any point, box, mask or text prompt, even an ambiguous one), a ...
DINOv2: Learning Robust Visual Features without Supervision
Published:
What they do Self-supervised image features had mostly been trained on ImageNet-1k or on large uncurated web sets, which hurt feature quality. DINOv2 tests whether self-supervised pretraining on a large curated datase...
Affordances from Human Videos as a Versatile Representation for Robotics
Published:
What they do Affordance models trained on human video had been tested on video datasets but not on robots. VRB (Vision-Robotics Bridge) represents an affordance in a form a robot can execute: a contact point and a pos...
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
Published:
What they do Fine bimanual tasks such as opening a condiment cup or slotting a battery need millimeter precision, while low-cost arms are accurate to only 5-8 mm. The paper pairs ALOHA, a teleoperation rig built from ...
Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning
Published:
What they do A low-cost Robotis OP3 humanoid (51 cm, 3.5 kg, 20 servos) learns simplified 1v1 soccer with deep RL, trained only in MuJoCo and deployed zero-shot on real robots. One policy maps proprioception plus moti...
TidyBot: Personalized Robot Assistance with Large Language Models
Published:
What they do Where things belong in a home depends on the person. TidyBot learns one user’s preferences from a handful of example placements, without collecting a preference dataset or training a model. An off-the-she...
Perpetual Humanoid Control for Real-time Simulated Avatars
Published:
What they do Simulated humanoids driven by noisy pose estimates tend to fall. Prior imitators either reset to the (unreliable) reference pose or push the root with a non-physical residual force. PHC is a single RL pol...
LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning
Published:
What they do LIBERO is a simulation benchmark for lifelong robot manipulation: one language-conditioned policy learns tasks in sequence without access to earlier tasks’ data. Its suites separate declarative knowledge ...
ANYmal Parkour: Learning Agile Navigation for Quadrupedal Robots
Published:
What they do A fully learned pipeline gets a quadruped to a target across parkour-like obstacles (boxes, gaps, tables, stairs) within a time limit. It uses no expert demonstrations, offline planning or prior map. Ever...
Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners
Published:
What they do LLM planners can be confidently wrong, and instructions are often ambiguous: “place the bowl in the microwave” when there is a plastic and a metal bowl. KnowNo asks for help only when needed. With human h...
AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System
Published:
What they do Vision-based teleoperation of dexterous arm-hand robots is usually built around one robot, one camera setup and one simulator or real rig. AnyTeleop is a single system that tracks the operator’s hand with...
VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models
Published:
What they do LLM-based robot planners usually call a library of hand-made motion primitives. VoxPoser instead has GPT-4 write Python that composes 3D voxel value maps in the robot’s observation space: where to go, wha...
CoTracker: It is Better to Track Together
Published:
What they do Point trackers such as PIPs++ and TAPIR follow each query point independently, even though points on the same object move together. CoTracker estimates many tracks at once with a transformer that attends ...
Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation
Published:
What they do 2D foundation-model features carry semantics but little 3D geometry. F3RM (Feature Fields for Robotic Manipulation) distills dense DINO or CLIP features into a NeRF of the tabletop, so every 3D point has ...
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
Published:
What they do RT-2 turns a large pretrained vision-language model (VLM) directly into a closed-loop robot policy. Robot actions are written as strings of integers and treated as text, so the VLM is fine-tuned on robot ...
A high-performance speech neuroprosthesis
Published:
What they do Participant T12 of the BrainGate2 trial has ALS and can no longer speak intelligibly. The authors implanted four 64-electrode arrays, two in area 6v (ventral premotor cortex) and two in area 44 (part of B...
Getting the Ball Rolling: Learning a Dexterous Policy for a Biomimetic Tendon-Driven Hand with Rolling Contact Joints
Published:
What they do The paper introduces the Faive Hand (prototype “Proto 0”), a biomimetic tendon-driven hand built from 3D-printed parts and servo motors, meant as an accessible platform for dexterous manipulation. Its fin...
3D Gaussian Splatting for Real-Time Radiance Field Rendering
Published:
What they do Novel-view synthesis from photos of a static scene. The best NeRF, Mip-NeRF360, needs 48 h of training and 10 s per frame; faster methods lose quality. 3D Gaussian Splatting replaces the continuous field ...
BridgeData V2: A Dataset for Robot Learning at Scale
Published:
What they do BridgeData V2 is a real-robot manipulation dataset built to be reused by other labs. It has 60,096 trajectories on a low-cost WidowX 250 arm, covering 13 skills, 24 environments (mostly toy kitchens, plus...
9DTact: A Compact Vision-Based Tactile Sensor for Accurate 3D Shape Reconstruction and Generalizable 6D Force Estimation
Published:
What they do 9DTact is a compact, open-source vision-based tactile sensor built from 3D-printed, laser-cut and off-the-shelf parts for about $15. It keeps the DTact principle (a translucent gel looks darker where cont...
Robot Parkour Learning
Published:
What they do Parkour on low-cost quadrupeds (Unitree A1 and Go1): climbing, leaping gaps, crawling under barriers, tilting through narrow slits and running, all from one end-to-end policy that uses only onboard depth ...
The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systems
Published:
What they do A review of safety filters: processes that monitor an autonomous system at runtime and override its intended control only when it could lead to a catastrophic failure. The authors argue that seemingly dif...
LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learning
Published:
What they do Real-world dexterous manipulation is held back by hardware. Shadow Hand costs over 100K USD and needs constant maintenance; Allegro (16K USD) has weak motors, closed-source parts that are hard to repair, ...
General In-Hand Object Rotation with Vision and Touch
Published:
What they do Earlier sim-to-real rotation policies on the Allegro hand, such as Hora, turned objects only about the z-axis. RotateIt rotates objects continuously about the x, y or z axis using only the fingertips, wit...
GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators
Published:
What they do Cheap teleoperation devices such as 3D mice and VR controllers command the end effector, so the operator cannot feel the robot’s joint limits, singularities or self-collisions. GELLO is a miniature replic...
Extreme Parkour with Legged Robots
Published:
What they do Parkour on a low-cost Unitree A1 with imprecise actuation and one front-facing depth camera that is low-frequency, jittery and prone to artifacts. Instead of building an elevation map and planning, a sing...
DTC: Deep Tracking Control
Published:
What they do Trajectory optimization (TO) places feet precisely but breaks when its model assumptions fail; RL is robust but struggles where valid footholds are rare, such as gaps and stepping stones. DTC has a TO pla...
ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning
Published:
What they do Dense maps with a vision-language feature per point scale poorly and hold no relations between objects. ConceptGraphs builds an object-centric 3D scene graph from posed RGB-D frames using only off-the-she...
Open X-Embodiment: Robotic Learning Datasets and RT-X Models
Published:
What they do Robot learning usually trains a separate policy per robot and setup. This paper tests whether one policy trained on many robots’ data can beat those specialists. Twenty-one institutions pooled 60 existing...
Brain decoding: toward real-time reconstruction of visual perception
Published:
What they do Image reconstruction from brain activity has mostly used fMRI, which yields about one snapshot every two seconds. This paper decodes seen images from magnetoencephalography (MEG), which is much faster but...
Eureka: Human-Level Reward Design via Coding Large Language Models
Published:
What they do Reward design for RL is mostly manual trial and error. Eureka hands it to a coding LLM (GPT-4): given the environment’s source code and the task description, it writes the reward as Python, trains policie...
A Unified, Scalable Framework for Neural Population Decoding
Published:
What they do Neural recordings share no vocabulary: each session, day and animal gives a different set of neurons with no known correspondence, so decoders are usually trained per session. POYO (Pre-training On manY n...
cuRobo: Parallelized Collision-Free Minimum-Jerk Robot Motion Generation
Published:
What they do cuRobo treats arm motion generation as a global optimization problem: it optimizes many seeds in parallel on a GPU and keeps the best, with every stage (kinematics, collision checking, inverse kinematics ...
MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations
Published:
What they do Imitation learning needs many teleoperated demonstrations, yet much of that data repeats the same skill in a different place. MimicGen takes a handful of human demos (10 per task) and automatically genera...
NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities
Published:
What they do NOIR is a brain-robot interface built on non-invasive, saline-based EEG. It follows hierarchical shared autonomy: the person watches the scene on a screen and communicates a high-level goal through brain ...
FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Information
Published:
What they do FisherRF uses Fisher Information to measure how much a candidate camera pose would tell a radiance field about its parameters, and captures the views with the highest expected information gain (EIG). The ...
Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensing
Published:
What they do In-hand rotation needs vision and touch, but touch is sparse and low-dimensional while vision is dense. Robot Synesthesia turns each triggered tactile sensor into points on that sensor’s mesh, so contact ...
SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM
Published:
What they do SplaTAM does dense SLAM from a single unposed RGB-D camera, using 3D Gaussians as the map. Because Gaussians rasterize fast, both camera tracking and mapping can use a dense loss over every pixel of the r...
FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects
Published:
What they do Pose methods for novel objects usually assume either a CAD model (model-based) or a few reference images (model-free), and either estimate or track. FoundationPose covers all four cases with one RGBD netw...
Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation
Published:
What they do During in-hand manipulation the fingers hide the object from the camera, yet a robot still needs its pose and shape. NeuralFeels combines an RGB-D camera, DIGIT touch sensors on the four fingertips of an ...
DUSt3R: Geometric 3D Vision Made Easy
Published:
What they do Classical reconstruction chains matching, pose estimation, triangulation and multi-view stereo, and needs camera parameters before dense reconstruction. DUSt3R replaces the chain with one network: given t...
Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation
Published:
What they do Most imitation-learning results are on table-top tasks. Mobile ALOHA puts the low-cost ALOHA bimanual puppeteering rig on a wheeled base, so one person can teleoperate both arms and the base at the same t...
Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data
Published:
What they do Depth Anything is a monocular relative-depth model meant to work on any image. Rather than new modules, it scales up data: a teacher trained on 1.5M labeled images from six public datasets pseudo-labels 6...
Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control
Published:
What they do A single RL framework that produces walking, running and jumping controllers for Cassie, a human-sized torque-controlled biped. Each policy covers a range of commands for its skill and can also stand. The...
Binding Touch to Everything: Learning Unified Multimodal Tactile Representations
Published:
What they do Touch data is expensive to collect and tactile sensors differ in their outputs, so tactile representations are usually trained per sensor. UniTouch trains a single touch encoder on four visuo-tactile data...
Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots
Published:
What they do Hand-held grippers can collect demonstrations anywhere, but earlier systems only transferred simple grasping or quasi-static pick-and-place to robots. UMI pairs a 3D-printed hand-held gripper, with a GoPr...
Expressive Whole-Body Control for Humanoid Robots
Published:
What they do Human motion-capture data has far more degrees of freedom than a real humanoid (69 in physics-based animation vs 19 on a Unitree H1), and animation controllers use actuator gains about ten times higher th...
Humanoid Locomotion as Next Token Prediction
Published:
What they do They treat humanoid control like language modeling: a causal transformer learns to predict the next token of sensorimotor trajectories. Because it predicts observations as well as actions, it can also lea...
Splat-Nav: Safe Real-Time Robot Navigation in Gaussian Splatting Maps
Published:
What they do Splat-Nav navigates a drone inside a pre-trained Gaussian Splatting (GSplat) map using only an RGB camera. Each Gaussian is treated as an ellipsoid, which gives the planner exact collision geometry to pla...
Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation
Published:
What they do Imitation policies trained on a few demos break under clutter and disturbances, and running RL on a real robot is slow and unsafe. RialTo builds a digital twin of the deployment scene, uses sparse-reward ...
3D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations
Published:
What they do Image-based Diffusion Policy needs 100-200 demonstrations per real task. DP3 keeps its action backbone but replaces image input with a sparse point cloud from a single camera, encoded into a compact featu...
Feel the Bite: Robot-Assisted Inside-Mouth Bite Transfer using Robust Mouth Perception and Physical Interaction-Aware Control
Published:
What they do Most feeding robots stop in front of the mouth, but many people with severe mobility limitations cannot lean forward to take the bite. This system places food inside the mouth. It keeps tracking the mouth...
Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation
Published:
What they do H2O teleoperates a full-sized Unitree H1 humanoid, whole body including the legs, in real time from a single RGB webcam. No exoskeleton, markers or force sensors on the operator. The main obstacle they id...
DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation
Published:
What they do Teleoperating dexterous hands is slow and needs the robot; vision-based hand tracking breaks under occlusion and lacks accurate 3D. DexCap is a portable mocap rig that records finger motion, 6-DoF wrist p...
DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation
Published:
What they do Earlier tactile simulators sit on rigid-body engines or handle only rigid objects. DiffTactile is a differentiable Taichi simulator of a GelSight sensor touching rigid, elastic, elastoplastic, articulated...
HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation
Published:
What they do HumanoidBench is a MuJoCo benchmark for whole-body humanoid control: a Unitree H1 with two Shadow Hands (forearms removed), 12 locomotion tasks and 15 whole-body manipulation tasks such as truck unloading...
DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
Published:
What they do DROID is a teleoperated robot manipulation dataset built for scene diversity: 76k successful demonstrations (350 hours) across 564 scenes in 52 buildings and 86 tasks. Fifty data collectors at 13 institut...
Learning Visuotactile Skills with Two Multifingered Hands
Published:
What they do Instead of the usual parallel-jaw grippers, this paper fits two UR5e arms with Psyonic Ability Hands, prosthetic hands repurposed for research with custom PCBs, whose fingertips carry six touch sensors ea...
Octo: An Open-Source Generalist Robot Policy
Published:
What they do Octo is a generalist robot policy pretrained on 800k demonstrations from 25 Open X-Embodiment datasets, released fully open source: checkpoints, finetuning and pretraining code, and data loaders. It takes...
Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI
Published:
What they do EEG deep learning models are usually built for one dataset and task, because recordings differ in electrode count, length and task design, and labeled sets are small. LaBraM (Large Brain Model) is a singl...
OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning
Published:
What they do OmniH2O trains a single whole-body policy for a Unitree H1 with Inspire hands that tracks a kinematic pose goal. Because many sources can produce that goal, the same policy serves VR teleoperation, an RGB...
OpenVLA: An Open-Source Vision-Language-Action Model
Published:
What they do Existing VLAs such as RT-2-X are closed, and their fine-tuning to new robots was unexplored. OpenVLA is a 7B open-source VLA trained on 970k Open X-Embodiment trajectories, evaluated out of the box on two...
HumanPlus: Humanoid Shadowing and Imitation from Humans
Published:
What they do HumanPlus is a full-stack system for a humanoid to learn from human data. An RL policy trained on human motion lets the robot shadow an operator in real time from a single RGB camera. The shadowing double...
Humanoid Parkour Learning
Published:
What they do Learning-based humanoid locomotion has mostly stayed on flat ground and relied on motion references or rewards that force the feet to lift. This paper trains one end-to-end policy for the Unitree H1 that ...
Open-TeleVision: Teleoperation with Immersive Active Visual Feedback
Published:
What they do In most teleoperation setups the operator watches the robot directly or through a fixed camera: they must be in the same room, the robot’s own body occludes the view, and close-up work is hard. Open-TeleV...
Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning
Published:
What they do Bunny-VisionPro teleoperates a 24-DoF bimanual robot, two UFactory xArm-7 arms with 6-DoF Ability hands, from an Apple Vision Pro. The operator’s finger poses drive the hands, their wrist poses drive the ...
Berkeley Humanoid: A Research Platform for Learning-based Control
Published:
What they do Berkeley Humanoid is a mid-scale humanoid research platform built almost entirely in-house: actuators, motor drivers, IMU, communication and power boards. It weighs 16 kg, stands 0.85 m tall, has two 6-Do...
SAM 2: Segment Anything in Images and Videos
Published:
What they do SAM 2 is one model for promptable segmentation of images and videos. Clicks, a box or a mask on any frame define an object; the model returns its mask on that frame, propagates it through the video as a “...
TacSL: A Library for Visuotactile Sensor Simulation and Learning
Published:
What they do Visuotactile sensors such as GelSight are expensive to simulate (contact, membrane deformation, lighting, imaging), so tactile policies have rarely been trained with online RL in simulation. TacSL is a GP...
SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splatting
Published:
What they do RGB policies trained in simulation rarely transfer because simulated images look nothing like real ones. SplatSim keeps the simulator (PyBullet) for physics but replaces its mesh rendering with a Gaussian...
emg2qwerty: A Large Dataset with Baselines for Touch Typing using Surface Electromyography
Published:
What they do Wrist-worn surface EMG could be a private, always-available text input, but there has been no large public dataset with a well-defined task. emg2qwerty records sEMG at both wrists while people touch type ...
HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots
Published:
What they do Learned humanoid controllers each fix one command interface: root velocity for locomotion, joint angles for expressive motion, or a few body keypoints for teleoperation. HOVER trains one policy for a 19-D...
Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning
Published:
What they do HIL-SERL trains vision-based manipulation policies with reinforcement learning directly on real robots, for tasks that are hard to learn from scratch: precise insertion, dual-arm assembly, a deformable ti...
Sparsh: Self-supervised touch representations for vision-based tactile sensing
Published:
What they do Vision-based tactile sensors such as DIGIT and GelSight usually get a custom model trained on labeled data for each task and sensor, and labels like force and slip need expensive lab instrumentation. Spar...
3D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensing
Published:
What they do Bimanual manipulation of fragile objects or tools held in the hand needs contact information that cameras miss under occlusion. 3D-ViTac adds dense, flexible tactile pads to soft fin-shaped grippers on a ...
$π_0$: A Vision-Language-Action Flow Model for General Robot Control
Published:
What they do π0 is a generalist vision-language-action (VLA) model built on the 3B-parameter PaliGemma VLM that outputs continuous actions with flow matching instead of discrete action tokens. One model is pre-trained...
DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning
Published:
What they do Teleoperating two arms with multi-fingered hands is slow, so human demos are scarce. DexMimicGen generates thousands more in simulation from a handful. It extends MimicGen, built for one arm with a parall...
Digitizing Touch with an Artificial Multimodal Fingertip
Published:
What they do Vision-based tactile sensors like Digit and GelSight mostly capture contact geometry, at camera rates of 30-60 Hz. Digit 360, from FAIR at Meta, is a fingertip-shaped sensor that keeps a high-resolution c...
TidyBot++: An Open-Source Holonomic Mobile Manipulator for Robot Learning
Published:
What they do Mobile manipulation data is scarce partly because suitable research hardware is expensive, large, or kinematically constrained. TidyBot++ is an open-source mobile manipulator whose base is holonomic: it c...
ExBody2: Advanced Expressive Humanoid Whole-Body Control
Published:
What they do ExBody2 trains one RL controller that lets a Unitree G1 humanoid track whole-body human motions (walking, crouching, dancing) from motion capture, then transfers it to the real robot. Raw mocap contains m...
Design and Control of a Bipedal Robotic Character
Published:
What they do Legged robots are usually designed for function. This one is designed for character: a small droid-like biped whose proportions, backward-bending knees and walking style were set by animators, with the me...
FAST: Efficient Action Tokenization for Vision-Language-Action Models
Published:
What they do Autoregressive VLAs such as RT-2 and OpenVLA turn each action dimension at each timestep into one of 256 bins. At high control frequencies consecutive tokens are nearly identical, so each new token carrie...
ToddlerBot: Open-Source ML-Compatible Humanoid Platform for Loco-Manipulation
Published:
What they do ToddlerBot is an open-source miniature humanoid (0.56 m, 3.4 kg) designed as a research platform that both runs policies and collects the data to train them, in simulation and in the real world. It is ent...
ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills
Published:
What they do Agile humanoid skills break on hardware because simulated and real dynamics differ. ASAP runs simulation-trained motion-tracking policies on a Unitree G1 and uses the recorded trajectories to learn a resi...
Brain-to-Text Decoding: A Non-invasive Approach via Typing
Published:
What they do Brain-to-text neuroprostheses work well but need brain surgery. This paper asks how far non-invasive recordings can go. 35 healthy, skilled typists read a short Spanish sentence, waited, then typed it fro...
Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulation
Published:
What they do Imitation-learning policies with action chunking, such as Diffusion Policy and ACT, execute each chunk open-loop, so they cannot react to touch mid-chunk. The paper adds two pieces: TactAR, a teleoperatio...
VGGT: Visual Geometry Grounded Transformer
Published:
What they do Classical 3D reconstruction relies on iterative optimization such as bundle adjustment (BA), and learned methods like DUSt3R and MASt3R process only two images at a time, fusing pairs in post-processing. ...
GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
Published:
What they do GR00T N1 is an open vision-language-action (VLA) model intended as one foundation policy for humanoids and other robots. Real humanoid data is scarce, so they train on a “data pyramid”: web data and human...
ORCA: An Open-Source, Reliable, Cost-Effective, Anthropomorphic Robotic Hand for Uninterrupted Dexterous Task Learning
Published:
What they do Anthropomorphic hands are either expensive and hard to maintain (the Shadow Hand costs over 100,000 CHF) or direct-driven and bulky. ORCA is an open-source, 3D-printable, tendon-driven hand the size of a ...
RUKA: Rethinking the Design of Humanoid Hands with Learning
Published:
What they do Robot hands trade off precision, compactness, strength and cost. RUKA is an open-source tendon-driven hand with the dimensions of an average human hand, built from 3D-printed parts and off-the-shelf compo...
$π_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
Published:
What they do π0.5 tests how far a vision-language-action model generalizes in homes it has never seen. Building on π0, it trains on about 400 hours of mobile-manipulator data from about 100 homes, plus a much larger m...
Demonstrating Berkeley Humanoid Lite: An Open-source, Accessible, and Customizable 3D-printed Humanoid Robot
Published:
What they do Most humanoids are either expensive closed commercial products or lab prototypes that need CNC machining. Berkeley Humanoid Lite is a mid-scale humanoid (0.8 m, 16 kg) that anyone can build: every non-sta...
TWIST: Teleoperated Whole-Body Imitation System
Published:
What they do TWIST teleoperates a humanoid by having it imitate the operator’s whole-body motion in real time. OptiTrack MoCap captures the human, an online retargeter converts the pose into robot joint positions and ...
DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation
Published:
What they do Teleoperating a dexterous hand gives no haptic feedback, and human hand video does not transfer directly to robot hands with different kinematics and appearance. In DexUMI a person does the task wearing a...
BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion
Published:
What they do Learning humanoid skills from human motion usually needs per-motion reward tuning, and the resulting trackers can only replay their clips. BeyondMimic trains motion trackers with one compact RL recipe, th...
A generic non-invasive neuromotor interface for human-computer interaction
Published:
What they do Meta’s CTRL-labs built a wristband that records surface electromyography (sEMG), the electrical activity of the forearm muscles, and decoders that work for people they were never trained on, unlike earlie...
HumanoidExo: Scalable Whole-Body Humanoid Manipulation via Wearable Exoskeleton
Published:
What they do Teleoperating a humanoid needs the robot and a skilled operator for every demo, while human video and simulation carry an embodiment gap. HumanoidExo records a person wearing an arm exoskeleton and a back...
Improving CMA-ES Convergence Speed, Efficiency, and Reliability in Noisy Robot Optimization Problems
Published:
What they do Human-in-the-loop exoskeleton tuning measures each candidate controller for minutes, and the measurement time trades speed against noise. Usually every condition gets the same fixed time. AS-CMA leaves CM...
Exo-Plore: Exploring Exoskeleton Control Space through Human-aligned Simulation
Published:
What they do Tuning an exoskeleton controller with human-in-the-loop optimization takes hours of walking, which people with mobility impairments often cannot do. Exo-plore moves the search into simulation: a musculosk...
Learning Hip Exoskeleton Control Policy via Predictive Neuromusculoskeletal Simulation
Published:
What they do Exoskeleton controllers that work across speeds and slopes are usually trained on motion-capture and force-plate data. This paper instead trains a hip-exoskeleton policy entirely in a predictive neuromusc...
Musculoskeletal Motion Imitation for Learning Personalized Exoskeleton Control Policy in Impaired Gait
Published:
What they do Tuning exoskeleton assistance usually needs human-in-the-loop optimization or large biomechanics datasets, both hard to do with patients. This paper trains a muscle-driven model with RL to imitate recorde...
Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?
Published:
What they do Precise assembly with a multi-fingered hand is contact-rich (hard to teleoperate) and sparse-reward (hard to explore with RL). Play2Perfect splits the problem in two. A goal-conditioned policy first learn...
UniExo: Unified Multi-Skill Policies for Musculoskeletal Locomotion and Co-Adaptive Exoskeleton Control
Published:
What they do Most exoskeleton controllers are built for one activity and need explicit mode switching. UniExo first trains one muscle-driven human policy (MyoLeg, 80 muscles) that walks, turns, runs, walks backward an...
Learning a Speed-adaptive Hip Exoskeleton Control Policy Via Sim-to-real Reinforcement Learning
Published:
What they do A hip exoskeleton has to get both the timing and the magnitude of its assistance right, at any walking speed. The authors learn the timing in simulation with RL against a musculoskeletal model. On the rea...
Multi-Objective Human-in-the-Loop Bayesian Optimization of a Lower-Limb Exoskeleton
Published:
What they do Human-in-the-loop optimization usually tunes an exoskeleton for one objective, or for a fixed weighted sum of several. MO-HILBO instead learns each wearer’s Pareto set: the controllers where neither comfo...
DexPort: A Hand-Held Skill Interface for Dexterous Hand Data Collection
Published:
What they do Teleoperating a five-fingered hand is slow, and human-hand demonstrations need retargeting and contain no robot actions. DexPort is a UMI-style handle for dexterous hands: the operator carries the actual ...
Learning Purposeful Co-designs of Bio-inspired Quadrupedal Robots
Published:
What they do Co-design of legged robots often searches a narrow mechanical space, such as knee elasticity alone. Dogger is a 6.5 kg quadruped with three-segment legs and a parallel knee spring; link lengths and spring...
RoboColosseum: Evaluating Physical Intelligence at Scale In-The-Wild
Published:
What they do Real-world evaluations of generalist robot policies are hard to reproduce and usually rest on a few rollouts. RoboColosseum is a distributed benchmark in which full-time operators run standardized robots ...
Energy-Based Policy Orchestration
Published:
What they do Hierarchical controllers usually map a planner’s target straight to the goal of one low-level policy. On a humanoid, slips, pushes or loads can make that goal infeasible faster than the planner updates. E...
W2A: Distilling World Priors into Action Models for Real-Time Robotic Control
Published:
What they do World action models (WAMs) use pretrained video models to imagine the future before acting, which helps long-horizon manipulation but makes each action slow. W2A keeps the video model only as a training-t...
MAPO: Mixture of Experts with Alternating Policy Optimization for Universal Humanoid Motion Tracking
Published:
What they do Universal humanoid motion trackers often use a dense mixture-of-experts (MoE) policy, where a router mixes the outputs of several experts. The authors show that when such a policy is trained with PPO, the...
DexArena: A Unified Benchmark for Long-Horizon Dexterous Manipulation
Published:
What they do DexArena is a simulation benchmark, built on SAPIEN, for testing whether dexterous-hand policies can learn single skills, generalize them to new scenes and chain them into long tasks. It has 10 long-horiz...
HeLM: Hierarchical Explicit Language Memory for Non-Markovian Manipulation
Published:
What they do Many manipulation tasks are non-Markovian: after pressing a button twice or wiping part of a window, the scene can look just as it did before, so a policy that sees only the current frame loses track. HeL...
Visuo-tactile Diffusion Policy for Robotic Manipulation with Auxiliary Tactile Dynamics Prediction
Published:
What they do A diffusion policy for a simulated UR5 assembly task (grasp an object, transport it to a target region, align it) that conditions on two camera views and two fingertip “tactile” images. The main addition ...
Beyond Policy Patches: Distilling RL Specialists into Vision-Language-Action Policies
Published:
What they do VLA policies often handle the easy stages of a long-horizon task but fail in short precision-critical phases such as insertion, grasping or handover. Small RL specialists can fix those phases, but they st...
SomaAlign: Reliability-Aware Multimodal Representation Learning for Soft Robotic Grasping
Published:
What they do Soft grippers deform, vibrate and slip, so their tactile and IMU signals are noisier than camera images, and most visuo-tactile datasets use rigid grippers with optical tactile sensors. The authors collec...
ReGATE: Recurrent Gated Visual Fusion for Reactive Humanoid-Object Interaction
Published:
What they do ReGATE is a whole-body controller that lets a Unitree G1 humanoid pick up, carry and put down a box using only proprioception and a head-mounted depth camera, with no motion capture or object-pose estimat...
AnchorHEC: Targetless Hand-Eye Calibration of Wrist and Fixed Cameras
Published:
What they do Many manipulation setups pair a wrist camera with fixed cameras, and each needs its transform to the robot: wrist camera to end effector, fixed camera to robot base. Target-based calibration needs a board...
PointPilot: Unified Robotic Manipulation via Object-Centric 3D Point Flow Prediction
Published:
What they do PointPilot is a manipulation policy whose output is the motion of 3D points on a “task-focused object”: the gripper itself while reaching, then the object rigidly held by it after the grasp. Because it pr...
ReContact: Reconstruct and Retarget Human Contacts from Video for Contact-Rich Humanoid Manipulation
Published:
What they do Humanoids often pass an object’s support between body parts, e.g. resting a box on a raised thigh while the palms slide underneath. Copying such contact transitions from human video fails when the body hi...
SyncHeC: Markerless Spatio-Temporal Hand-Eye Calibration via Differentiable Rendering and Continuous-Time Joint Optimization
Published:
What they do Using camera images for control requires knowing where the camera is relative to the robot (the hand-eye extrinsics). It also requires knowing which joint state each image shows: exposure, transport and c...
BeyondContact: A Reinforcement Learning Framework for Airflow-Based Non-Contact Manipulation
Published:
What they do Airflow can push objects that an arm cannot reach, or should not touch. RL for this has been held back by the lack of a fast airflow simulator. BeyondContact adds airflow forces to Isaac Lab using a jet m...
KlaskTron-IL: A Contact-Rich Adversarial Imitation Learning Benchmark with a Validated Sim-to-Real Bridge
Published:
What they do KlaskTron is an open-hardware robot (under $2,500) for the two-player tabletop game Klask, with an Isaac Lab twin that already transfers PPO policies zero-shot. This paper adds an imitation-learning layer...
KlaskTron: An Open-Source Platform for Physical Adversarial Multi-Agent RL
Published:
What they do Physical multi-agent RL has few cheap, reproducible testbeds. KlaskTron is a desktop robot that plays KLASK, a two-player game in which magnetically coupled strikers knock a ball into the opponent’s goal....
VerifLM: Physically Verified Image-Edit-Guided Goal Pose Generation for Robotic Manipulation
Published:
What they do Many placement goals, such as a plate in a specific slot of a stand, are easy to picture but hard to write down as poses or keypoint constraints. VerifLM asks an image-editing model to draw the goal, reco...
ForceLDM: A Force-aware Latent Dynamics Model for Contact-Rich Manipulation
Published:
What they do Force-aware VLAs usually feed force/torque in as one more input and learn a direct observation-to-action mapping, while video world models predict future frames and mix control-relevant dynamics with appe...
Safe-Ballbot: Tilt-Consistent MPC with Control Barrier Functions for Ballbot Navigation
Published:
What they do A ballbot balances on one ball driven by three omni wheels and has to lean to accelerate, so its planner must respect the balance dynamics. Planning on the full nonlinear model is slow, and a separate til...
Exploring Proximity and Tactile Sensing for Robotic Manipulation
Published:
What they do Tactile sensors only respond once contact has happened, and wrist cameras are often occluded right where the fingers meet the object. ToFTouch adds proximity sensing at the fingertip: a 45 x 30 x 10 mm mo...
SNAP: Semantic Navigation Among Pedestrians
Published:
What they do City robots must avoid pedestrians and also follow norms such as staying on sidewalks. Crowd-navigation methods usually need explicit pedestrian positions and velocities, while semantic planners treat the...
When Robots Outweigh Objects: How Inertia Mismatch Shapes Robot Policy Learning
Published:
What they do In public simulation datasets, robot arms present a mean effective mass of 6 to 16 kg at contact, while the objects weigh 8 to 250 g. The paper asks whether this inertia mismatch costs robot learning anyt...
PanoTac: A Dense Three-Axis Tactile Sensor for Robust Touch-Centric Manipulation
Published:
What they do Piezoresistive skins give dense contact maps but no shear. Magnetic skins give shear but only at a few points. PanoTac stacks the two in one compliant fingertip for a parallel-jaw gripper, producing a den...
TAMPAgent: Agentic Belief-Space Planning for Mobile Manipulation
Published:
What they do Agents that write code calling robot tools (Code as Policies, CaP) check no geometric constraints before acting; task and motion planning (TAMP) checks them but needs hand-written problems and fully obser...
D&A: Decoupling Description and Assessment Enables VLMs as Zero-Shot Robotic Reward Models
Published:
What they do Asked directly for per-frame task progress, VLMs tend to assume the robot succeeded and hallucinate progress on failed runs. D&A (Describe & Assess) is a zero-shot, purely prompt-based reward mode...
Pixel-UMI: Learning Environment-Aware Whole-Body Manipulation from Sensorized Human Demonstrations
Published:
What they do Handheld, UMI-style demonstrations record where the head and grippers go, but not how a mobile manipulator’s base, torso and arms should get there. Pixel-UMI converts human demonstrations, collected with ...
Pixel: A Co-Designed Full-Stack Robot System for Learning General-Purpose Mobile Manipulation
Published:
What they do Pixel is a mobile-manipulation stack built around one low-cost wheeled humanoid. The robot, its data-collection interfaces, the whole-body controller and a hierarchical policy were designed together. As a...
Robot-Matched Soft Fingers for UMI Data Collection: Lessons from 100 Hours of Pick and Place
Published:
What they do Handheld Universal Manipulation Interfaces (UMIs) collect demonstrations without a robot, but existing ones need external base stations, use rigid fingers that give no force signal, or use parallel jaws t...
AIRoA-0: Long-Horizon, Precise, Cross-Embodiment Manipulation from Handheld Data
Published:
What they do AIRoA-0 (A-0) learns bimanual manipulation from 16,151 hours of demonstrations recorded with YUBI, an open-source handheld gripper. It uses no teleoperation, RL, simulation or on-robot corrections. The po...
FEIJOA: Task-Driven Grounding and Execution for Mobile Manipulation
Published:
What they do Scene maps for mobile manipulation are built before the instruction is known, so they can miss what a task hinges on, such as “the plush toy with a bow tie”. FEIJOA is a language-model agent (Gemini-3.8-F...
MABEL: An Open Anthropomorphic Mobile Bimanual Platform for Teleoperation and Learning
Published:
What they do Commercial robots that combine mobility, floor-to-overhead reach, active gaze and dexterous hands cost US$69k-120k. MABEL packs all of these into a fully open design with a US$8,056 parts cost. CAD, firmw...
RAIPAL: A Powerful and Agile Humanoid Arm for Human Tasks
Published:
What they do Humanoid arms tend to be either strong and slow or fast and weak. RAIPAL (KAIST) is a 7-DoF, 8.9 kg arm built to be both. Its force, speed and endurance targets come from published human upper-limb measur...
RAFT: Reasoning-Aligned Fine-Tuning Improves Robustness of VLA Post-Training
Published:
What they do Fine-tuning a generalist VLA on a few demonstrations of a new task raises success in the training setup but makes the policy brittle to new objects, backgrounds and phrasings. RAFT adds a step before acti...
ReflexArc: Bottom-Up Integration of Instantaneous Tactile Reflexes into Whole-Body Humanoid Control
Published:
What they do Learned humanoid controllers run at 50 Hz, so a contact can wait up to 20 ms before the policy responds, and a tracking policy treats a protective withdrawal as an error to correct. ARC (Ascending Reflex ...
Spectrally Routed Diffusion Policy for Multi-Task Robotic Manipulation
Published:
What they do A diffusion policy trained jointly on several tasks usually does worse than one specialist per task. The authors blame two things: image encoders whose features do not separate tasks that share a workspac...
All-For-One: Distilling Visual Foundation Models to a Fast, Memory-Efficient Visuomotor Policy
Published:
What they do Vision foundation models (VFMs) give depth and segmentation that help manipulation, but running them in the control loop is slow, and their per-frame outputs can be wrong or flicker. All-For-One uses them...
Hybrid Particle-Voxel Dynamics for Multimodal 3D World Simulation
Published:
What they do Most action-conditioned world models predict 2D video, which makes it hard to stay consistent across viewpoints. 3D World Simulator predicts how the 3D scene evolves instead. Each state decodes into Gauss...
Physical Diffusion Policy via MPM-Guided 3D Gaussian Simulation
Published:
What they do Diffusion policies see objects only by appearance, so one policy trained on soft, foamy and rigid objects tends to average incompatible behaviors. This paper estimates each object’s material parameters fr...
Action Intent as a Bridge: Transferring Manipulation Knowledge from Human-Centric Videos to Robots
Published:
What they do Human videos are plentiful but have no action labels, and latent actions learned from them also absorb scene changes such as background and lighting. Action-Intent Transfer (AIT) bridges human and robot d...
Future Representation Distillation for Imagination-free World Action Models
Published:
What they do World Action Models (WAMs) get robot control out of pretrained video diffusion models, but usually by generating future video at test time, which is slow and fails when the predicted video is wrong. Dream...
Coordinating Humanoid Loco-Manipulation through Action-Conditioned Latent Foresight
Published:
What they do A plausible-looking humanoid command sequence can still drop a held object or leave a task unfinished. This paper samples several body-motion and hand command sequences, predicts in latent visual space wh...
Caging Vision-Language Navigation for Legged Robots via Barrier-Certified Safety Architecture
Published:
What they do Vision-language models can turn spoken instructions into navigation commands for legged robots, but those commands ignore the robot’s footprint and the obstacles around it. This paper “cages” the model: i...
HiLo-WAM: Decoupling Long-Horizon Task Evolution and Local Dynamics in World Action Models
Published:
What they do World action models (WAMs) predict future video and robot actions together, but usually over a short window, which says little about how a long task is progressing. HiLo-WAM splits the video prediction in...
Do Agentic Policies Need to Look at Every Step? Geometric Abstraction for Reasoning and Action
Published:
What they do LLM agents that drive manipulation usually re-read camera images at every decision. This paper instead gives the model an explicit 3D geometric state in text, refreshed locally from calibrated RGB-D, and ...
HLO-AMP: Hierarchical Loco-Manipulation from Object-Aware Motion Priors
Published:
What they do Humanoid loco-manipulation stacks usually put a task policy on top of a generic whole-body controller. That controller is trained without the object, with random end-effector forces standing in for the pa...
DexHOT: Learning Hand-Object Configuration Transfer for Dexterous Manipulation
Published:
What they do Most learned dexterous policies are given an object pose as the goal, so how the hand ends up holding the object is left to chance. DexHOT makes the goal a hand-object configuration: the hand’s joint angl...
WayTrace: A Map-to-Language-to-Trace Hierarchy for Embodied Urban Navigation
Published:
What they do Web maps give walking directions for almost any city, but point-goal navigation policies depend on accurate GNSS, and vision-language navigation models are brittle even on single instructions. WayTrace sp...
Asynchronous Robot Policies
Published:
What they do A standard VLA runs its vision-language model and action expert one after the other inside the control loop. Latency therefore adds up and grows with VLM size. ARP decouples the two into asynchronous proc...
RoboTryOn: Synthesizing Dexterous Robot Demonstrations from Human Videos via Virtual Try-On
Published:
What they do Teaching a dexterous robot a new task usually means teleoperating it. RoboTryOn instead has a person do the task in front of a camera at the robot’s head-camera position, then converts that video into a r...
GSBR: Geometric Support Behavior Regularization for Online RL
Published:
What they do Online RL on top of a frozen diffusion or flow-matching policy is usually regularized toward it. Behavior-cloning penalties stop the actor from dropping poor modes; hard support constraints forbid anythin...
PACT: Pose-Robust Actuation Co-Design for Tendon-Driven Robotic Hands
Published:
What they do Tendon-driven hands move their actuators into the palm and forearm to fit a human-sized envelope. But routing, moment arms and capstan sizes together set both fingertip force and joint speed, and a design...
PyRUA-DataFarm: LLM Agent as a Cost-Efficient Teleoperator for Real-Ready Robot Data
Published:
What they do Teleoperated demonstrations cost robot and operator time for every episode, and every new task starts from scratch. DataFarm gives the teleoperator’s job to a coding agent. From a data request and one sho...
GUILD: Guided Iterative Data Collection for Imitation Learning
Published:
What they do Robots are often deployed where collecting demonstrations is hard (an end-user home), while teleoperation is easy in other labs or mock setups. GUILD treats data collection itself as the optimization prob...
PolicyIK: High-Precision Inverse Kinematics for Robot Arms with Reinforcement Learning
Published:
What they do Many VLA policies output end-effector poses and leave the conversion to joint commands to an IK controller, so IK accuracy limits every policy built on top of it. On a position-controlled arm, gravity, fi...
DreamX: Video Diffusion Model is a Universal Latent Action Learner
Published:
What they do Latent action models learn actions from unlabeled video, but their latents often also encode viewpoint and scene appearance, which limits transfer between embodiments. DreamX builds both the latent-action...
Ctrl-F: Force as an Interface for Humanoid Loco-Manipulation
Published:
What they do Humanoid motion trackers reproduce where the body should be, not how hard it should push. In the authors’ tests a force-blind tracker cannot lift 1.5-6 kg boxes because they slide off its palms. Ctrl-F ad...
ELA: End-to-End Latent Action Policy Learning from Video Demonstrations
Published:
What they do Latent-action methods let a policy learn from video without action labels, but they usually learn the latent space first and train the policy on it afterwards. The resulting latents can be hard to predict...
Correct Robots Before They Make Mistakes: Proactive Human-in-the-Loop Intervention via Preference Learning
Published:
What they do Human-in-the-loop imitation learning usually collects corrections reactively: the operator takes over after the robot has started to go wrong, so the correction data are full of recovery motions. PHIL (Pr...
OUCH: Objective, Uniform, Controlled Haptic Stimulation for Automated Neurological Assessment
Published:
What they do Bedside coma assessments judge how a patient moves in response to a painful pinch or press. A clinician applies that stimulus by hand, so its force, duration and location vary between examiners. OUCH is a...
GRASP: Gradient-Guided Sampling for Contact-Rich Physics-Based Motion Retargeting
Published:
What they do Kinematic retargeting maps human motion onto a robot’s geometry, but the result need not be physically executable: a hand may follow an object without ever grasping it. GRASP is a physics-based retargetin...
RACE: Calibrated Cumulative Scoring for Rollout-Free Robot Policy Checkpoint Selection
Published:
What they do Choosing which training checkpoint to deploy usually means real-robot trials for every candidate. The cheap alternative, validation loss on held-out demonstrations, often ranks checkpoints in the wrong or...
LumiTac: Compact, High-Rate Six-Axis Optical Tactile Sensing for Human-Robot Interaction
Published:
What they do Six-axis force/torque sensing at a fingertip usually means a bulky load cell or a camera-based tactile sensor with limited update rate. LumiTac is a camera-free optical sensor, 12 mm in diameter and 4.75 ...
Learning Object-Centric Motion Priors from Human for Robotic Dexterous Manipulation
Published:
What they do Rather than copying human hand motions directly, the authors train a model on human hand-object interaction data to predict how both the hand and the object will move next. The predicted object motion the...
Fluidic Body: Swarm Robots as a Fluid Extension of the Hand
Published:
What they do Swarm interfaces usually let a person say where the robots should go, but not how the group should behave. Fluidic Body puts a simulated fluid between the hand and a tabletop swarm. Each robot is a partic...
Online Adaptation to Latent Physical Properties for Learning from Demonstration
Published:
What they do A robot cannot see where the weight sits inside a pouch or a box. It has to find out by interacting with the object. This paper learns that adaptation from teleoperated demonstrations in which the operato...
TwinBridge: Digital Twin-Driven Policy and Reward Transfer for Real-World Reinforcement Learning in Laboratory Robotics
Published:
What they do Real-world RL can learn precise lab manipulation, but it needs demonstrations of every stage and rewards that show intermediate progress. TwinBridge gets both from a digital twin of the bench: a policy pr...
$A^3$-VLA: Automatic Perception-Guided Attention Alignment for Robot Manipulation
Published:
What they do VLAs fine-tuned only on an action loss can stop looking at the object the instruction names, and pick a look-alike instead. A3-VLA adds perception supervision during fine-tuning without any manual labels....
Playing Soccer Like Humans Do: Learning Versatile and Agile Humanoid Soccer via Latent Motion Priors
Published:
What they do Earlier humanoid soccer systems cover only some skills, chain separate controllers with a state machine, or learn without human motion data and move unnaturally. SoccerBot learns moving, dribbling, trappi...
Dynamic Shape Control of Soft Trunk Robots
Published:
What they do Soft trunk robots meant for confined spaces need control over the shape of their whole body, not just the tip, but their compliance makes the shape dynamics strongly nonlinear. The authors learn an adiaba...
SCRUB: Contact-Based Surface-Frame Estimation for Force-Motion Control with an Armless Quadruped
Published:
What they do Wiping and sanding need force-motion control: press a tool along the surface normal while sliding it along the surface. On unmapped surfaces the normal is unknown, and contact-based estimators that assume...
Which Physical Tactile Cues Matter for Contact-Rich Imitation Learning? A Controlled Real-Robot Study
Published:
What they do Vision-based tactile sensors carry several physical signals in one image stream, but policies usually treat that stream as a plain RGB image. This paper asks which signal actually helps imitation learning...
PyRUA-REAL: Robot Exploration and Adaptive Learning through RSI in the Real World
Published:
What they do PyRUA-REAL studies whether coding agents can turn real-world trial and error into robot skills that last. Agents write manipulation programs for a dual-arm Franka, run them, diagnose failures and revise t...
Open-HumanWAM: An Open Recipe for Scaling Human and Robot Experience for Embodied Foundation Models
Published:
What they do An open recipe for pretraining robot policies on egocentric human video. The authors curate Open-HumanWAM-50K (50,000 hours from five public datasets, with hand-motion and language labels) and measure rob...
Reinforcement Learning Enables Scaling Generalist Robot Policy Improvement
Published:
What they do RL post-training of vision-language-action models is usually done one deployment task at a time. This paper asks whether a generalist policy can be improved online on many tasks at once, and whether that ...
Perceptive Humanoid Motion from Simulation
Published:
What they do Humanoid policies that map pixels and language to joint commands need paired data, and teleoperation or wearable capture costs at least an hour of human time per hour of data. HALO generates that data in ...
cuRoboV2-Fast: Fast Torque-Limited Collision-Free Motion Planning
Published:
What they do cuRoboV2-Fast is a single-GPU planner for collision-free, torque-limited arm motions that cuts both planning latency and execution time. The payload is part of the dynamics, so a heavy object can change t...
Spatial-UMI-Dream: Scaling World Action Models with Metric Geometry for UMI Manipulation
Published:
What they do UMI handheld grippers collect demonstrations without a robot, but their moving, close-range fisheye cameras leave scene scale and distances implicit in RGB. Spatial-UMI-Dream is a 5B World Action Model (W...
Adaptive Hybrid Force-Vision Control with EMG-Derived References for Deformable Bottle Manipulation
Published:
What they do Squeeze bottles are common lab equipment, but force feedback alone says little about how deformed a bottle is: similar forces appear at different deformation levels, and a force-controlled gripper does no...
An Empirical Study of Reinforcement Learning for Keypoint Positioning of Deformable Linear Objects
Published:
What they do A study of model-free RL (PPO) for planar manipulation of a deformable linear object (DLO) whose one end is grasped by a Franka Panda. The policy moves the end effector so that keypoints on the rope reach...
CRISP: Constraint-Resolved Interactions between Subsystems for Predictive Whole-Body Cooperation
Published:
What they do When a humanoid grips one object with both hands, the arms and the object form a closed kinematic chain. The hand forces then include an internal squeeze that moves nothing, and they feed back into balanc...
VREgo: Consumer VR Headsets for Learning Robot Manipulation from Egocentric Demonstrations
Published:
What they do Policies learned from egocentric human video usually use Project Aria glasses. Their head and hand poses come from Meta’s cloud processing, and their single camera needs a walk-up to the table to triangul...
MessyNav: Zero-Shot Navigation via Manipulation in Messy Environments
Published:
What they do Standard navigation stacks treat every obstacle as a wall. MessyNav lets a mobile manipulator decide, zero-shot, whether a blocking object should be swiped aside, picked up or avoided, based on the VLM’s ...
LMGN: A Unified Policy Learning Model for Last-Mile Grasp-Navigation
Published:
What they do A quadruped with an arm that reaches an object still has to decide where to stand, which side to approach from and when to move the arm instead of the base. LMGN learns this “last-mile” step as one policy...
Semantics-to-Contact: A Stagewise Framework for Robust Contact-Rich Manipulation
Published:
What they do Real-world RL can learn precise insertions directly on hardware, but image-based policies overfit to the look of the training scene. S2C splits each task in two. A vision-language model finds where contac...
AffectiveVLA: Safe and Intent-Aware VLA Control for Human-Humanoid Social Interaction
Published:
What they do AffectiveVLA lets a Unitree G1 humanoid with BrainCo dexterous hands shake hands, fist-bump and hug. A small video classifier recognizes which interaction the person is starting and turns it into the VLA’...
Flexible Piezoelectric Tactile Sensing System for Intelligent Object Recognition
Published:
What they do Piezoelectric tactile sensors respond to changing force. They spike when a gripper closes and opens and stay nearly silent during a steady hold. The authors put two 16-element flexible piezoelectric array...
TactileLab: Sim-to-Real Dexterous Robot Manipulation with Efficient Shear-Sensitive Tactile Simulation
Published:
What they do Rigid-body tactile simulators are fast but give only contact depth, which misses shear. Soft-body simulators capture shear but are too slow for large-scale RL. TactileLab, built on IsaacLab, adds simulate...
A Unified Fractal Impedance Control Framework
Published:
What they do Fractal Impedance Control (FIC) is a passive, asymptotically stable interaction controller whose variants have so far lived in separate implementations. This paper packages them into one C++ library with ...
HITCH: Teaching a Vision-Language Model to Operate a Humanoid through a Whole-Body Harness
Published:
What they do Pretrained VLMs understand 3D space but know little about how a humanoid body moves, and existing humanoid agents mostly call skill libraries that leave leg and torso posture to the controller. HITCH give...
Terrain-Indexed Motion Priors and Geometric Support for Extreme Humanoid Parkour
Published:
What they do Human parkour clips show how to get over an obstacle, but a robot with different proportions may need its hands and knees elsewhere. One perceptive humanoid policy is trained in three stages: make human c...
Leave It Better Than Found: Value-Guided Cooperation in Sequential Tasks
Published:
What they do When robots from different makers share a space, one robot’s actions change the scene the next one inherits. PAVE (Policy Adaptation via Value Exchange) handles this sequential task continuation without j...
STEP: Spatiotemporal Training with Explicit Prediction for Perceptive Humanoid Locomotion
Published:
What they do STEP trains a depth-based humanoid locomotion policy in a single stage: the PPO policy and its multimodal state representation learn together. Two training-only prediction losses tell the encoder which bo...
GPWM: Modeling the World as 3D Particle Motion
Published:
What they do Learned 3D dynamics models are usually deterministic, short-horizon and built for one kind of material. GPWM is a generative world model that represents a scene as tracked particles and rigid bodies and g...
MOMASPLAT: Dynamic Novel-View Demonstration Augmentation for Mobile Manipulation
Published:
What they do Visuomotor policies break when the camera moves, and a mobile manipulator rarely stops at exactly the base pose where it was taught. Earlier novel-view augmentation treats the scene as static, so it can r...
SLA: Scaling Spatial-Language-Action Model via Continual Pre-training
Published:
What they do VLA models inherit their vision from 2D image-text pretraining, and a depth probe on their frozen visual tokens recovers no meaningful 3D structure. SLA adds spatial supervision while continuing to pre-tr...
A Multimodal Haptic Interface for Simultaneous Softness and Thermal Modulation
Published:
What they do Most tactile displays render one cue at a time. This paper builds a 150 mm round display that renders softness and temperature together. A particle-jamming core sets stiffness, and a thin silicone skin ca...
Social Humanoid: Socially Responsive Human-Humanoid Interaction
Published:
What they do Social Humanoid makes a 29-DoF Unitree G1 react to a human partner in real time. A camera watches the person, a generative model produces the humanoid’s whole-body response and predicts the partner’s next...
Failure Prediction and Classification in Vision-Language-Action Models Via Calibrated Latent Probing
Published:
What they do VLA-Tug is a runtime failure monitor for vision-language-action policies. A small probe reads the VLA’s own backbone embeddings at every policy step and outputs a failure score and a predicted failure mod...
Latent RL Priors for Data-Efficient Dexterous Manipulation
Published:
What they do RL in simulation can learn rich dexterous control, but the resulting policies rely on privileged object and goal states. Latent RL Prior distills such a policy into a flow-matching action decoder, freezes...
OopsieRecover: Learning Transferable VLA Corrections from Semantic Interventions
Published:
What they do VLAs fail in systematic ways, and DAgger-style finetuning needs many corrections that mostly fix only the states where they were collected. OopsieRecover asks the human for a correction plus a sentence on...
Q-OIL: Optimistic Exploration in Learning from Human Interventions
Published:
What they do Methods that learn online from human takeovers are usually developed against simulated humans who intervene as soon as an action is suboptimal and then correct optimally. The authors log real intervention...
Meituan-Robotics-0: A Vision-Language-Action Foundation Model for Desktop Manipulation
Published:
What they do MR0 is a vision-language-action (VLA) policy plus an agent layer for stationary tabletop manipulation. The policy is trained in three stages on 147M vision-language samples and 15,060 hours of curated tra...
CS-Pi: Constraint-Satisfying Flow Policy Sampling
Published:
What they do A flow-matching policy deployed in the field often meets a constraint its demonstrations never contained, such as an obstacle next to the target. CS-Pi enforces the constraint at inference time without re...
ForPC: Force-Aware Whole-Body Control for Precise Humanoid Cart Pushing
Published:
What they do When a humanoid pushes a cart with both hands, the forces that move the cart also push back on the robot, and the payload changes how the cart responds. The cart has no motion sensors and the G1’s hands h...
VICTR: Value-Informed In-Context Learning for Vision-Language-Action Models
Published:
What they do Retrieval-based in-context learning puts a chunk of a stored demonstration into a VLA’s context. The VLA can then adapt to a new task from a few demos without fine-tuning. Earlier work (RICL) retrieves th...
STRIDE: Learning Humanoid Locomotion in Minutes with Acceleration-Based Rewards
Published:
What they do Learning to walk directly on a humanoid would avoid the sim-to-real gap. But standard locomotion rewards track base velocity, which needs motion capture or a well-calibrated state estimator, and humanoids...
HOC2R: Hand-Object Contact Trajectories for Human-to-Robot Motion Transfer
Published:
What they do Human videos show plenty of manipulation but not where a gripper should touch or how it should move. HOC2R extracts hand-object contact trajectories from such videos: the visible finger-object contact poi...
RevoSim: Scalable Multimodal Tactile Simulation for Dexterous Manipulation
Published:
What they do Training tactile policies at scale needs fast simulation of the specific sensors on a hand, but most tactile simulators cover one sensor family or a generic taxel layout. RevoSim simulates both tactile mo...
ToolDex: An Ego-Exo Human Demonstration Dataset for Creative Physical Tool Use
Published:
What they do ToolDex is a real-world dataset of humans using tools at a table, including creative substitutions such as sweeping nuts with a book instead of a brush. It separates three roles, hand, tool and target obj...
Augmentron: Scalable Multi-View Visual Augmentation for Robot Learning
Published:
What they do Robot demonstrations tend to repeat the same scenes, so policies break when task-irrelevant visuals change. A pi0.5 policy fine-tuned on their real data drops from 66.7% to 18.0% success once distractor o...
SUGARv2: Learning Diverse Visual Humanoid Loco-Manipulation Skills from Synthesized Interaction References
Published:
What they do Teaching a humanoid to pick up and carry objects from its own camera usually takes teleoperated demonstrations or a hand-tuned reward for each task. SUGARv2 replaces both with synthesized interaction refe...
MDF++: Visuo-Tactile World Action Model with End-to-end Representation Learning
Published:
What they do Threading a bolt or nut depends on contact cues that a camera alone misses. World-action models (WAMs), which predict future observations along with actions, usually work in the latent space of a frozen, ...
Horizon-Coupled Obstacle Guidance for Relative-Action Diffusion Policies
Published:
What they do A diffusion policy trained on obstacle-free demonstrations drives straight into an obstacle placed in its path at deployment. This paper steers a frozen DP3 policy around known static obstacles at inferen...
Mechanical Interaction Sensing for Robotic Kiwifruit Handling with an Optoelectronic Soft Suction Cup
Published:
What they do For handling delicate fruit, the authors adapt an optoelectronic soft suction cup to kiwifruit. Then they test how well its optical signals estimate the normal contact force on real, biologically variable...
DKWBC: Decoupling Motion Tracking and Balancing for Humanoid Whole-Body Control
Published:
What they do RL motion trackers for humanoids usually output joint position targets for a PD controller, so one policy must learn both the torques each motion needs and the corrections that keep the robot balanced. DK...
Autonomous Particle Image Velocimetry Using Zero-Shot Sim-to-Real Reinforcement Learning
Published:
What they do In particle image velocimetry (PIV), the camera position is usually fixed before the measurement, so a flow feature can fall outside the frame or be poorly resolved. This paper treats camera placement as ...
A wearable input/output interface learns personalized models of haptic perception
Published:
What they do Wearable haptics usually assume a stimulus is felt where the actuator sits, but the same electrode pair can feel different from one person to the next. This sleeve is both an output and an input device: i...
Autonomous Whole-body Loco-manipulation Using a Snake Robot
Published:
What they do A snake robot has no separate base and arm: the body that propels it is also what touches the object. Earlier snake loco-manipulation mostly used a gripper on one end, or ran whole-body loco-manipulation ...
Grasp-TAGS: Task-Aware Grasp Selection via Critic Distillation for Prehensile Loco-Manipulation
Published:
What they do Learned loco-manipulation policies that move large objects such as chairs and carts usually assume one fixed grasp. Grasp-TAGS treats the grasp as a decision. It trains one RL policy over several candidat...
On Agentic Manipulation Policies
Published:
What they do Instead of training a VLA on demonstrations, AMP1 keeps a general-purpose frontier model (Gemini 4 Argon, Claude Opus 5.5 and others) in the control loop. The model writes code that looks at the cameras, ...
UniLatent: Cross-Embodiment Transfer via Latent Observation Alignment
Published:
What they do Demonstrations collected on one robot arm transfer poorly to another, because a third-person camera sees a different arm, gripper and occlusion pattern even when the motion is the same. Prior work edits t...
BATCH: Behavior Alignment for Cross-Embodiment Learning from Humans
Published:
What they do Co-training a robot policy on human egocentric video helps, but humans and robots execute the same task at different speeds, and human clips contain segments with no robot counterpart. BATCH first finds w...
GUARD: Anomaly-Informed Q-Steering for Generative Robot Policies in Dexterous Manipulation
Published:
What they do Diffusion and flow-matching policies cloned from demonstrations drift into states their training data barely covers, and then fail. This is worst in contact-rich dexterous manipulation. GUARD keeps the pr...
Detecting and Explaining Robot Policy Failures from Sparse Internal Features
Published:
What they do When a vision-language-action (VLA) policy fails, the video shows what went wrong but not why. MIND decomposes the policy’s activations into sparse, named features and uses them both to detect a failing e...
SimDuoReal: Object-Centric Bimanual Policy for Emergent Dexterity
Published:
What they do Bimanual dexterous systems usually specify which hand acts and when the object changes hands, through task-specific rewards or demonstrations. SimDuoReal extends the object-centric interface of SimToolRea...
Deploying Physically and Epistemically Compliant Agents by Distilling Belief-Space Task and Motion Planners
Published:
What they do Some rules depend on what the agent knows, for example “do not take an irreversible action before you have the needed information”. RL rewards for such rules are hard to optimize, and online belief-space ...
Grip, Trace, Write: Force-Aware Imitation and Reinforcement Learning for High-DoF Robotic Handwriting
Published:
What they do Robot handwriting usually bolts the pen to the arm and tracks a path. Here the pen is a free rigid body held in a tripod grasp by a multi-fingered hand, so the fingers must keep hold of it while the arm a...
Tactile Sensors Are an Effective Substitute for Wrist Cameras in Contact-Rich or Occluded Manipulation
Published:
What they do Manipulation policies usually get close-range feedback from a wrist camera, but the fingers often hide the contact that matters. This paper asks whether finger-mounted touch can take the wrist camera’s pl...
Which Taxels, Not How Many: Tactile Sensor Placement for Dexterous Policies
Published:
What they do Simulation studies of tactile dexterous policies usually spread taxels uniformly over the hand. This paper tests that default. It fixes the teacher policy, the training procedure and a pool of 2000 candid...
MoE-DAgger: Transition-Aware Router-Entropy Gating for Selective Human Correction in Phase-Structured Manipulation
Published:
What they do In DAgger with human takeover, the costly part is the operator’s time, so the robot should ask for help only on rollouts that are going wrong, and early enough to matter. The authors use a policy with one...
DreamVLA: From Generated Human Motions to Humanoid Vision-Language-Action Policies
Published:
What they do Teleoperating a humanoid to collect whole-body demonstrations for a vision-language-action (VLA) policy is slow. DreamVLA replaces the operator with generated human motion: a motion model proposes how to ...
Zero-Shot VLA Adaptation with Retrieved Guidance
Published:
What they do Vision-language-action models degrade under small shifts in viewpoint, background, objects or embodiment. RAGu brings retrieval-augmented generation to VLAs without retraining: it pulls demonstrations of ...
Explicitly capturing demonstrator multi-modality for distillation into diffusion policies
Published:
What they do Test-time search and policy improvement need a policy whose samples are diverse. Standard behavior cloning data pairs each observation history with a single action, so a diffusion policy that memorizes it...
BAT-WAM: Bridging Action and Touch in World-Action Models for Dexterous Hands
Published:
What they do World-action models for dexterous hands need the hand configuration and its contacts, but joint-state and whole-hand tactile readings can be partly or wholly missing, and RGB alone does not recover them. ...
RoboMoCap: A Large-Scale Whole-Body Motion Capture Dataset with Egocentric Vision for Robot Learning
Published:
What they do Head-mounted cameras rarely see the wearer’s legs and feet, so egocentric datasets give weak supervision for locomotion. RoboMoCap records people in an optical motion-capture studio while they wear a six-...
ObjectWorld: Scaling Object-Centric World Models with A Large-Scale 3D Full-Object Flow Dataset
Published:
What they do World models for manipulation need an output a robot can act on. ObjectWorld proposes 3D full-object flow: dense 3D trajectories for points over the complete surface of the target object, including the oc...
SoftPAT3D: Physics-Augmented Text-to-3D Scene Generation with Deformables
Published:
What they do Text-to-3D scene pipelines settle their layouts with a physics engine, but every object in that stage is rigid. A generated tablecloth or pile of clothes arrives already draped, as a closed and sometimes ...
Control-Aware Command Generation for Asynchronous Control and Action Inference with Vision-Based Policies towards Fast Execution
Published:
What they do Vision-based policies such as ACT and OpenVLA are slow to execute because the robot waits for each action inference. Running inference asynchronously removes the wait, but new action chunks arrive late an...
From Festival Video to the Stage: A Sim-to-Real Ladder for Long-Horizon Expressive Whole-Body Dance on a Commercial Humanoid
Published:
What they do The paper takes one ordinary monocular video of Awa Odori, a Japanese festival dance, and deploys the full routine on a 29-DoF Unitree G1 EDU, free-standing on a hard floor. It is written as a deployment ...
MOSU2: Long-range Navigation System with Multi-modal Scene Understanding
Published:
What they do Robots that travel long distances outdoors must stay on terrain they can handle and also decide when to proceed, yield or detour around pedestrians, cyclists, cars, traffic signals and construction. A VLM...
DexForesight: Grounding Dexterous Policies with Implicit Spatial Foresight
Published:
What they do A dexterous policy needs to anticipate how fingers and scene will move, often while fingers are hidden from the cameras. World-action models usually get this by generating future frames at inference. DexF...
Off-Policy RL with Online Inference Tuning: My Solution to the CoRL 2026 RoboEval Challenge
Published:
What they do The CoRL 2026 RoboEval challenge scores eight bimanual Franka tasks in MuJoCo with the RoboEval Diagnostic Score (RDS), which combines success and stage progress with safety, efficiency and coordination m...
ForeAct: Geometry-Guided Action Learning from World Representations
Published:
What they do World action models get useful features for control by generating future video, which needs many denoising steps and carries no explicit geometry. Geometry-based policies add a separate 3D backbone and on...
Kimodo-Policy: Learning Kinematic Action Models for Humanoid Loco-Manipulation
Published:
What they do Humanoid policies usually start from a vision-language model or a video model, neither of which is pretrained on robot motion itself. This paper proposes Kinematic Action Models: use a pretrained whole-bo...
Enhancing Balance and Posture in Material Handling Tasks with a Wearable Robotic Tail
Published:
What they do Placing a load on a shelf in front of you pushes your centre of pressure (CoP) toward your toes and makes you lean forward. Instead of an exoskeleton, the authors test a supernumerary limb: a wearable tai...
HINT-VLM: Humanoid Interactive Navigation over Rough Terrain with Embodiment-Grounded Vision-Language Model
Published:
What they do Navigation Among Movable Obstacles (NAMO) has mostly been studied on flat floors, with robots that learn whether an object moves only by pushing it. HINT-VLM lets a humanoid on rough terrain estimate from...
Streaming Failure Forecasting: VLA Failure Monitors That Keep Learning After Deployment
Published:
What they do Failure monitors for VLA policies are trained offline and then frozen, but after deployment the lighting, objects, camera, task and policy version change without notice. Streaming Failure Forecasting (SFF...
Data-Driven Precision Control of Soft Helicoid Robots
Published:
What they do Soft robots are nonlinear and hard to model from first principles. This paper learns a low-dimensional model of a soft trunk robot purely from hardware data, using the theory of adiabatic spectral submani...
Accelerated Reinforcement Learning for Soft Robotics: Hours Instead of Years
Published:
What they do Soft robots lack fast, faithful simulators, and model-free RL on delicate hardware is impractical. This paper trains an RL policy entirely offline on a tiny data-driven model: the arm’s dynamics restricte...
PAIR: Proprioception-Aligned Image-Ray Representations for Viewpoint-Robust Manipulation
Published:
What they do An image is a viewpoint-dependent projection of the scene: the same robot state looks different from two cameras, and different states can look alike. PAIR trains the vision encoder of an imitation policy...
Decoupled Asynchronous System for Humanoids (DASH): Latency-Agnostic Scaling of Vision-Language-Action Models for Humanoid Manipulation
Published:
What they do In a standard VLA the vision-language backbone sits inside the control loop: while it computes, the robot keeps moving, so a bigger backbone means staler actions. DASH splits the policy into a slow VLM (S...
Design of a hybrid tendon-linkage finger with coupled and adaptive motion
Published:
What they do Actuating every joint of a robot finger independently means more motors and more tendons routed through the proximal joints. This paper builds a finger with four mechanical degrees of freedom and three ac...
Distilling Instructions into Weights: Closed-Form Edits Fix a Robot Policy’s Misreading
Published:
What they do Generalist policies often fail at one step of a task whose skills they have. This paper asks whether such a failure can be repaired by editing weights directly, with no gradients, data or prompt change. T...
Reliability-Adaptive Multimodal Relative Pose Tracking for Underwater Leader-Follower Robots
Published:
What they do For two underwater vehicles to fly in formation, the follower must keep estimating where the leader is, without GPS and with cameras that fail in dark or murky water. The follower here combines three chea...
Flip Anything with Agentic Robotics Self-Learning
Published:
What they do Flipping an object 180 degrees on a table needs different strategies for different shapes: a hammer can be handed between two grippers, a wide multimeter has to be pushed and toppled. FlipAnything is a sh...
CSG-Nav: Confidence-Aware Semantic-Geometric Fusion for Robust Exploration and Accurate Zero-Shot Object-Goal Navigation
Published:
What they do In zero-shot object-goal navigation, a robot must find a named object such as “a sofa” in an unseen building, with no task-specific training. Methods built on foundation models often feed VLM scores strai...
Thermo-VLA: A Thermal-Aware Vision-Language-Action Model with Gated Residual Modulation for Robotic Manipulation
Published:
What they do VLAs that take thermal images usually treat them as one more camera, which helps little when an instruction depends on temperature (“the coldest cup”, “the one with no heat signature”). Thermo-VLA adds a ...
Grasp-Taxonomy Conditioning as a Mode Switch for Dexterous Grasping of Climbing Holds
Published:
What they do A dexterous hand has to grasp different objects with different hand shapes, but a diffusion policy trained on several grasp types gets nothing in its input that says which one to use. BetaGrasp gives a DP...
Learning Contextual Motion Constraints for Object Rearrangement Exploration
Published:
What they do How far a drawer or door can move depends on its joint limits and on whatever stands in its way. MoCo-Net takes a partial point cloud of a cluttered scene and a query point on one movable part, and predic...
Compass 1: Towards Open-World and Persistent Embodied Navigation from Monocular RGB Videos
Published:
What they do Navigation models trained in simulation often fail on real robots: they stop early, turn the wrong way, get stuck in left-right deadlocks and barely use their history. Compass 1 trains on real monocular R...
SAM-VTP: Video Memory and Measured Geometry for Continuous Traversability Segmentation
Published:
What they do Vision-based traversability models usually predict each frame on its own, so the drivable region can flicker from frame to frame and leave the planner with no path. SAM-VTP fine-tunes SAM2’s video memory ...
SPARK: Simple Post-training for Adapting pRetrained Knowledge to Robot Control
Published:
What they do Vision encoders such as CLIP, SigLIP and DINOv2 are trained for recognition and language alignment, not for the scene changes that matter for control. Training robot encoders from scratch, on the other ha...
GAP: Generalized Adaptation Prior for Dexterous Piano Playing
Published:
What they do Piano trajectories trained in simulation lose accuracy on a real dexterous hand, and fixing them with residual RL costs a lot of hardware time. GAP learns how to adapt instead of how to act: a Gradient Gu...
Compact and Wide-Force-Range Magnetorheological Hand Exoskeleton for Kinesthetic Feedback in Teleoperation
Published:
What they do Force-feedback gloves for teleoperating robot hands have to trade force range against size and weight. This paper builds a five-finger hand exoskeleton whose feedback comes from small magnetorheological (...
LHNav: Long-Horizon Humanoid Navigation via Multirate Coordination and Physics-Grounded Action Data
Published:
What they do A humanoid gets one long language instruction that implies several destinations in order (bring the notes to the bed, then leave them on the desk) and has to walk to each of them through a home. LHNav spl...
Embodied First Exam
Published:
What they do Coding agents are starting to be used to control robots, but robotics benchmarks are built for visuomotor policies. The connects unmodified coding agents (a model in a harness such as Codex or Claude Cod...
RAIL: Representation-Aligned Imitation Learning for Student-Compatible Teacher Policies
Published:
What they do Teacher-student RL trains a teacher on privileged state and distills it into a student that sees only deployment observations. When the teacher acts on information the student cannot recover, no student c...
Be My Eye: Learning to See Beyond a Robot’s Own Perspective
Published:
What they do In multi-robot work, one robot often sees what its partner cannot. Instead of sending raw images or generic visual features, each robot encodes its camera views into a few task-conditioned tokens that the...
ExoStack: A Modular and Reconfigurable Software Framework for Wearable Robotics Using ROS 2
Published:
What they do Personalizing an exoskeleton needs more than a joint controller. Body-worn boards must share one time base, gait events must trigger assistance, and each measured cost must be credited to the parameters t...
Rethinking Radar 3D Perception: Representation Learning from Raw Spectra
Published:
What they do Radar perception usually reduces the raw spectrum to a sparse point cloud (for example with CFAR detection) before any learning, which fixes in advance what counts as noise. The authors learn from the unr...
PCTac: Mechanochromic Photonic-Crystal Tactile Sensor for Dexterous Manipulation
Published:
What they do GelSight-style sensors image how a gel deforms, and force has to be inferred from that shape. PCTac replaces the gel coating with a mechanochromic photonic-crystal (PC) film: pressing on it changes its pe...
Bridging Task Planning and Execution for Language-Conditioned Bimanual Manipulation
Published:
What they do VLM task planners reason about what to do but usually leave implicit how each action can physically be carried out, so a box wedged between two others looks graspable to them. This zero-shot framework (no...
VLA Adaptation with Real and Simulated Data: A Case Study on Robotic Chess
Published:
What they do The paper asks how much data and training a VLA needs for a new task, how to pick a checkpoint, and whether simulation can replace real demonstrations. The test task: an SO-101 arm picks up a chess king a...
Benchmarking Real-World Robotic Manipulation through Stochastic Challenges
Published:
What they do Real-world manipulation results are hard to compare across labs: fixed object sets invite overfitting, and existing distributed benchmarks fix the robot or the task and stay on the tabletop. This paper st...
Learning Heterogeneous Tactile Representation for General Dexterous Manipulation
Published:
What they do Tactile sensors differ in physics, layout, resolution and rate, so data from one cannot be pooled with data from another. LingBot-Touch is a tactile encoder trained once on 4,983 hours of fingertip record...
SWAM: Scaling World Action Models with Cross-Modal Alignment
Published:
What they do A world action model (WAM) starts from a pretrained video generator and adds action prediction. SWAM asks which data and supervision keep the video model’s language-video alignment intact while actions ar...
World In-Context Learning for Agentic Robot Manipulation
Published:
What they do Trained manipulation policies store skill in weights and lose it under distribution shift. In-context learning from one demonstration avoids retraining, but the demonstration carries no feedback from the ...
Robo-MLT: Dual-System Robotic Manipulation for Long-Horizon Medical Laboratory Tasks
Published:
What they do Clinical lab procedures are short but strictly ordered, and different stages can look the same: a tube at the analyzer inlet must be inserted before the assay and withdrawn after it. Robo-MLT keeps the pr...
Ground Reaction Force-Aware Model Predictive Control for Underactuated Upright Physical Assistance
Published:
What they do Robots that support a person standing or moving upright usually track a reference with impedance control: they react to position errors but ignore the forces the person is putting into the ground. GRF-MPC...
Incorporating World Models in Decision Making for Next Best View Selection in Active Sensing
Published:
What they do A pretrained world model can predict which surface a robot-mounted depth sensor would see from a candidate pose. A visible surface does not always give valid depth, though: on reflective parts a structure...
Curie’s Jarvis: Autonomous Robotic Chemistry Experiments from Printed Protocols
Published:
What they do Lab automation usually adapts the experiment to the robot: scripted procedures, fixed container positions, fiducial markers. Curie’s Jarvis goes the other way. A Franka Panda with four RealSense D415 came...
AHEADance: Anticipatory Humanoid Dance from Streaming Audio
Published:
What they do A humanoid dancing to live music has to commit each chunk of motion before it hears the music that chunk goes with, and it cannot revise motion it has already sent to the robot. AHEADance generates refere...
Rhythmic-Semantic Co-Speech Gestures for Emotional Dialogue in Robots
Published:
What they do Robot co-speech gesture systems usually produce either rhythmic beat motion or iconic gestures that carry meaning, rarely both. This paper generates both in one stream for the non-humanoid social robot Ha...
ExRL: Sample-Efficient Online Adaptation of Long-Chunk Policies via Execution Length Control
Published:
What they do Policies such as pi0.5 predict chunks of up to 50 actions. RL methods that adapt them by optimizing whole chunks, or their latent noise, face a search space that grows with the chunk length and need many ...
Few-Shot Residual Adaptation of Pretrained Latent World Models
Published:
What they do A latent world model trained under one set of physical conditions can fail when an object’s dynamics change at deployment. This workshop paper keeps a pretrained model (DUET-DINO, a V-JEPA 2-AC variant) f...
EgoHMP: Achieving Precise Human Motion Prediction in 3D Scenes via Egocentric Cues
Published:
What they do Predicting how a person will move through a room depends on what they intend to interact with. Recent methods infer that intent from eye gaze, which drifts, is ambiguous and needs an eye tracker. EgoHMP i...
A computational framework for understanding embodied plans
Published:
What they do People read goals and plans off the bodily actions of others. Existing computational accounts treat the agent as a point in a 2D grid world. GenEA (Generative Embodied Agent) is an agent model with an art...
Ego Guidance: Enhancing Robot Policy Learning with Action-Free Human Egocentric Videos
Published:
What they do Human egocentric video is plentiful but mostly has no action labels. The paper asks whether it can still help a robot policy without recovering actions. Its model, SPAR, trains a VLM backbone on two predi...
CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation
Published:
What they do Robots record actions in different coordinates, dimensions and control rates, so demonstrations from one robot do not transfer directly to another. CIALLO is an action codec that separates a shared, short...
Camera equalisation of an 11.5x drop in environmental illuminance: four imitation-learning policies on an SO-101 arm
Published:
What they do A real-robot check of a common assumption in policy benchmarks: that logging room illuminance in lux describes the lighting a policy sees. Four policies trained on the same 50 teleoperated demonstrations ...
ReACT: Reflection-Consistent Modulation and Cross-Load Consistency for Humanoid Velocity Tracking Under Asymmetric Loads
Published:
What they do A humanoid that carries an object in one hand has a persistent asymmetric upper-body load. Walking stays stable, but velocity tracking picks up lateral and yaw-rate biases that accumulate into position an...
Optimization-Based Path Planning for Industrial Manipulators: Experimental Validation in Additive Manufacturing
Published:
What they do In robotic additive manufacturing the process dictates how the tool or workpiece moves. The robot still needs a joint path that respects joint limits, self-clearance and smoothness. The authors build one ...
LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
Published:
What they do World action models (WAMs) predict future video and robot actions together, and usually pass global scene features straight to the action head. LaST0.5 inserts an intermediate “latent expert” that turns t...
RedirectMimic: Local Motion Content Redaction for Humanoid Whole-Body Control
Published:
What they do A humanoid motion tracker normally reproduces whatever reference it is given. RedirectMimic edits the policy so that a specified arm movement (an overhead reach, a fast outward extension) is replaced by a...
A human-in-the-loop Control Paradigm for Construction Robotics: From VR-Guided Teleoperation to Semi-Autonomous Dual-Arm Operation
Published:
What they do Construction tasks such as drilling a facade and fastening brackets are contact-rich and precise, which makes pure teleoperation hard and full autonomy brittle. This system lets an operator move along a s...
DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
Published:
What they do Teleoperating a dexterous hand to collect VLA training data is costly. DexSeed starts from human hand-object recordings (GRAB) instead and expands each one into many robot pick-and-place demonstrations th...
LEASH: Stay Safe in Agentic Robotics
Published:
What they do Agentic harnesses now let foundation models such as GPT and Claude drive robot arms directly. Classical safety checks such as joint limits and collision-free paths do not catch semantic hazards like putti...
Elastic-WAM: Adaptive Spatiotemporal Imagination for Efficient Robot Control
Published:
What they do World-action models (WAMs) predict future video alongside actions, and denoising that video dominates inference cost. Existing shortcuts spend a fixed refinement budget in every state. On 10 LIBERO-Plus t...
From Prediction to Action: Learning Physical Context for Deformable Linear Object Manipulation
Published:
What they do Cables and ropes with the same shape respond differently to the same action, depending on stiffness, friction and interaction history. The paper asks whether the information that best predicts a deformabl...
From Video Frames to Robot Meta-Frames: A Unified Latent Interface for World Action Models
Published:
What they do World Action Models adapt a pretrained video diffusion model to predict robot actions and future observations together. Low-dimensional signals (proprioception, action chunks, value) must enter a latent s...
Gaussian Process-Augmented Model Predictive Contouring Control for 6-DoF Trajectory Tracking with a Holonomic AUV
Published:
What they do Small holonomic AUVs such as MBARI’s MOLA can control position and attitude independently, which suits close-range inspection, but their hydrodynamics are hard to identify. The authors run a nonlinear mod...
SIGMO: Sigmoid Gated Interaction-Aware Motion Planning
Published:
What they do In a narrow street or a packed merge there is often no gap that is safe on its own, so the ego vehicle has to negotiate: push in when others will yield, back off when they won’t. SIGMO adds this to a stan...
One Policy for 100 Household Tasks: Our Solution for the 2026 BEHAVIOR Challenge
Published:
What they do A report on the 9th-place entry (the only independent team in the top 10) to the 2026 BEHAVIOR Challenge: 100 long-horizon household tasks for a Galaxea R1Pro in the OmniGibson simulator, with 200 teleope...
UGG: Unified Generative Geometry Model
Published:
What they do Feedforward models predict depth and camera pose from video; generative models synthesize new views. UGG treats both as queries to one generative model of a 4D scene. RGB video, depth and camera pose are ...
SunLightBots: Turning sunlight into self-sustained, high-power motion in soft robots
Published:
What they do Light-driven soft robots harvest energy at low power, but jumping or fast propulsion needs high power. SunLightBots are subgram, untethered robots without electronics that run under continuous simulated s...
Open-Source Transtibial Prosthetic Bypass Preserving Joint Alignment: Gait Learning and Kinematic Evaluation
Published:
What they do A prosthetic bypass lets an able-bodied person walk on a prosthesis, so new prosthetic devices can be tested before trials with amputees. Existing transtibial bypasses either mount the prosthesis under th...
Learning Low-Latency Action Verification for Frozen Vision-Language-Action Models
Published:
What they do VLA policies such as pi0.5 can sample many different action chunks for the same observation, but deployments usually execute just one. HDH (Hindsight Decision Model with History) samples two candidates fr...
Adapting Monocular Depth Estimation in Novel Scenes through Disparity Refinement and Robot Ego-Motion
Published:
What they do Depth foundation models give good relative depth but no reliable metric scale in new scenes. EM-ViT adapts one while the robot moves: Depth Anything V2 (ViT-S) stays frozen, and a 912K-parameter head is r...
From a Single RGB Image to Robot Execution: Metric Scene Alignment and Contact-Guided Retargeting
Published:
What they do Generating humanoid motions that interact with a scene, such as climbing onto a box, usually needs a prebuilt 3D model of that scene. This pipeline starts from a single RGB image instead. It reconstructs ...
ILA: A Transferable Interface-Level Adapter for Compliant Humanoid Motion Tracking
Published:
What they do Whole-body motion trackers follow their reference closely, so they resist a person guiding the robot’s arm. ILA (Interface-Level Adapter) adds compliance without retraining the tracker, estimating wrenche...
Fast Inference-Time Policy Refinement via Action-Affine World Models
Published:
What they do Imitation policies fail when small execution errors push them off the demonstrated states. A2WM corrects a frozen policy at test time without on-policy residual training or sampling many candidate actions...
Genie Sim Benchmark: Real-to-Sim Fidelity and Evaluation
Published:
What they do A simulation benchmark is only useful if its scores say something about real robots. This report tests Genie Sim Benchmark, which has 78 simulated manipulation tasks covering instruction following, spatia...
Does Your World Model Know Actions Have Consequences?
Published:
What they do World models are valued because they can imagine what happens under alternative actions, yet they are usually judged by reconstruction quality, prediction error or task success. This short paper argues th...
MiniPose: Miniature-Based Real Data Collection for 6D Pose Estimation of Large Industrial Pipe Segments
Published:
What they do Learned 6D pose estimators have mostly been developed on small objects like those in YCB-Video. Industrial pipe segments of up to 1.5 m x 1.5 m x 0.5 m and about 10 kg are hard to reposition for real trai...
CrowdMAPS: Crowdsourcing Multiple Actions per State for Real-World Imitation Learning
Published:
What they do A teleoperated demonstration records only the one action that was executed at each state, although many states admit several valid next actions. CrowdMAPS keeps the real robot for collecting states and cr...
Decide Before You Move: Diffusion-Generated Search Reference for Zero-Shot Open-Vocabulary Object Navigation in Semi-Static Environments
Published:
What they do The robot must find an object given only by name, such as “tray with fruits”. Walls and furniture rarely move but small objects do, so PreNav keeps only a map of walls and furniture boxes and plans the wh...
Elastra: A Simulation Framework for Elastic Bodies
Published:
What they do Robot learning environments usually treat the ground as rigid. Mattresses, cushions and trampolines instead sink where they are pressed and recover when the load moves on, so every push changes where the ...
PolyMorphWBC: Whole-Body Control from End-Effector Trajectories for Quadrupedal Mobile Manipulation across Arm Morphologies with Extended Reach
Published:
What they do Visuomotor and VLA policies increasingly output short end-effector (EE) trajectories. On a quadruped with an arm, someone still has to decide when and how the base should move, and that depends on which a...
ARTS: An Open-Source, Reproducible Three-Arm Platform for Autonomy Research in Minimally Invasive Surgery
Published:
What they do Learning-based surgical autonomy needs data, but most of it comes from the da Vinci Research Kit (dVRK), which runs on retired clinical hardware that labs cannot buy. ARTS is an open three-arm platform bu...
Autonomous Trail Racing with a Bicycle Robot
Published:
What they do Racing a two-wheeled, statically unstable robot down a mountain-bike trail couples perception and agile control. Instead of a planner sending velocity commands to a locomotion policy, the authors encode t...
APG-Planner: Learning Vision-Based High-Speed Trajectory Planning via Analytic Policy Gradient
Published:
What they do Learned planners for vision-based high-speed flight are usually trained by imitating an expert, which is limited by the demonstrations, or by guidance from a planning cost that stays local to each decisio...
BiPOP: Partial-Order Planning for Long-Horizon Bimanual Robotic Manipulation
Published:
What they do Many long-horizon bimanual tasks are a sequence of one-arm skills plus a few two-handed ones. The hard part is deciding which arm does what, in which order, and what can safely run at the same time. BiPOP...
Interactive Perception of Granular Materials via Robotic Scooping and Force-Torque Sensing
Published:
What they do How a robot should carry a spoonful of rice or flour depends on properties such as grain size and density. This work infers them by touch. A Franka Panda scoops material with a spoon mounted on a 6-axis A...
Bind-WAM: Binding-Uncertainty-Guided Selective Future Imagination in World Action Models
Published:
What they do World action models (WAMs) predict future video and actions together, but imagining the future at every replanning step is slow and not always useful. Bind-WAM imagines only when language-object binding u...
Low-Cost Mobile Bimanual Manipulation for Long-Horizon Toilet Cleaning
Published:
What they do They build a low-cost, reproducible mobile bimanual robot and use it to compare open vision-language-action (VLA) models on the same robot, data and task. The task is long-horizon toilet cleaning in five ...
Actuation and Control of Passive Tendon-Driven Soft Extensions Through Rigid Manipulator Motion
Published:
What they do A soft extension on an industrial arm combines the arm’s reach with compliant access to confined spaces, but usually needs its own tendon motors. Here the extension is passive: each tendon runs from an an...
ActionCodec2: A Streaming Action Tokenizer
Published:
What they do Most action tokenizers for autoregressive VLAs encode one to two seconds of motion as a chunk that must be fully decoded before any action runs, and small per-step errors add up: in LIBERO replays, FAST r...
What Makes a VLA Reactive? A Study of Execution Strategies for Dynamic Manipulation
Published:
What they do VLAs predict chunks of future actions. When objects move, a chunk can go stale before it has finished executing. This study takes one pretrained VLA, π0.5, and asks what makes it react to motion. It runs ...
Reading the Table: Sim-to-Real Game State Detection for the Chef’s Hat Card Game
Published:
What they do To play the Chef’s Hat card game with people, a robot has to read the game state: which of 18 card designs lie in which of 11 zones of the playing field. The 13 ingredient cards share an almost identical ...
Generalized Inverse Kinematics via Conditional Flow
Published:
What they do Planners need many IK solutions, not one: alternative base placements, starts from which a whole path stays reachable, collision-free postures in clutter. GenGenIK samples whole-body configurations, or tr...
Helping Hand: A Continuum Robotic Grasper for Traction in Single-Channel Endoscopic Submucosal Dissection
Published:
What they do In endoscopic submucosal dissection (ESD), a physician cuts a large polyp out of the colon wall in one piece. Pulling on the partly freed flap (traction) keeps the cutting plane visible. However, existing...
Single-View RGB-D Force–Deformation Estimation for Robotic Manipulation of Elasto-Plastic Materials
Published:
What they do Elasto-plastic materials such as clay and dough keep their deformation, so every push changes the shape the robot sees next. This paper measures how a robot’s intrinsic force estimate relates to the persi...
Recover, Don’t Just Continue: Learning from Feedback for Physical Failure Understanding and Correction
Published:
What they do When a robot fails partway through a task, a planner that only sees the instruction tends to repeat the same command. Embodied-Corrector is a multimodal model that runs after each subtask. It reads the be...
Embedded Sim-to-Real Visual Servoing: Learned Reaching and Force-Guided Placement
Published:
What they do They build one embedded pipeline for a pick-and-insert task on a Franka Research 3. A wrist RealSense D435i and a Jetson AGX Orin turn a segmented object or hole into a metric target in the robot base fra...
GraSLA: A Lightweight Spatial-Language-Action Model with Grasp Hypotheses as Geometric Priors
Published:
What they do Policies that plan to a predicted grasp pose can choose among many grasps but execute open-loop. Closed-loop continuous-action policies react to the scene but are limited by how varied their demonstration...
RUMI: A Robotiq-Matched Universal Manipulation Interface for Robot Learning with Augmented Reality and Haptic Feedback
Published:
What they do RUMI is a handheld data-collection gripper that copies the Robotiq 2F-85’s nonlinear finger kinematics, finger mount and wrist-camera pose, so demonstrations match a common industrial gripper. The same de...
Runtime Safety and Emergency Stops on a Full-Size Humanoid: Deployment Lessons and a Case for Graduated Force Authority
Published:
What they do Golem is an open-source ROS 2 autonomy stack for the 1.78 m, 70 kg Unitree H1-2, built for disassembling electric-vehicle battery packs. This extended abstract covers its safety parts: a runtime safety la...
MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
Published:
What they do VLAs such as π0.5 act from 2D images and language, so in clutter they must infer obstacle geometry implicitly, which can lead to collisions. MOSAIC adds modules to a pretrained VLA that make the target ob...
Video-Action Models for Data-Efficient Industrial Manipulation
Published:
What they do Industrial tasks such as inserting parts into tight fixtures, routing wire harnesses and kitting foam parts look nothing like the household data that generalist robot policies are trained on, and demonstr...
DexRemo: A Force-Sensing Hand Platform and Task Suite for Contact-Rich Removal Manipulation
Published:
What they do Peeling a glued part off a build plate, scraping off Blu-Tack, wiping a stain or digging a fossil out of sand all depend on contact states that cameras and joint encoders barely see. DexRemo is a low-cost...
JustRUA: A Controlled Study of GPT-6 Robot-Use Agents
Published:
What they do GPT-6 Astra can drive a bimanual arm with no robot training: a harness turns each of its decisions into end-effector targets for a motion planner. Systems built this way keep raising scores on the RoboDoj...
Visibility-Aware Task and Motion Planning with Task-Centric Perception
Published:
What they do A task and motion planning (TAMP) solver plans as if it knows every object’s shape, which fails when objects are occluded or only partly in view. VA-TAMP keeps an off-the-shelf, fully observed solver (cuT...
Robot Navigation and Obstacle Avoidance from Randomly Positioned External Cameras
Published:
What they do Indoor rooms often have fixed cameras that see the whole layout at once. The authors ask whether a robot can navigate to a point goal using only such external cameras. Plain PPO, which works with an onboa...
Alloy: A Multi-Material Physics Engine for Robot Learning
Published:
What they do Simulators for robot learning at scale are strongest on rigid bodies, and adding cloth, cables or liquids usually means a second engine plus hand-written glue. Alloy puts all of these materials in one sce...
Memory-Efficient Parallel Policy Learning for Collapse-Reactive Retreat via Hybrid Terrain Modeling
Published:
What they do At disaster sites, ground can give way only once a robot is on it. The paper trains one proprioceptive policy for a Unitree Go2W wheeled quadruped that drives normally and, when a wheel loses support, kee...
ManiLadder: Benchmarking Robot Manipulation Through a Categorized and Multi-Difficulty Level Task Ladder
Published:
What they do Most simulation benchmarks add breadth, with more scenes and more pick-and-place variants, but do not show how hard a task a method can solve. ManiLadder is a benchmark of 112 tasks in the Genesis simulat...
I See What You Want: Decoding Hierarchical Operator Intent from Visual Teleoperation History
Published:
What they do Can a model watching a teleoperated robot tell what the operator is doing and why, with no instruction given? The authors split intent into the ongoing action (“pick up the apple”) and the mission it serv...
BlinkMimic: Training a Humanoid Motion Tracker in Minutes on a Single GPU
Published:
What they do Training a humanoid whole-body motion tracker for a library of motions has usually taken hours of GPU time. BlinkMimic builds on SuperMimic (backpropagating through a learned world model instead of policy...
VOKE: Visual Object Kinematics Estimator for Dexterous In-Hand Manipulation
Published:
What they do Pose estimators report where an object is, not how it moves, and differentiating noisy pose estimates amplifies the noise, especially for acceleration. VOKE attaches to an existing hand-object pose estima...
HALO: Human-Aligned Data Collection for WhoLe-body MObile Manipulation
Published:
What they do VR teleoperation gives indirect control without force feedback, and Mobile ALOHA’s leader arms keep the operator away from the grippers while the powered base gets in the way. HALO has the operator move t...
STVLA: Spatial-aware and Temporal-memory for Vision-Language-Action Models
Published:
What they do Many vision-language-action policies act on the current frame alone. They lose track of which steps are already done, and 2D features say little about 3D structure such as the height of a surface. Memorie...
Reins: Frontier Multimodal Models Driving Real Robot Arms on Long-Horizon Tasks, Zero-Shot
Published:
What they do Frontier models can plan and call tools, but on real robots they struggle with long-horizon, contact-rich tasks, partly because they are blind while the arm moves and while they think. Reins is a harness ...
Safe and Efficient Navigation Considering Texting Pedestrians via Attention-Aware Cost-map Tuning
Published:
What they do Social navigation planners usually assume every pedestrian will notice the robot and help avoid it. Someone texting while walking often won’t. This paper detects whether each oncoming pedestrian is distra...
EgoX: Egocentric Cross-Embodiment Manipulation with Embodiment Dreaming
Published:
What they do Humans, robots and simulated agents produce demonstrations with different bodies, cameras and action spaces. EgoX treats them all as embodiments of equal standing: it pretrains one policy, MXT+, on their ...
ACE-CaP-0: Real-World Code-as-Policies in the Era of Experience
Published:
What they do With Code as Policies, what a coding agent works out while developing a skill is kept in a program that later runs without model calls. Physical trials are expensive, so the open question is how the agent...
An Integrated Architecture for Data-Efficient Real-Time Neural Imitation of Nonlinear Time-Optimal Model Predictive Control
Published:
What they do A time-optimal MPC can swing a Furuta pendulum up and stabilize it within one 60-step horizon, but a solve takes 114 ms on average and up to 733 ms. The authors imitate it with a small neural network. Exp...
Your Vision Foundation Model Is Secretly an EEG Foundation Model
Published:
What they do EEG foundation models are pretrained on thousands of hours of recordings, yet their frozen features transfer poorly to new tasks. SeeEEG instead uses a frozen image model, DINOv3 ViT-B/16, that has never ...
SALA: Semantically Aligned Latent Actions Enable Dual-Axis Scaling of Embodied Foundation Models
Published:
What they do Latent action models let a policy learn from videos without action labels, but latents learned only by reconstructing each clip tend to cluster by scene rather than by action, so a human doing a task and ...
FORGE: Force-Aware Multi-Granularity Control for Efficient Humanoid Loco-Manipulation
Published:
What they do FORGE is a single whole-body RL policy for a 29-DoF G1 humanoid that accepts base velocity, planar footholds, bilateral wrist poses and an optional desired wrist force, alone or combined. It is trained wi...
Inner Product Regularization for Imitation Learning
Published:
What they do Policies learned by imitation often react to state changes that do not matter, such as a fruit moved on the table. Existing regularizers penalize the norm of the policy Jacobian (or Fisher information) at...
UAV-Flow++: A Unified Suite for Benchmarking and Learning Language-Guided UAV Flight
Published:
What they do Language-guided drone research has mostly targeted long-horizon route planning in simulation. UAV-Flow++ targets short maneuvers relative to what the drone sees, such as “Approach the right side of the si...
HEIR: Harness Egocentric Intent for Human-Robot Interactions
Published:
What they do A home robot has to work out what a person means (“bring me that thing”) and then carry it out over several navigation and manipulation steps. HEIR is a dataset for this. A participant wearing eye-trackin...
Racing the Clock: Teaching VLNs Budget-Conditioned Object Search
Published:
What they do Vision-language navigation (VLN) models are usually run under a step limit they never see. This paper defines budget-conditioned object search: given a target category and a countdown of remaining steps i...
Grounding Object-to-Object Hazard Reasoning into 3D Risk Fields for Task-Conditioned Safety
Published:
What they do Some manipulation hazards come from how objects interact: carrying coffee over a laptop risks a spill, and a metal cup in a microwave is a fire hazard whatever the path. A VLM sees the scene and the task ...
Natural Swing-Up of an Underpowered Double Pendulum
Published:
What they do The direct-drive motors of this planar two-link robot deliver 1.1 Nm, about 6% of the shoulder’s gravity torque. At full torque, link 2 tilts only 16.3°, so the robot must build up energy over many swings...
X-Nav Technical Report: Real-World Long-Horizon Indoor-to-Outdoor Navigation on a Humanoid Robot
Published:
What they do Vision-language navigation models score well on 10-15 m indoor benchmark episodes, but real humanoid runs have stayed short. X-Nav takes a bipedal Lingxi X2 about a hundred metres from an office lobby, th...
TouchTwin: Scaling Human Touch Data via Real-to-Sim-to-Real
Published:
What they do Real-to-sim-to-real pipelines multiply human demonstrations by rebuilding them in simulation, but they rebuild the interaction from vision. A hand pose shows where each finger stops on the surface, not ho...
On the Utility of Pre-Trained Visual Representations for Policy Learning of Contact-Rich Insertion Tasks
Published:
What they do Studies of pre-trained visual features for robot policies span many tasks and reach conflicting conclusions. This paper narrows the question to tight insertion. It adds pre-trained or hand-designed featur...
ERA: Error-Resistant Autonomous Object Navigation with Predicted Depth
Published:
What they do Zero-shot object navigation systems build their maps from a depth sensor. Swapping in depth predicted from RGB by Depth Anything 3 breaks them: VLFM falls from 68.2% to 10.8% success on HM3D v2. The autho...
Multimodal Person Concept Model for Person Identification under Varying Observation Conditions
Published:
What they do A service robot should recognize people it has talked to, though it sees each briefly and may meet them again in other clothes or a noisy room. This paper forms “person concepts” from a few minutes of mul...
Reusing Locomotion Training Recipes Across Humanoids via Body-Aligned Virtual Joints
Published:
What they do Humanoids mount their leg joint axes differently (oblique hips, inclined pitch axes, different orderings). RL locomotion policies output native joint angles, and action scales, bounds, gains and penalties...
PoliMop: Policy-in-the-Loop Motion Planning in 3D Spaces for Mobile Robots
Published:
What they do Whether a robot can reach a goal depends on its body, its control policy and the current scene. PoliMop is a kinodynamic RRT planner that puts a physics simulator (MuJoCo) and the robot’s deployed locomot...
Objectives as Policies: Vision-Language Model Cost Programs for Contact-Rich Manipulation
Published:
What they do Contact-implicit MPC handles contact well but needs costs engineered for each task and scene; vision-language-action models bring semantics but no physics model. Objectives as Policies (OaP) has a VLM wri...
PROOF: Perceptive Robustness via Out-of-Distribution Fallback for Legged Locomotion
Published:
What they do Depth-based parkour policies fail when the camera is occluded, overexposed or sees reflections, because the corrupted image drives the actions. PROOF trains a quadruped parkour policy that detects out-of-...
Heavy Metal Acrobatics: Learning Dynamic Control for Skid-Steer Loaders
Published:
What they do A skid-steer loader has a hydraulic drivetrain with strong delays and nonlinearities, and it steers by giving its left and right wheels different speeds. The authors train RL policies in simulation that m...
When Will You Hand It Over? A Hidden Start Delay Makes Self-Play Handover Robust to Unfamiliar Partners
Published:
What they do Cooperative self-play can teach a humanoid to take an object from a giver without any human data, but the trained pair may settle on a private convention. In a simulated parcel handover between a fixed-ba...
Toward Resilient Self-Driving Laboratories through Telerobotic Interventions
Published:
What they do Self-driving labs run experiments with robot arms, but a dropped vial or a missing holder halts the workflow. This paper studies remote teleoperation as the recovery step: an operator takes over the lab’s...
Disturbance-Aware Informative Planning in Spatiotemporal Environments
Published:
What they do A robot that persistently monitors a changing quantity, such as temperature or water quality, loses information over time and has to keep revisiting regions. Unknown, spatially varying wind pushes it off ...
Modeling the Frictional Contact of Continuum Robots with Environments through Linear Complementarity Problem
Published:
What they do Continuum robots such as concentric-tube robots and tendon-driven catheters often rub against their surroundings, and that friction can move the tip by centimeters. Earlier models assume the robot always ...
FETTUCCINE: Fast and efficient brain-to-text decoding on mobile devices
Published:
What they do Speech neuroprostheses decode attempted speech from electrodes implanted in motor cortex. The standard pipeline (a GRU phoneme decoder, then an n-gram language model and LLM rescoring) needs a workstation...
Training Reasoning Models in Simulation to Natively Use Real Robots
Published:
What they do Robot-use agents let a vision-language model operate a robot directly, but so far only proprietary frontier models do this well. The authors train an open-weight reasoning model, Qwen3.8-27B, to use robot...
TMA: Sim-to-Real Forceful Humanoid Manipulation via Tactile-Driven Motor Adaptation
Published:
What they do Opening a door with a round knob is hard for a humanoid hand. The fingers must hold a frictional grasp, build up enough torque to release a spring-loaded latch without slipping, and then push the door whi...
Leaning Over to Reach Far-Away Objects
Published:
What they do People reach far across a table by leaning in and bracing a forearm on it. This paper gets a Unitree H1-2 to do the same with an online whole-body controller instead of a learned policy or a precomputed c...
Ego2Sim2Humanoid: Agentic Real-to-Sim for Learning Loco-Manipulation VLAs without Robot Demonstrations
Published:
What they do Humanoid VLAs are usually trained on teleoperated robot demonstrations. E2S2H uses recordings from Project Aria Gen 2 glasses instead, which capture RGB-D video, the head trajectory and in-view hand poses...
Continuous Lower-Limb Joint-Angle Estimation from sEMG and IMU: The Conditional Value of Cross-Activity and Cross-Subject Data under Update-Matched Controls
Published:
What they do Exoskeletons driven by the wearer’s intent need a continuous, causal estimate of joint angles, yet a new user brings little calibration data. This study asks which extra data are worth adding: the user’s ...
Towards Probabilistically Safe Reinforcement Learning: Theory and Deployment on Humanoid and Aerial Robots
Published:
What they do Safe RL methods with guarantees usually handle only expected-cumulative-cost constraints and assume unbiased gradient estimates, which rules out PPO. The authors extend robust safe gradient flow RL (RSGF-...
SGIP: Learning Scene-Grounded Interaction Priors for Scene-Aware Human Motion Prediction
Published:
What they do In a cluttered room the same observed approach could end at one of several chairs, at the sofa, or nowhere in particular. Earlier scene-aware predictors leave this choice implicit or use gaze, which needs...
Cleanly-DINO: Efficiently Learning Visual Task Spaces for Robotics
Published:
What they do In tasks like cleaning, progress shows up as a change in appearance, not in an object’s pose. Cleanly-DINO learns a per-patch progress score from a handful of demonstration videos and a mask of the task r...
DAPPT: Domain-Aware Policies from Procedural Trajectories for Sim-to-Real Dexterous Tool Use
Published:
What they do Tool use with sand, dough or liquids is slow to simulate and hard to transfer, which rules out large-scale RL. DAPPT never simulates the non-rigid material. It simulates only the rigid tool and trains a p...
ARMFUL: Learning Whole-Arm Grasp Configurations for Single-Arm Pickup of Large Objects
Published:
What they do Chairs, storage boxes and racks are too big for a parallel-jaw gripper, but a single arm can pick them up by wrapping its links around them. Such wraps are hard to teleoperate, and existing datasets cover...
SG-ODOM: Event-Driven Proprioceptive State Estimation with Stability-Gated Contact Updates for Legged Robots
Published:
What they do Proprioceptive odometry for legged robots fuses IMU propagation with leg kinematics and assumes a foot in contact is stationary. Right after touchdown, though, a foot can still bounce or settle, and repea...
Multi-CONDOR: Learning Multistable Motion Primitives from Demonstration
Published:
What they do Motion primitives built as stable dynamical systems usually converge to a single goal. Many tasks have several valid endpoints (either handle of a container) or a continuum of them (any point along a rim)...
ScaleVLA: Scaling Vision-Language-Action Models to 552 Billion Parameters
Published:
What they do Most VLAs build on VLMs with a few billion parameters. ScaleVLA tests whether a frontier-scale VLM helps: it attaches the frozen 552B mixture-of-experts DeepSeek-V4.1-Flash to an action head trained from ...
publications
Development of New Baseline Models for U.S. Medium Office Buildings Based on Commercial Buildings Energy Consumption Survey Data
Published in Science and Technology for the Built Environment, Volume 26, Issue 9, 2020
The construction of new models for energy usage prediction for medium office buildings in various environments.
Enabling Close Proximity Human Robot Collaboration via Distributed, Self-Calibrating Robotic Skin
Published in University of Colorado Boulder Undergraduate Thesis in Computer Science, 2021
My undergraduate thesis covering my prior 2 years of robotics research.
Contact Anticipation for Physical Human-Robot Interaction with Robotic Manipulators using Onboard Proximity Sensors
Published in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems [IROS], 2021
A system for enabling robotic manipulators with onboard proximity sensors to anticipate and react on contact.
Self-Contained Kinematic Calibration of a Novel Whole-Body Artificial Skin for Human-Robot Collaboration
Published in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems [IROS], 2021
A system for calibrating and implementing a arbitrally placed robotic skin for physical human-robot interaction.
Volumetric Data Fusion of External Depth and Onboard Proximity Data For Occluded Space Reduction
Published in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems [IROS] 4th Workshop on Proximity Perception, 2021
A probabilistic fusion of external depth and onboard proximity data into a volumetric 3-D map of a robot’s environment, reducing the space a manipulator must treat as unknown.
Evaluating Performance of Different Generative Adversarial Networks for Large-Scale Building Power Demand Prediction
Published in Energy and Buildings, 2022
A comprehensive evaluation of different types of GANs at scale for building power demand prediction.
Individualized Empirical Baselines for Evaluating the Energy Performance of Existing Buildings
Published in Science and Technology for the Built Environment, 2022
The evaluation of empirical baselines for evaluating the energy performance of multiple buildings.
SHIRO: Soft Hierarchical Reinforcement Learning
Published in Arxiv, 2022
A method for soft hierarchical reinforcement learning to accelerate learning for challenging locomotion tasks
Touch-GS: Visual-Tactile Supervised 3D Gaussian Splatting
Published in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024
Touch-GS combines the power of vision and touch to generate high-quality few-shot and challenging scenes, such as few-view object centric scenes, mirrors, and transparent objects.
Next Best Sense: Guiding Vision and Touch with FisherRF for 3D Gaussian Splatting
Published in IEEE International Conference on Robotics and Automation (ICRA), 2025
Next Best Sense draws upon state of the art vision models to train few-shot Gaussian Splatting scenes, and turns to impressive next best view selection methods to guide robotic manipulator next best view and touch sel...
DexFruit: Dexterous Manipulation and Gaussian Splatting Inspection of Fruit
Published in IEEE Robotics and Automation Letters (RA-L), 2025
DexFruit is a robotic manipulation framework that enables gentle, autonomous handling of fragile fruit using optical tactile sensing and introduces FruitSplat, a novel technique to represent and quantify visual damage...
SPACeR: Self-Play Anchoring with Centralized Reference Models
Published in International Conference on Learning Representations (ICLR), 2026
SPACeR uses a pretrained tokenized autoregressive motion model as a centralized reference policy to guide decentralized self-play RL, matching human driving distributions with 10x faster inference and 50x fewer parame...
TensorTouch: Calibration of Tactile Sensors for High Resolution Stress Tensor and Deformation for Dexterous Manipulation
Published in IEEE Transactions on Robotics (T-RO), 2026
TensorTouch is a comprehensive framework for stress tensor estimation from 3D optical tactile sensors, combining finite element analysis with deep learning to extract detailed contact information including stress tens...
Learning to Drive is a Free Gift: Large-Scale Label-Free Autonomy Pretraining from Unposed In-The-Wild Videos
Published in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026
LFG learns a unified pseudo-4D representation of geometry, semantics, motion, and short-term future evolution directly from unposed, unlabeled single-view driving videos, reaching state-of-the-art planning with a sing...
MR. CRABS: Mobile RGB-D Camera-LIDAR Robot for Autonomous Bimanual Manipulation & Sensing
Published in ICRA Workshop on Act to Sense to Act Better, 2026
A fully integrated perception system for near-ground bimanual mobile manipulation: a holonomic bimanual robot with dual RealSense depth cameras, a small LIDAR, and a pan-and-tilt bottom camera that improves mapping an...
UniQueR: Unified Query-based Feedforward 3D Reconstruction
Published in European Conference on Computer Vision (ECCV), 2026
UniQueR formulates feedforward reconstruction as sparse 3D query inference, learning compact 3D anchor points that infer scene structure including occluded geometry in a single forward pass.
J-PARSE: Jacobian-based Projection Algorithm for Resolving Singularities Effectively in Inverse Kinematic Control of Serial Manipulators
Published in IEEE Transactions on Robotics (T-RO), 2026
J-PARSE is a method for smooth first-order inverse kinematic control of serial manipulators near kinematic singularities, expanding the available workspace for applications in servoing, teleoperation, and learning.
VisTacAlign: Co-Training Dexterous Policies on Tactile Human and Robot Demonstrations
Published in In submission, ICRA 2027, 2026
VisTacAlign co-trains 3D-visual-tactile dexterous policies on human and robot demonstrations by closing the human-robot gap in every modality: glove-tracked hand motion is retargeted to the robot hand, the human hand ...
Real2Sem2Real: 3D Semantic Augmentations for Robot Learning from a Single Human RGB Demonstration
Published in In submission, 2026
Real2Sem2Real turns one monocular RGB video of a human demonstration into thousands of robot demonstrations across the object instances of a category: it tracks each object with DINOv3-distilled Gaussian splats, align...
talks
Slingshot Interview Series: Introduction to Robotics
Published:
For a startup, I was invited as a workshop speaker to give a talk on an Introduction to Robotics. This was an interview with a Slingshot ambassador.
PhD Forum: 4th Workshop on Proximity Perception in Robotics
Published:
At the IROS 2021 4th Workshop on Proximity Perception in Robotics, Caleb Escobedo (my colleague in the HIRO group) and I presented a talk on Contact Anticipation for Physical Human-Robot Interaction with Robotic Manip...
HackCU Workshop Invited Speaker: A Gentler Introduction to Robotics
Published:
At HackCU 2022, I was invited as a workshop speaker to give a talk on an Introduction to Robotics.
HackCU Workshop Invited Speaker: Introduction to AI
Published:
At HackCU 2024, I was invited as a workshop speaker to give a talk on an Introduction to AI.
teaching
Teaching experience 1
Undergraduate course, University 1, Department, 2014
This is a description of a teaching experience. You can use markdown like any other post.
Teaching experience 2
Workshop, University 1, Department, 2015
This is a description of a teaching experience. You can use markdown like any other post.
