EgoHMP: Achieving Precise Human Motion Prediction in 3D Scenes via Egocentric Cues

Xin Zhao, Pengzhan Zhou, Ziyi Li, Riheng Jia, Zhida Qin, Yu Liu

November 2026

TL;DR EgoHMP predicts where a person will go in a 3D scene and how their body will move, using their head-camera video and head motion in place of eye gaze. On GIMO it cuts trajectory error from 575 to 481 mm and pose error from 141 to 115 mm compared with the best gaze-based method.

What they do

Predicting how a person will move through a room depends on what they intend to interact with. Recent methods infer that intent from eye gaze, which drifts, is ambiguous and needs an eye tracker. EgoHMP instead uses the person’s egocentric video and head motion, together with a 3D point cloud of the scene. From 6 observed frames it predicts the next 10 frames of full-body SMPL-X motion.

How it works

Vision-motion alignment. DINOv2 features of the egocentric frames and an embedding of the head motion are pre-trained to align with a KL term plus a contrastive loss.
Intent predictor. Learnable per-modality scaling tokens damp the gradients of unreliable cues (for example, an erratic view). PointNet++ scene features then cross-attend to the fused tokens to produce a per-point interaction probability map, supervised with pseudo-labels from the true contact point and the head position.
Trajectory first. A transformer predicts the global trajectory toward the most likely interaction point. That trajectory prompts a diffusion model that only has to generate body poses.
Contact-aware denoising. The denoiser predicts per-vertex body contact and uses it to refine a coarse pose; a GCN decoder outputs the final motion.

Results

GIMO: trajectory error 481 mm vs 575 mm for GAP3DS, pose error 114.9 vs 141.2 mm, intent error (TDE@1) 161 vs 211 mm, scene penetration 0.81% vs 0.95%. GTA-IM shows similar gains.
Feeding gaze in place of the egocentric view raises the trajectory error to 589 mm.
Ego-Exo4D: pose error 128.6 mm, against 144.7 mm for EgoAllo given ground-truth trajectories. Without the scene, EgoHMP (213.5 mm) does not beat EgoAllo (207.4 mm).

Limitations

Misleading visual and head cues can still mislead the intent estimate. The prediction length is fixed, so actions get cut off or end in stationary poses when the real sequence length differs. Evaluation is on recorded datasets only, with no robot in the loop.

Figures

From Figure 1: EgoHMP takes egocentric video clips, the head motion sequence and a 3D scene point cloud, and predicts... — From Figure 1: EgoHMP takes egocentric video clips, the head motion sequence and a 3D scene point cloud, and predicts the future full-body pose sequence in the scene.

From Figure 2: Pipeline. A pre-trained vision-motion alignment encoder extracts aligned features, an egocentric inten... — From Figure 2: Pipeline. A pre-trained vision-motion alignment encoder extracts aligned features, an egocentric intent predictor queries the scene point cloud with them to infer an interaction probability map, and a contact-aware motion decoder generates the motion sequence.

From Figure 4: Predictions of GAP3DS, EgoHMP and the ground truth in two scenes, with each method's interaction proba... — From Figure 4: Predictions of GAP3DS, EgoHMP and the ground truth in two scenes, with each method's interaction probability map. Blue, green and tan meshes are the observed, predicted and ground-truth motion.

Abstract

The complex interactive dependencies between human behavior and 3D scenes make accurately predicting human intent a major challenge for human motion prediction (HMP). While recent studies use human gaze to infer intent, gaze signals suffer from ambiguity, drift, and a reliance on eye-tracking hardware. Research in cognitive science indicates that egocentric views inherently encode potential interactions, while head motion serves as a crucial precursor in motion planning. Motivated by this, we propose EgoHMP, a novel framework that uses egocentric views and head motions as robust carriers of interaction intent. By translating this intent into an envisioning of future interactions, EgoHMP achieves precise HMP in 3D scenes. EgoHMP first employs a vision-motion alignment encoder to align the visual and motion feature spaces. Based on these aligned features, an egocentric intent predictor with a modality-aware modulation mechanism adaptively suppresses invalid interaction cues to predict an accurate interaction probability map. Furthermore, to enhance the stability and realism of the generated motions, we propose a contact-aware motion decoder. Specifically, inspired by the human cognitive process of planning trajectories prior to action execution, we first predict a global trajectory to serve as a self-prompt, effectively mitigating the optimization conflict in diffusion models. Subsequently, we introduce a multi-stage denoiser, which relieves motion artifacts by incorporating human contact modeling through a multi-stage optimization strategy. Finally, a GCN-based motion decoder is employed to synthesize physically plausible and semantically consistent motions. Extensive experiments demonstrate that EgoHMP achieves state-of-the-art performance.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong