Safe and Efficient Navigation Considering Texting Pedestrians via Attention-Aware Cost-map Tuning

Shunya Tadano, Yusuke Tamura, Ankit A. Ravankar, Yasuhisa Hirata

November 2026

TL;DR A wheelchair robot classifies each oncoming pedestrian as attentive or texting from 3D skeleton keypoints and widens its cost-map safety margin around distracted ones. In a 12-person, 252-trial corridor study, people rated it significantly smoother (p=0.003) and more predictable (p=0.034) than LiDAR-only or prediction-only navigation, with no change in travel time.

What they do

Social navigation planners usually assume every pedestrian will notice the robot and help avoid it. Someone texting while walking often won’t. This paper detects whether each oncoming pedestrian is distracted, gives distracted people a wider berth, and tests with a user study whether people walking past the robot notice the difference.

How it works

Attention classification. A Random Forest (300 trees) labels each pedestrian attentive or distracted from 33 MediaPipe BlazePose 3D keypoints. It was trained on 243 hand-labeled JRDB images.
Trajectory prediction. A pretrained Human Scene Transformer predicts 4 s ahead from 2 s of positions and keypoints.
Adaptive cost map. The cost radius around predicted points is 0.5 m for distracted pedestrians (Hall’s personal distance) and 0.3 m for attentive ones; TEB plans the local path.
Platform. A WHILL Model CR wheelchair with LiDARs and a RealSense D455, computing on a Jetson AGX Orin.

Results

The classifier scores 95.1% accuracy and 95.9% F1 on 61 held-out images.
12 participants each did 21 trials (252 total), walking normally, texting or switching mid-trial. Methods compared: LiDAR-only (A), prediction with a fixed 0.5 m radius (B), and theirs (C).
Prediction cut jerk RMS from 37.45 (A) to 11.01 (B) and 11.42 m/s³ (C), yet A and B did not differ on any subjective rating. Only C raised smoothness (4.89 vs 4.14 and 3.96 out of 7, p=0.003) and predictability (4.48 vs 4.01 and 3.69, p=0.034).
Travel time was unchanged (16.8-17.0 s). For 10 of 12 participants, C’s maximum lateral deviation was larger than B’s, and 8 of 12 picked C as the most noticeable method while using their phones.

Limitations

The participants were twelve graduate students aged 22-28, and only one pedestrian walked at a time. The forward-facing camera loses people once they pass. The classifier is binary and frame-by-frame, and participants found the avoidance direction unclear when a pedestrian started texting mid-trial. The LiDAR baseline detected pedestrians in only 39% of trials, so its separation distances can’t be compared.

Figures

From Figure 1: The robot keeps a standard safety margin around an attentive pedestrian (right) and a larger one aroun... — From Figure 1: The robot keeps a standard safety margin around an attentive pedestrian (right) and a larger one around a smartphone-distracted pedestrian (left).

From Figure 2: The system has three modules: pedestrian trajectory prediction, attention classification from 33 3D sk... — From Figure 2: The system has three modules: pedestrian trajectory prediction, attention classification from 33 3D skeletal keypoints, and adaptive cost-map generation that sets the margin from the attention state before motion planning.

From Figure 3: Experimental setup in an indoor corridor intersection (main corridor 3.0 m wide), where participants w... — From Figure 3: Experimental setup in an indoor corridor intersection (main corridor 3.0 m wide), where participants walk toward the wheelchair robot.

From Figure 4: Feature importance of the Random Forest attention classifier; right-hand keypoints (wrist, index, pink... — From Figure 4: Feature importance of the Random Forest attention classifier; right-hand keypoints (wrist, index, pinky, thumb) rank highest.

Abstract

Smartphone-distracted pedestrians deviate from social norms unpredictably and cannot be relied upon to yield to approaching robots -- a safety challenge that existing navigation methods are poorly equipped to handle, since they treat all pedestrians uniformly regardless of attention state. We propose an attention-aware navigation framework that classifies each pedestrian as attentive or distracted in real-time using 3D skeletal features and a Random Forest classifier, then dynamically widens cost-map safety margins around distracted individuals to drive proactive avoidance. A within-subjects user study with 12 participants across 252 trials demonstrates significant improvements in perceived smoothness (p = 0.003) and predictability (p = 0.034) over both a LiDAR-only baseline and a trajectory-prediction-only method, without any increase in travel time. Notably, trajectory prediction alone does not improve subjective navigation quality unless coupled with attention-state-aware cost adjustment -- confirming that appropriately responding to who is in front of the robot can matter more than trajectory smoothness alone.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong