Reliability-Adaptive Multimodal Relative Pose Tracking for Underwater Leader-Follower Robots

Mingyang Yang, Zeyu Sha, Xiaorui Wang, Jun Wang, David Chen Wang, Feitian Zhang

November 2026

TL;DR A follower underwater robot tracks the 6-DoF pose of a moving leader by fusing a camera, ultrasonic ranging and a pressure-sensor "lateral line" that feels the leader's wake, weighting each by its current reliability. Over 648 pool trials it averages 3.8 cm position and 2.4° orientation error, and it runs in 17.62 ms on a Jetson AGX Orin for closed-loop formation keeping.

What they do

For two underwater vehicles to fly in formation, the follower must keep estimating where the leader is, without GPS and with cameras that fail in dark or murky water. The follower here combines three cheap sensors that work at different ranges and conditions: a camera, ultrasonic range sensors, and a pressure array that detects the leader’s propeller wake. A network fuses them and leans on whichever is reliable at the moment.

How it works

One encoder per modality. YOLO segments the leader, and an information-bottleneck encoder compresses the result; an acoustic encoder adds the sensor geometry as a physics prior; a CNN-BiLSTM wake encoder reads pressure sequences, regularized with a KL term.
Reliability-weighted fusion. A small weight generator scores each modality’s confidence, and Mamba state-space blocks fuse the features over time in linear time.
Closed loop. The 5 Hz pose estimate feeds a sliding-mode controller on a Jetson AGX Orin aboard a 1 m OpenAUV.

Results

Dataset: 648 pool trials (2 leader vehicles, 3 lighting levels, 9 trajectories) with motion-capture ground truth. Average errors are 3.8 cm in position and 2.4° in orientation; yaw is the worst axis at 5.7°.
In extremely dim light, translation error rises 15.7% and orientation error 26.5%. Removing the wake branch costs the most accuracy (yaw error 5.6° to 6.6°).
Errors are 20-32% lower than with plain feature concatenation. Accuracy is similar to Transformer fusion, but the method runs 2.69x faster on the Orin (17.62 vs 47.48 ms).
Holding 2.10 m behind a moving leader, the follower’s position RMSE is 12.2 cm, 9.9 cm and 2.6 cm along the three axes.

Limitations

Tests ran only in a 4 m x 2 m pool; currents, turbidity and larger separations need open-water trials. While the training data was recorded, the follower hovered in place. Closed-loop results are reported for one representative 30 s segment.

Figures

From Figure 1: The follower AUV senses the leader through a monocular camera, ultrasonic ranging sensors and a pressu... — From Figure 1: The follower AUV senses the leader through a monocular camera, ultrasonic ranging sensors and a pressure sensor array that picks up the leader's wake, and fuses the three to estimate the leader's relative pose for formation control.

From Figure 3: The follower's sensing module, combining an RGB camera (optical), three acoustic ranging sensors and e... — From Figure 3: The follower's sensing module, combining an RGB camera (optical), three acoustic ranging sensors and eight pressure sensors (hydrodynamic).

From Figure 5: The perception network. A YOLO front end detects and segments the leader to build a five-channel visua... — From Figure 5: The perception network. A YOLO front end detects and segments the leader to build a five-channel visual input; separate encoders handle vision (IB-VE), acoustic ranges (PA-GATE) and wake pressures (KLD-WE). An adaptive weight generator and Mamba blocks fuse the three, and a head regresses the leader's 6-DoF pose relative to the follower.

From Figure 10: Closed-loop tracking over a representative 30 s segment with the leader moving. Left: relative positi... — From Figure 10: Closed-loop tracking over a representative 30 s segment with the leader moving. Left: relative position and distance against their references; right: relative roll, pitch, yaw and leader speed, all measured by motion capture.

Abstract

Underwater leader-follower formation control requires continuous relative-pose estimation despite degraded sensing conditions and limited onboard computation. This paper presents a reliability-adaptive multimodal perception framework for six-degree-of-freedom relative-pose tracking between two moving underwater robots. The framework integrates complementary visual, acoustic-ranging, and hydrodynamic measurements while accounting for their distinct physical characteristics and time-varying reliability. It adaptively weights modality contributions and models temporal dependencies with linear computational complexity, enabling efficient pose estimation on embedded hardware. Extensive experiments were conducted using a custom-built AUV platform, with 648 pool trials spanning two leader vehicle types, three illumination conditions, and nine trajectory patterns. Evaluation yields an average translational error of 3.8 cm and an average orientation error of 2.4°. On an onboard NVIDIA Jetson AGX Orin, the fusion network achieves an inference latency of 17.62 ms, providing a 2.69-fold speedup over the evaluated Transformer-based fusion baseline. Closed-loop experiments further demonstrate stable formation maintenance with a moving leader under onboard computational constraints. A multimodal dataset containing measurements from all 648 trials will be made publicly available.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong