MiniPose: Miniature-Based Real Data Collection for 6D Pose Estimation of Large Industrial Pipe Segments

TL;DR Collect real RGB-D training data for 6D pose estimation of large industrial pipes by capturing a 3D-printed half-scale replica and rescaling its depth. Adding 1,000 miniature images per pipe type to 10,000 synthetic ones raises mean ADD-S AUC on real full-scale pipes from 91.9 to 94.5 for PVN3D and from 93.5 to 94.5 for RDPN6D.
What they do
Learned 6D pose estimators have mostly been developed on small objects like those in YCB-Video. Industrial pipe segments of up to 1.5 m x 1.5 m x 0.5 m and about 10 kg are hard to reposition for real training data, and synthetic depth lacks the structured errors of a real depth sensor. MiniPose takes the real data from a scaled-down replica instead and mixes it with synthetic renders of the full-scale pipe.
How it works
- Miniature capture. A 50%-scale replica (about 100-200 g), 3D-printed from the same CAD model and painted like the real pipe, is placed in random poses and captured with an Azure Kinect (ToF) at 0.9-1 m, half the 1.8-2 m used for full-scale pipes.
- Depth rescaling. Scaling the object and the camera distance together leaves the image unchanged, so RGB is used as is and depth is multiplied by 1/s (2 for s = 0.5). SAM 2 masks the pipe so that the scaled background depth is cropped away.
- Training mix. 10,000 Isaac Sim renders per pipe type plus 1,000 miniature images (500 for Straight), under 10% of the data. Two estimators: PVN3D (keypoint voting, given a symmetry-aware keypoint loss) and RDPN6D (dense coordinate regression), tested on 100 real full-scale images for each of six pipe types.
Results
- Mean ADD-S AUC on real pipes rises from 91.9 (synthetic only) to 94.5 for PVN3D, best on every pipe type, with the largest gain on Straight (89.0 to 94.5). RDPN6D rises from 93.5 to 94.5, with gains on all six types.
- Test-time depth denoising with CDM lowers the AUC (89.3 and 92.2), a hand-built ToF noise model adds little (92.1 and 93.9), and Gaussian depth noise does not help (91.3 and 93.5).
- Real RGB alone and real depth alone each help; using both is best among settings without full-scale real data. Training with 100 full-scale real images per type instead gives 93.4 for PVN3D (below MiniPose) but 95.5 for RDPN6D (above).
Limitations
One ToF camera, one miniature scale (0.5) and one capture environment for the full-scale data. Sensor noise does not scale linearly with distance, so rescaled miniature depth only approximates full-scale measurements. Gains are 1-2.6 AUC points, and no robot uses the pose estimates in the paper.
Figures



Abstract
6-DOF pose estimation of large and heavy industrial objects, such as large piping systems in shipyards and structural components in petrochemical plants, remains relatively underexplored in robotic perception. Repeatedly repositioning large objects is physically and spatially demanding, making it difficult to acquire real training data with sufficient pose diversity. Synthetic data offer a scalable alternative, but faithfully reproducing real RGB-D observations, particularly the structured measurement errors of real depth sensors, remains difficult. We propose MiniPose, a practical pipeline for miniature-based real RGB-D data collection that addresses both the data-acquisition burden and the sim-to-real gap. We fabricate a scaled-down replica and capture RGB-D data of it in diverse poses using a real RGB-D camera. RGB images are used directly, while depth maps are converted to a geometrically consistent representation of the full-scale object via a simple inverse scaling transformation. This allows training to incorporate real appearance and sensor measurement characteristics that are difficult to reproduce using synthetic data alone. To examine whether the benefit depends on a particular network, we evaluate two RGB-D pose estimators with different formulations, PVN3D and RDPN6D. Mixing at most 1,000 miniature images per pipe type, less than 10\% of the training data, with 10,000 synthetic images improves the mean ADD-S AUC across pipe types over synthetic-only training for both estimators. A comparison with training on full-scale real data further suggests that miniature data can serve as a proxy where full-scale collection is difficult, reducing the data-collection burden for large industrial objects.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
