ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras

Raúl Mur-Artal, Juan D. Tardós

★ Must read October 2016

TL;DR ORB-SLAM2 extends the feature-based monocular ORB-SLAM to stereo and RGB-D cameras, with loop closing, relocalization and map reuse running in real time on a CPU. It is the most accurate of the compared SLAM systems on most KITTI, EuRoC and TUM RGB-D sequences, with relative drift generally under 1% on KITTI.

What they do

ORB-SLAM2 extends the monocular, feature-based ORB-SLAM to stereo and RGB-D cameras, keeping loop closing, relocalization and map reuse, and releases it as open source. Depth from a stereo pair or depth sensor makes scale observable and removes the need for a structure-from-motion initialization: the first frame becomes a keyframe and seeds the map.

How it works

Sensor-agnostic keypoints. ORB features are extracted and the images discarded. A stereo keypoint is (uL, vL, uR); for RGB-D, depth becomes a virtual right coordinate using an 8 cm baseline. Points closer than 40 baselines are “close” and triangulated from one frame, far points only from multiple views, and points without depth stay monocular.
Three threads. Tracking runs motion-only BA per frame, local mapping runs local BA over covisible keyframes, and loop closing uses DBoW2 place recognition and a rigid-body pose-graph optimization, then launches full BA in a separate thread and merges its result through the spanning tree.
Keyframe rule. A new keyframe is inserted when tracked close points drop below 100 and the frame could create at least 70 new close points.
Localization mode. Mapping and loop closing are switched off; tracking combines visual-odometry matches with map-point matches for drift-free localization in a mapped area.

Results

KITTI stereo (11 sequences): relative translation error generally below 1%, lower than Stereo LSD-SLAM on most sequences; all loops are detected except in sequence 09. On highway sequence 01: 1.39% vs 2.36%, and 0.21 deg/100 m rotation error.
EuRoC MAV: 0.018-0.119 m translation RMSE, e.g. 0.020 m vs 0.074 m for Stereo LSD-SLAM on V1_02_medium.
TUM RGB-D: lowest error on 5 of 7 sequences against ElasticFusion, Kintinuous, DVO-SLAM and RGB-D SLAM, e.g. 0.004 m on fr2/xyz vs 0.011 m for ElasticFusion.
Mean tracking time stays below the frame period on an Intel i7-4790 without a GPU: 25.6 ms at 30 Hz (TUM), 41.7 ms at 20 Hz (EuRoC), 49.5 ms at 10 Hz (KITTI).

Limitations

Tracking is lost in parts of EuRoC V2_03_difficult due to severe motion blur, and translation is less accurate on highway sequence 01, where few close points can be tracked. The freiburg2 depth maps were corrected for a 4% scale bias, which the authors say could partly explain the better TUM results. Localization mode assumes no significant change in the environment, and the map is sparse rather than a dense reconstruction.

Videos

Figures

From Figure 1: Stereo input on KITTI 00, the estimated trajectory and sparse map of an urban drive with multiple loop... — From Figure 1: Stereo input on KITTI 00, the estimated trajectory and sparse map of an urban drive with multiple loop closures.

From Figure 2: The three parallel threads (tracking, local mapping, loop closing) around a shared map and place-recog... — From Figure 2: The three parallel threads (tracking, local mapping, loop closing) around a shared map and place-recognition database, with a fourth thread for full BA after a loop closure.

From Figure 3: Tracked points on the KITTI 01 highway. Green points are closer than 40 stereo baselines, blue points ... — From Figure 3: Tracked points on the KITTI 01 highway. Green points are closer than 40 stereo baselines, blue points are farther; far points constrain orientation but give weak translation and scale information.

From Figure 7: Dense point clouds from backprojecting sensor depth maps at the estimated keyframe poses in TUM RGB-D ... — From Figure 7: Dense point clouds from backprojecting sensor depth maps at the estimated keyframe poses in TUM RGB-D fr3_office, fr1_room, fr2_desk and fr3_nst.

Abstract

We present ORB-SLAM2 a complete SLAM system for monocular, stereo and RGB-D cameras, including map reuse, loop closing and relocalization capabilities. The system works in real-time on standard CPUs in a wide variety of environments from small hand-held indoors sequences, to drones flying in industrial environments and cars driving around a city. Our back-end based on bundle adjustment with monocular and stereo observations allows for accurate trajectory estimation with metric scale. Our system includes a lightweight localization mode that leverages visual odometry tracks for unmapped regions and matches to map points that allow for zero-drift localization. The evaluation on 29 popular public sequences shows that our method achieves state-of-the-art accuracy, being in most cases the most accurate SLAM solution. We publish the source code, not only for the benefit of the SLAM community, but with the aim of being an out-of-the-box SLAM solution for researchers in other fields.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong