Matt Strong

Matt Strong

CS PhD at Stanford University

← Paper Digest

Open3D: A Modern Library for 3D Data Processing

Qian-Yi Zhou, Jaesik Park, Vladlen Koltun

Seminal January 2018

TL;DR Open3D is an open-source C++11 library with Python bindings for point clouds, meshes and RGB-D images, covering I/O, visualization, registration and volumetric reconstruction. Its ICP is up to 25x faster than PCL's, and its scene reconstruction pipeline up to an order of magnitude faster than the original implementation.

What they do

An image can be loaded and shown with a few lines of OpenCV, but 3D data has no equally easy library; the authors describe PCL as bloated and largely dormant. Open3D is written from a clean slate around two principles, usefulness and ease of use: a small set of dependencies, one community-standard solution per problem, and everything exposed in both C++ and Python. It is released under the MIT license.

How it works

  • Data. PointCloud, TriangleMesh and Image (plus RGBDImage), with numpy access to the underlying arrays and depth-image readers for NYU, TUM, SUN3D and Redwood.
  • Nine modules, including odometry, registration (FPFH + RANSAC global registration, point-to-point and point-to-plane ICP, pose-graph multiway registration), volumetric integration and an OpenGL visualizer with animation and key callbacks.
  • Dependencies. Eigen, GLFW, FLANN and pybind11. Boost and Ceres are left out in favor of in-house Gauss-Newton and Levenberg-Marquardt solvers, and dependency sources ship with the library so it compiles on Linux, OS X and Windows.
  • Speed. Key steps were profiled and optimized, and many functions use OpenMP, including a parallel reduction for the Jacobian products in the least-squares solvers.

Results

  • A load, downsample and estimate-normals workflow is about half as long in Python as in Open3D C++, and about five times shorter than with PCL.
  • ICP is up to 25x faster than PCL’s. The reimplemented Choi et al. scene reconstruction pipeline is up to an order of magnitude faster than the original, and OpenMP speeds up the most expensive functions 3-6x on a modern CPU.
  • A complete RGB-D scene reconstruction (fragments, global alignment, integration) is a Python script on top of the library, demonstrated on SceneNN. Open3D also runs in the Tanks and Temples evaluation server.

Limitations

A short system paper. Speed-ups are given as “up to” factors, with no benchmark tables or hardware details, and are compared only with PCL and the original reconstruction code. The algorithm set is limited on purpose: only problems of broad interest are covered, and a new algorithm is added only if it is significantly stronger on a well-known benchmark.

Figures

From Figure 1: A simple task (load a point cloud, downsample it, estimate normals) implemented with the Open3D Python...
From Figure 1: A simple task (load a point cloud, downsample it, estimate normals) implemented with the Open3D Python interface, the Open3D C++ interface and PCL. The Python version is the shortest.
From Figure 3: Pairwise point cloud registration after RANSAC-based global registration (left) and after ICP local re...
From Figure 3: Pairwise point cloud registration after RANSAC-based global registration (left) and after ICP local refinement (right).
From Figure 4: An indoor scene from SceneNN reconstructed and visualized in Open3D.
From Figure 4: An indoor scene from SceneNN reconstructed and visualized in Open3D.
Abstract

Open3D is an open-source library that supports rapid development of software that deals with 3D data. The Open3D frontend exposes a set of carefully selected data structures and algorithms in both C++ and Python. The backend is highly optimized and is set up for parallelization. Open3D was developed from a clean slate with a small and carefully considered set of dependencies. It can be set up on different platforms and compiled from source with minimal effort. The code is clean, consistently styled, and maintained via a clear code review mechanism. Open3D has been used in a number of published research projects and is actively deployed in the cloud. We welcome contributions from the open-source community.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.