AMASS: Archive of Motion Capture as Surface Shapes
TL;DR MoSh++ fits the SMPL body model, including hands and soft-tissue motion, to raw mocap markers and uses it to convert 15 mocap datasets into one archive, AMASS, with 42 hours, 346 subjects and 11,451 motions. On 4D scans recorded alongside mocap, it recovers body shape to 7.4 mm, versus 12.1 mm for the original MoSh.
What they do
Public mocap datasets are small, and each uses its own markerset and skeleton, so they are hard to combine. Earlier merges retargeted everything to one fixed skeleton, which changes the data and throws away body shape. The authors extend MoSh into MoSh++, which fits a body model’s surface directly to the markers. They run it on 15 datasets with 37 to 91 markers each, giving one SMPL-format archive, AMASS, where every frame has shape, pose (including hands) and soft-tissue parameters.
How it works
- Body model. SMPL-H (52 joints, 28 of them in the hands) combined with DMPL soft-tissue blend shapes. It replaces the BlendSCAPE model used by MoSh, which standard graphics software cannot load.
- Stage I. On 12 random frames per subject, optimize one body shape (16 components), the poses and the latent marker positions on the body. Hand pose is regularized in a 24-D MANO subspace. The data term is rescaled by the marker count, and annealing helps avoid local optima.
- Stage II. Per frame, solve for pose and 8 soft-tissue coefficients with temporal smoothness. The pose prior grows up to 3.5x as more markers go missing.
- Tuning. Weights come from line search on SSM, a new dataset of 3 subjects and 30 motions captured by 67-marker mocap synchronized with a 60 fps 4D scanner.
Results
- Shape error on held-out SSM scans with a standard 46-marker set: 7.4 mm for MoSh++ vs 12.1 mm for MoSh, using 16 shape components instead of MoSh’s 100.
- Per-frame error is 8.1 mm vs 10.5 mm without soft tissue, and 7.3 mm vs 10.24 mm with it.
- AMASS holds 42 hours of motion from 346 subjects in 11,451 motions.
Limitations
There is no ground truth for hands, so hand articulation is shown only qualitatively. MoSh++ is far from real time: on a laptop CPU, Stage I takes about 25 min per sequence and Stage II about 2 s per frame with soft tissue. Swapped or mislabeled markers had to be found by manual inspection, and faces are not captured.
Videos
Figures
Abstract
Large datasets are the cornerstone of recent advances in computer vision using deep learning. In contrast, existing human motion capture (mocap) datasets are small and the motions limited, hampering progress on learning models of human motion. While there are many different datasets available, they each use a different parameterization of the body, making it difficult to integrate them into a single meta dataset. To address this, we introduce AMASS, a large and varied database of human motion that unifies 15 different optical marker-based mocap datasets by representing them within a common framework and parameterization. We achieve this using a new method, MoSh++, that converts mocap data into realistic 3D human meshes represented by a rigged body model; here we use SMPL [doi:10.1145/2816795.2818013], which is widely used and provides a standard skeletal representation as well as a fully rigged surface mesh. The method works for arbitrary marker sets, while recovering soft-tissue dynamics and realistic hand motion. We evaluate MoSh++ and tune its hyperparameters using a new dataset of 4D body scans that are jointly recorded with marker-based mocap. The consistent representation of AMASS makes it readily useful for animation, visualization, and generating training data for deep learning. Our dataset is significantly richer than previous human motion collections, having more than 40 hours of motion data, spanning over 300 subjects, more than 11,000 motions, and will be publicly available to the research community.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.