PIXIE: Collaborative Regression of Expressive Bodies using Moderation (3DV'21)
Recovering expressive humans from images is essential for understanding human behavior. Methods that estimate 3D bodies, faces, or hands have progressed significantly, yet separately. Face methods recover accurate 3D shape and geometric details, but need a tight crop and struggle with extreme views and low resolution. Whole-body methods are robust to a wide range of poses and resolutions, but provide only a rough 3D face shape without details like wrinkles. To get the best of both worlds, we introduce PIXIE, which produces animatable, whole-body 3D avatars with realistic facial detail, from a single image. For this, PIXIE uses two key observations. First, existing work combines independent estimates from body, face, and hand experts, by trusting them equally. PIXIE introduces a novel moderator that merges the features of the experts, weighted by their confidence. All part experts can contribute to the whole, using SMPL-X’s shared shape space across all body parts. Second, human shape is highly correlated with gender, but existing work ignores this. We label training images as male, female, or non-binary, and train PIXIE to infer “gendered” 3D body shapes with a novel shape loss. In addition to 3D body pose and shape parameters, PIXIE estimates expression, illumination, albedo and 3D facial surface displacements. Quantitative and qualitative evaluation shows that PIXIE estimates more accurate whole-body shape and detailed face shape than the state of the art.
Paper: https://ps.is.mpg.de/uploads_file/att...
Code: https://github.com/YadiraF/PIXIE
Project: https://pixie.is.tue.mpg.de/
Bibtex:
@inproceedings{PIXIE:2021,
title={Collaborative Regression of Expressive Bodies using Moderation},
author={Yao Feng and Vasileios Choutas and Timo Bolkart and Dimitrios Tzionas and Michael J. Black},
booktitle={International Conference on 3D Vision (3DV)},
year={2021}
}
Voice: https://www.fiverr.com/christianhmiles
Watch video PIXIE: Collaborative Regression of Expressive Bodies using Moderation (3DV'21) online without registration, duration hours minute second in high quality. This video was added by user Michael Black 15 December 2021, don't forget to share it with your friends and acquaintances, it has been viewed on our site 3,605 once and liked it 82 people.