View in EDS

UNOC: Understanding Occlusion for Embodied Presence in Virtual Reality.

Saved in:
Bibliographic Details
Title: UNOC: Understanding Occlusion for Embodied Presence in Virtual Reality.
Authors: Parger, Mathias1 mathias.parger@icg.tugraz.at, Tang, Chengcheng2 tangchengcheng717@gmail.com, Xu, Yuanlu2 yuanluxu@ieee.org, Twigg, Christopher D.2 cdtwigg@gmail.com, Tao, Lingling2 taolingling06@gmail.com, Li, Yijing2 yijingli@fb.com, Wang, Robert2 rywang@csail.mit.edu, Steinberger, Markus1 steinberger@icg.tugraz.at
Source: IEEE Transactions on Visualization & Computer Graphics. Dec2022, Vol. 28 Issue 12, p4240-4251. 12p.
Subjects: Motion capture (Human mechanics), Pose estimation (Computer vision), Virtual reality, Machine learning, Deep learning
Abstract: Tracking body and hand motions in 3D space is essential for social and self-presence in augmented and virtual environments. Unlike the popular 3D pose estimation setting, the problem is often formulated as egocentric tracking based on embodied perception (e.g., egocentric cameras, handheld sensors). In this article, we propose a new data-driven framework for egocentric body tracking, targeting challenges of omnipresent occlusions in optimization-based methods (e.g., inverse kinematics solvers). We first collect a large-scale motion capture dataset with both body and finger motions using optical markers and inertial sensors. This dataset focuses on social scenarios and captures ground truth poses under self-occlusions and body-hand interactions. We then simulate the occlusion patterns in head-mounted camera views on the captured ground truth using a ray casting algorithm and learn a deep neural network to infer the occluded body parts. Our experiments show that our method is able to generate high-fidelity embodied poses by applying the proposed method to the task of real-time egocentric body tracking, finger motion synthesis, and 3-point inverse kinematics. [ABSTRACT FROM AUTHOR]
Copyright of IEEE Transactions on Visualization & Computer Graphics is the property of IEEE and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Description
Abstract:Tracking body and hand motions in 3D space is essential for social and self-presence in augmented and virtual environments. Unlike the popular 3D pose estimation setting, the problem is often formulated as egocentric tracking based on embodied perception (e.g., egocentric cameras, handheld sensors). In this article, we propose a new data-driven framework for egocentric body tracking, targeting challenges of omnipresent occlusions in optimization-based methods (e.g., inverse kinematics solvers). We first collect a large-scale motion capture dataset with both body and finger motions using optical markers and inertial sensors. This dataset focuses on social scenarios and captures ground truth poses under self-occlusions and body-hand interactions. We then simulate the occlusion patterns in head-mounted camera views on the captured ground truth using a ray casting algorithm and learn a deep neural network to infer the occluded body parts. Our experiments show that our method is able to generate high-fidelity embodied poses by applying the proposed method to the task of real-time egocentric body tracking, finger motion synthesis, and 3-point inverse kinematics. [ABSTRACT FROM AUTHOR]
ISSN:10772626
DOI:10.1109/TVCG.2021.3085407