View in EDS

UNOC: Understanding Occlusion for Embodied Presence in Virtual Reality.

Saved in:
Bibliographic Details
Title: UNOC: Understanding Occlusion for Embodied Presence in Virtual Reality.
Authors: Parger, Mathias1 mathias.parger@icg.tugraz.at, Tang, Chengcheng2 tangchengcheng717@gmail.com, Xu, Yuanlu2 yuanluxu@ieee.org, Twigg, Christopher D.2 cdtwigg@gmail.com, Tao, Lingling2 taolingling06@gmail.com, Li, Yijing2 yijingli@fb.com, Wang, Robert2 rywang@csail.mit.edu, Steinberger, Markus1 steinberger@icg.tugraz.at
Source: IEEE Transactions on Visualization & Computer Graphics. Dec2022, Vol. 28 Issue 12, p4240-4251. 12p.
Subjects: Motion capture (Human mechanics), Pose estimation (Computer vision), Virtual reality, Machine learning, Deep learning
Abstract: Tracking body and hand motions in 3D space is essential for social and self-presence in augmented and virtual environments. Unlike the popular 3D pose estimation setting, the problem is often formulated as egocentric tracking based on embodied perception (e.g., egocentric cameras, handheld sensors). In this article, we propose a new data-driven framework for egocentric body tracking, targeting challenges of omnipresent occlusions in optimization-based methods (e.g., inverse kinematics solvers). We first collect a large-scale motion capture dataset with both body and finger motions using optical markers and inertial sensors. This dataset focuses on social scenarios and captures ground truth poses under self-occlusions and body-hand interactions. We then simulate the occlusion patterns in head-mounted camera views on the captured ground truth using a ray casting algorithm and learn a deep neural network to infer the occluded body parts. Our experiments show that our method is able to generate high-fidelity embodied poses by applying the proposed method to the task of real-time egocentric body tracking, finger motion synthesis, and 3-point inverse kinematics. [ABSTRACT FROM AUTHOR]
Copyright of IEEE Transactions on Visualization & Computer Graphics is the property of IEEE and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 160687502
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: UNOC: Understanding Occlusion for Embodied Presence in Virtual Reality.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Parger%2C+Mathias%22">Parger, Mathias</searchLink><relatesTo>1</relatesTo><i> mathias.parger@icg.tugraz.at</i><br /><searchLink fieldCode="AR" term="%22Tang%2C+Chengcheng%22">Tang, Chengcheng</searchLink><relatesTo>2</relatesTo><i> tangchengcheng717@gmail.com</i><br /><searchLink fieldCode="AR" term="%22Xu%2C+Yuanlu%22">Xu, Yuanlu</searchLink><relatesTo>2</relatesTo><i> yuanluxu@ieee.org</i><br /><searchLink fieldCode="AR" term="%22Twigg%2C+Christopher+D%2E%22">Twigg, Christopher D.</searchLink><relatesTo>2</relatesTo><i> cdtwigg@gmail.com</i><br /><searchLink fieldCode="AR" term="%22Tao%2C+Lingling%22">Tao, Lingling</searchLink><relatesTo>2</relatesTo><i> taolingling06@gmail.com</i><br /><searchLink fieldCode="AR" term="%22Li%2C+Yijing%22">Li, Yijing</searchLink><relatesTo>2</relatesTo><i> yijingli@fb.com</i><br /><searchLink fieldCode="AR" term="%22Wang%2C+Robert%22">Wang, Robert</searchLink><relatesTo>2</relatesTo><i> rywang@csail.mit.edu</i><br /><searchLink fieldCode="AR" term="%22Steinberger%2C+Markus%22">Steinberger, Markus</searchLink><relatesTo>1</relatesTo><i> steinberger@icg.tugraz.at</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22IEEE+Transactions+on+Visualization+%26+Computer+Graphics%22">IEEE Transactions on Visualization & Computer Graphics</searchLink>. Dec2022, Vol. 28 Issue 12, p4240-4251. 12p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Motion+capture+%28Human+mechanics%29%22">Motion capture (Human mechanics)</searchLink><br /><searchLink fieldCode="DE" term="%22Pose+estimation+%28Computer+vision%29%22">Pose estimation (Computer vision)</searchLink><br /><searchLink fieldCode="DE" term="%22Virtual+reality%22">Virtual reality</searchLink><br /><searchLink fieldCode="DE" term="%22Machine+learning%22">Machine learning</searchLink><br /><searchLink fieldCode="DE" term="%22Deep+learning%22">Deep learning</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Tracking body and hand motions in 3D space is essential for social and self-presence in augmented and virtual environments. Unlike the popular 3D pose estimation setting, the problem is often formulated as egocentric tracking based on embodied perception (e.g., egocentric cameras, handheld sensors). In this article, we propose a new data-driven framework for egocentric body tracking, targeting challenges of omnipresent occlusions in optimization-based methods (e.g., inverse kinematics solvers). We first collect a large-scale motion capture dataset with both body and finger motions using optical markers and inertial sensors. This dataset focuses on social scenarios and captures ground truth poses under self-occlusions and body-hand interactions. We then simulate the occlusion patterns in head-mounted camera views on the captured ground truth using a ray casting algorithm and learn a deep neural network to infer the occluded body parts. Our experiments show that our method is able to generate high-fidelity embodied poses by applying the proposed method to the task of real-time egocentric body tracking, finger motion synthesis, and 3-point inverse kinematics. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of IEEE Transactions on Visualization & Computer Graphics is the property of IEEE and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=160687502
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1109/TVCG.2021.3085407
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 12
        StartPage: 4240
    Subjects:
      – SubjectFull: Motion capture (Human mechanics)
        Type: general
      – SubjectFull: Pose estimation (Computer vision)
        Type: general
      – SubjectFull: Virtual reality
        Type: general
      – SubjectFull: Machine learning
        Type: general
      – SubjectFull: Deep learning
        Type: general
    Titles:
      – TitleFull: UNOC: Understanding Occlusion for Embodied Presence in Virtual Reality.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Parger, Mathias
      – PersonEntity:
          Name:
            NameFull: Tang, Chengcheng
      – PersonEntity:
          Name:
            NameFull: Xu, Yuanlu
      – PersonEntity:
          Name:
            NameFull: Twigg, Christopher D.
      – PersonEntity:
          Name:
            NameFull: Tao, Lingling
      – PersonEntity:
          Name:
            NameFull: Li, Yijing
      – PersonEntity:
          Name:
            NameFull: Wang, Robert
      – PersonEntity:
          Name:
            NameFull: Steinberger, Markus
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 12
              Text: Dec2022
              Type: published
              Y: 2022
          Identifiers:
            – Type: issn-print
              Value: 10772626
          Numbering:
            – Type: volume
              Value: 28
            – Type: issue
              Value: 12
          Titles:
            – TitleFull: IEEE Transactions on Visualization & Computer Graphics
              Type: main
ResultId 1