MoNeRF: Deformable Neural Rendering for Talking Heads via Latent Motion Navigation.
Saved in:
| Title: | MoNeRF: Deformable Neural Rendering for Talking Heads via Latent Motion Navigation. |
|---|---|
| Authors: | Li, X.1 (AUTHOR) xxue@hnu.edu.cn, Ding, Y.1 (AUTHOR) ding@hnu.edu.cn, Li, R.1 (AUTHOR) liruihui@hnu.edu.cn, Tang, Z.1 (AUTHOR) ztang@hnu.edu.cn, Li, K.1 (AUTHOR) lkl@hnu.edu.cn |
| Source: | Computer Graphics Forum. Feb2025, Vol. 44 Issue 1, p1-13. 13p. |
| Subjects: | Video processing, Orthogonal codes, Human body, Source code, Radiance |
| Abstract: | Novel view synthesis for talking heads presents significant challenges due to the complex and diverse motion transformations involved. Conventional methods often resort to reliance on structure priors, like facial templates, to warp observed images into a canonical space conducive to rendering. However, the incorporation of such priors introduces a trade‐off‐while aiding in synthesis, they concurrently amplify model complexity, limiting generalizability to other deformable scenes. Departing from this paradigm, we introduce a pioneering solution: the motion‐conditioned neural radiance field, MoNeRF, designed to model talking heads through latent motion navigation. At the core of MoNeRF lies a novel approach utilizing a compact set of latent codes to represent orthogonal motion directions. This innovative strategy empowers MoNeRF to efficiently capture and depict intricate scene motion by linearly combining these latent codes. In an extended capability, MoNeRF facilitates motion control through latent code adjustments, supports view transfer based on reference videos, and seamlessly extends its applicability to model human bodies without necessitating structural modifications. Rigorous quantitative and qualitative experiments unequivocally demonstrate MoNeRF's superior performance compared to state‐of‐the‐art methods in talking head synthesis. We will release the source code upon publication. [ABSTRACT FROM AUTHOR] |
| Copyright of Computer Graphics Forum is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Text: Availability: 1 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 183820845 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: MoNeRF: Deformable Neural Rendering for Talking Heads via Latent Motion Navigation. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Li%2C+X%2E%22">Li, X.</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> xxue@hnu.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Ding%2C+Y%2E%22">Ding, Y.</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> ding@hnu.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Li%2C+R%2E%22">Li, R.</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> liruihui@hnu.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Tang%2C+Z%2E%22">Tang, Z.</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> ztang@hnu.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Li%2C+K%2E%22">Li, K.</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> lkl@hnu.edu.cn</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Computer+Graphics+Forum%22">Computer Graphics Forum</searchLink>. Feb2025, Vol. 44 Issue 1, p1-13. 13p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Video+processing%22">Video processing</searchLink><br /><searchLink fieldCode="DE" term="%22Orthogonal+codes%22">Orthogonal codes</searchLink><br /><searchLink fieldCode="DE" term="%22Human+body%22">Human body</searchLink><br /><searchLink fieldCode="DE" term="%22Source+code%22">Source code</searchLink><br /><searchLink fieldCode="DE" term="%22Radiance%22">Radiance</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Novel view synthesis for talking heads presents significant challenges due to the complex and diverse motion transformations involved. Conventional methods often resort to reliance on structure priors, like facial templates, to warp observed images into a canonical space conducive to rendering. However, the incorporation of such priors introduces a trade‐off‐while aiding in synthesis, they concurrently amplify model complexity, limiting generalizability to other deformable scenes. Departing from this paradigm, we introduce a pioneering solution: the motion‐conditioned neural radiance field, MoNeRF, designed to model talking heads through latent motion navigation. At the core of MoNeRF lies a novel approach utilizing a compact set of latent codes to represent orthogonal motion directions. This innovative strategy empowers MoNeRF to efficiently capture and depict intricate scene motion by linearly combining these latent codes. In an extended capability, MoNeRF facilitates motion control through latent code adjustments, supports view transfer based on reference videos, and seamlessly extends its applicability to model human bodies without necessitating structural modifications. Rigorous quantitative and qualitative experiments unequivocally demonstrate MoNeRF's superior performance compared to state‐of‐the‐art methods in talking head synthesis. We will release the source code upon publication. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Computer Graphics Forum is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=183820845 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1111/cgf.15274 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 13 StartPage: 1 Subjects: – SubjectFull: Video processing Type: general – SubjectFull: Orthogonal codes Type: general – SubjectFull: Human body Type: general – SubjectFull: Source code Type: general – SubjectFull: Radiance Type: general Titles: – TitleFull: MoNeRF: Deformable Neural Rendering for Talking Heads via Latent Motion Navigation. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Li, X. – PersonEntity: Name: NameFull: Ding, Y. – PersonEntity: Name: NameFull: Li, R. – PersonEntity: Name: NameFull: Tang, Z. – PersonEntity: Name: NameFull: Li, K. IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 02 Text: Feb2025 Type: published Y: 2025 Identifiers: – Type: issn-print Value: 01677055 Numbering: – Type: volume Value: 44 – Type: issue Value: 1 Titles: – TitleFull: Computer Graphics Forum Type: main |
| ResultId | 1 |
