PIDSNeRF: pose interpolation depth supervision neural radiance fields for view synthesis from challenging input.
Saved in:
| Title: | PIDSNeRF: pose interpolation depth supervision neural radiance fields for view synthesis from challenging input. |
|---|---|
| Authors: | Wang, Jianxin1 (AUTHOR) jxwang@stu.just.edu.cn, Shao, Haijian1,2 (AUTHOR) jsj_shj@just.edu.cn, Deng, Xing1 (AUTHOR) xdeng@just.edu.cn, Jiang, Yingtao1 (AUTHOR) yingtao.jiang@unlv.edu |
| Source: | Multimedia Tools & Applications. Jun2025, Vol. 84 Issue 20, p22539-22559. 21p. |
| Subjects: | Probability density function, Gaussian function, Image processing, Artificial intelligence, Task performance |
| Abstract: | Recently, Neural Radiance Fields(NeRF) have shown remarkable performance in the task of novel view synthesis through multi-view. The present study introduces an advanced optimization framework, termed Pose Interpolation Depth Supervision Neural Radiance Fields (PIDSNeRF), designed to address the challenges encountered by NeRF in novel view synthesis. These challenges manifest as artifacts, texture details loss, and geometric inconsistencies, particularly in scenes characterized by different lighting conditions, Textureless regions, and sparse images as inputs. The principle of PIDSNeRF involves interpolating virtual camera positions in the 3D space domain based on known cameras, Subsequently, the sparse point clouds formed during camera pose estimation is re-projected onto these virtual poses, employing an inverse angle weighting strategy, thereby generating depth-supervised rays. Furthermore, we propose a depth diffusion method that spreads depth-supervised rays along the pixel plane, forming Gaussian functions with different variances based on the diffusion distances. Finally, depth supervision during volume rendering is achieved by optimizing the Kullback-Leibler(KL) divergence between the weight distribution of sampled points along each ray and the probability density function of the corresponding Gaussian function along that ray. The above process enables a more comprehensive optimization of multi-resolution grid features that align with the sampled points along the rays. The experiments on the DTU, LLFF, and DL3DV-10K datasets demonstrate, PIDSNeRF can synthesize complete novel views within minutes, and the various metrics of the synthesized images achieve the best performance. [ABSTRACT FROM AUTHOR] |
| Copyright of Multimedia Tools & Applications is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Text: Availability: 1 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 186337329 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: PIDSNeRF: pose interpolation depth supervision neural radiance fields for view synthesis from challenging input. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Wang%2C+Jianxin%22">Wang, Jianxin</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> jxwang@stu.just.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Shao%2C+Haijian%22">Shao, Haijian</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<i> jsj_shj@just.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Deng%2C+Xing%22">Deng, Xing</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> xdeng@just.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Jiang%2C+Yingtao%22">Jiang, Yingtao</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> yingtao.jiang@unlv.edu</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Multimedia+Tools+%26+Applications%22">Multimedia Tools & Applications</searchLink>. Jun2025, Vol. 84 Issue 20, p22539-22559. 21p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Probability+density+function%22">Probability density function</searchLink><br /><searchLink fieldCode="DE" term="%22Gaussian+function%22">Gaussian function</searchLink><br /><searchLink fieldCode="DE" term="%22Image+processing%22">Image processing</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+intelligence%22">Artificial intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Task+performance%22">Task performance</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Recently, Neural Radiance Fields(NeRF) have shown remarkable performance in the task of novel view synthesis through multi-view. The present study introduces an advanced optimization framework, termed Pose Interpolation Depth Supervision Neural Radiance Fields (PIDSNeRF), designed to address the challenges encountered by NeRF in novel view synthesis. These challenges manifest as artifacts, texture details loss, and geometric inconsistencies, particularly in scenes characterized by different lighting conditions, Textureless regions, and sparse images as inputs. The principle of PIDSNeRF involves interpolating virtual camera positions in the 3D space domain based on known cameras, Subsequently, the sparse point clouds formed during camera pose estimation is re-projected onto these virtual poses, employing an inverse angle weighting strategy, thereby generating depth-supervised rays. Furthermore, we propose a depth diffusion method that spreads depth-supervised rays along the pixel plane, forming Gaussian functions with different variances based on the diffusion distances. Finally, depth supervision during volume rendering is achieved by optimizing the Kullback-Leibler(KL) divergence between the weight distribution of sampled points along each ray and the probability density function of the corresponding Gaussian function along that ray. The above process enables a more comprehensive optimization of multi-resolution grid features that align with the sampled points along the rays. The experiments on the DTU, LLFF, and DL3DV-10K datasets demonstrate, PIDSNeRF can synthesize complete novel views within minutes, and the various metrics of the synthesized images achieve the best performance. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Multimedia Tools & Applications is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=186337329 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1007/s11042-024-19978-z Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 21 StartPage: 22539 Subjects: – SubjectFull: Probability density function Type: general – SubjectFull: Gaussian function Type: general – SubjectFull: Image processing Type: general – SubjectFull: Artificial intelligence Type: general – SubjectFull: Task performance Type: general Titles: – TitleFull: PIDSNeRF: pose interpolation depth supervision neural radiance fields for view synthesis from challenging input. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Wang, Jianxin – PersonEntity: Name: NameFull: Shao, Haijian – PersonEntity: Name: NameFull: Deng, Xing – PersonEntity: Name: NameFull: Jiang, Yingtao IsPartOfRelationships: – BibEntity: Dates: – D: 11 M: 06 Text: Jun2025 Type: published Y: 2025 Identifiers: – Type: issn-print Value: 13807501 Numbering: – Type: volume Value: 84 – Type: issue Value: 20 Titles: – TitleFull: Multimedia Tools & Applications Type: main |
| ResultId | 1 |
