PIDSNeRF: pose interpolation depth supervision neural radiance fields for view synthesis from challenging input.
Saved in:
| Title: | PIDSNeRF: pose interpolation depth supervision neural radiance fields for view synthesis from challenging input. |
|---|---|
| Authors: | Wang, Jianxin1 (AUTHOR) jxwang@stu.just.edu.cn, Shao, Haijian1,2 (AUTHOR) jsj_shj@just.edu.cn, Deng, Xing1 (AUTHOR) xdeng@just.edu.cn, Jiang, Yingtao1 (AUTHOR) yingtao.jiang@unlv.edu |
| Source: | Multimedia Tools & Applications. Jun2025, Vol. 84 Issue 20, p22539-22559. 21p. |
| Subjects: | Probability density function, Gaussian function, Image processing, Artificial intelligence, Task performance |
| Abstract: | Recently, Neural Radiance Fields(NeRF) have shown remarkable performance in the task of novel view synthesis through multi-view. The present study introduces an advanced optimization framework, termed Pose Interpolation Depth Supervision Neural Radiance Fields (PIDSNeRF), designed to address the challenges encountered by NeRF in novel view synthesis. These challenges manifest as artifacts, texture details loss, and geometric inconsistencies, particularly in scenes characterized by different lighting conditions, Textureless regions, and sparse images as inputs. The principle of PIDSNeRF involves interpolating virtual camera positions in the 3D space domain based on known cameras, Subsequently, the sparse point clouds formed during camera pose estimation is re-projected onto these virtual poses, employing an inverse angle weighting strategy, thereby generating depth-supervised rays. Furthermore, we propose a depth diffusion method that spreads depth-supervised rays along the pixel plane, forming Gaussian functions with different variances based on the diffusion distances. Finally, depth supervision during volume rendering is achieved by optimizing the Kullback-Leibler(KL) divergence between the weight distribution of sampled points along each ray and the probability density function of the corresponding Gaussian function along that ray. The above process enables a more comprehensive optimization of multi-resolution grid features that align with the sampled points along the rays. The experiments on the DTU, LLFF, and DL3DV-10K datasets demonstrate, PIDSNeRF can synthesize complete novel views within minutes, and the various metrics of the synthesized images achieve the best performance. [ABSTRACT FROM AUTHOR] |
| Copyright of Multimedia Tools & Applications is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Recently, Neural Radiance Fields(NeRF) have shown remarkable performance in the task of novel view synthesis through multi-view. The present study introduces an advanced optimization framework, termed Pose Interpolation Depth Supervision Neural Radiance Fields (PIDSNeRF), designed to address the challenges encountered by NeRF in novel view synthesis. These challenges manifest as artifacts, texture details loss, and geometric inconsistencies, particularly in scenes characterized by different lighting conditions, Textureless regions, and sparse images as inputs. The principle of PIDSNeRF involves interpolating virtual camera positions in the 3D space domain based on known cameras, Subsequently, the sparse point clouds formed during camera pose estimation is re-projected onto these virtual poses, employing an inverse angle weighting strategy, thereby generating depth-supervised rays. Furthermore, we propose a depth diffusion method that spreads depth-supervised rays along the pixel plane, forming Gaussian functions with different variances based on the diffusion distances. Finally, depth supervision during volume rendering is achieved by optimizing the Kullback-Leibler(KL) divergence between the weight distribution of sampled points along each ray and the probability density function of the corresponding Gaussian function along that ray. The above process enables a more comprehensive optimization of multi-resolution grid features that align with the sampled points along the rays. The experiments on the DTU, LLFF, and DL3DV-10K datasets demonstrate, PIDSNeRF can synthesize complete novel views within minutes, and the various metrics of the synthesized images achieve the best performance. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 13807501 |
| DOI: | 10.1007/s11042-024-19978-z |
