Robust novel view synthesis from multi-view feature stereo matching priors: Robust novel view synthesis from multi-view...: J. Wang et al.
Saved in:
| Title: | Robust novel view synthesis from multi-view feature stereo matching priors: Robust novel view synthesis from multi-view...: J. Wang et al. |
|---|---|
| Authors: | Wang, Jianxin1 (AUTHOR) jxwang@stu.just.edu.cn, Shao, Haijian1,2 (AUTHOR) jsj_shj@just.edu.cn, Deng, Xing1 (AUTHOR) xdeng@just.edu.cn, Lian, Shuheng1 (AUTHOR) 222210714217@stu.just.edu.cn |
| Source: | Multimedia Systems. Jun2025, Vol. 31 Issue 3, p1-14. 14p. |
| Subjects: | Stereo image processing, Gaussian function, Robust optimization, Artificial intelligence, Radiance |
| Abstract: | NeRF (Neural Radiance Fields) exhibits the capability to synthesize images from unknown views. However, it faces challenges stemming from factors such as occlusion, non-Lambertian surfaces, sparse inputs, and weak textures commonly encountered in multi-view images. These complexities often result in fitting erroneous scene geometries, leading to suboptimal novel view synthesis quality. This paper addresses this challenge by harnessing the potential of feature-based Multi-View Stereo Matching (MVS) priors to assist NeRF in perceiving deeper information within the scene. The method is distinguished by its adaptive construction of Gaussian functions based on MVS-estimated depth values, uncertainties, and depth intervals, offering flexibility and adaptability to arbitrarily scaled scenes. Building upon this, the optimization of NeRF's training process is achieved by comparing the difference between this distribution and the weight distribution of sampled points on the volume rendering ray. Furthermore, we propose an effective Riemann sum approximation strategy to further enhance the performance of depth loss. Quantitative metrics applied to three real-scene datasets, namely LLFF, IBRNet, and DTU, demonstrate that the method presented in this paper significantly improves the quality of novel view synthesis compared to current advanced methods, achieving enhancements ranging from 3.8% to 26.9%. Visualization experiments reveal robust optimization results, particularly in challenging regions where conventional NeRF encounters difficulties. [ABSTRACT FROM AUTHOR] |
| Copyright of Multimedia Systems is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | NeRF (Neural Radiance Fields) exhibits the capability to synthesize images from unknown views. However, it faces challenges stemming from factors such as occlusion, non-Lambertian surfaces, sparse inputs, and weak textures commonly encountered in multi-view images. These complexities often result in fitting erroneous scene geometries, leading to suboptimal novel view synthesis quality. This paper addresses this challenge by harnessing the potential of feature-based Multi-View Stereo Matching (MVS) priors to assist NeRF in perceiving deeper information within the scene. The method is distinguished by its adaptive construction of Gaussian functions based on MVS-estimated depth values, uncertainties, and depth intervals, offering flexibility and adaptability to arbitrarily scaled scenes. Building upon this, the optimization of NeRF's training process is achieved by comparing the difference between this distribution and the weight distribution of sampled points on the volume rendering ray. Furthermore, we propose an effective Riemann sum approximation strategy to further enhance the performance of depth loss. Quantitative metrics applied to three real-scene datasets, namely LLFF, IBRNet, and DTU, demonstrate that the method presented in this paper significantly improves the quality of novel view synthesis compared to current advanced methods, achieving enhancements ranging from 3.8% to 26.9%. Visualization experiments reveal robust optimization results, particularly in challenging regions where conventional NeRF encounters difficulties. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 09424962 |
| DOI: | 10.1007/s00530-025-01757-x |
