Novel View Synthesis on DSLR Images
The novel view synthesis task is to render images from novel viewpoints given a dense RGB capture of the scene. The images are captured by a fisheye DSLR camera, and camera poses from COLMAP are provided for every training and test image.
We also provide the undistorted evaluation track: rendering undistorted perspective (pinhole) images of the given poses. The training images and the GT are generated from the raw fisheye images using the ScanNet++ Toolbox.
Evaluation and Metrics
We evaluate the similarity beween the ground truth and generated RGB images. Our evaluation metrics are peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and learned perceptual image patch similarity (LPIPS). For each pair of generated and ground-truth images, we compute these three metrics, and the numbers reported in the table are the average over all the images across all the scenes.
Evaluation is carried out on GT images with resolution 1752 x 1168. Submitted images will be automatically resized if their resolutions differ from this.
Evaluation excludes the pixels which are anonymized. Anonymized pixels are specified in resized_anon_masks and original_anon_masks.
Results
| Methods | PSNR | SSIM | LPIPS |
|---|---|---|---|
| Zip-NeRF | 25.007 | 0.879 | 0.326 |
| Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, Peter Hedman. Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields. ICCV 2023 | |||
| RPBG | 24.355 | 0.873 | 0.280 |
| Zizhuang Wei, Qingtian Zhu, et al. RPBG: Towards Robust Neural Point-based Graphics in the Wild. ECCV 2024 | |||
| FeatSplat | 24.247 | 0.869 | 0.314 |
| Tomas Berriel Martins, Javier Civera. Feature Splatting for Better Novel View Synthesis with Low Overlap. BMVC 2024 | |||
| Nerfacto | 24.049 | 0.861 | 0.342 |
| Matthew Tancik, Ethan Weber, Evonne Ng, Ruilong Li, Brent Yi, Justin Kerr, Terrance Wang, Alexander Kristoffersen, Jake Austin, Kamyar Salahi, Abhik Ahuja, David McAllister, Angjoo Kanazawa. Nerfstudio: A Modular Framework for Neural Radiance Field Development. SIGGRAPH 2023 | |||
| TensoRF | 23.978 | 0.849 | 0.407 |
| Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, Hao Su. TensoRF: Tensorial Radiance Fields. ECCV 2022 | |||
| Gaussian Splatting | 23.891 | 0.871 | 0.319 |
| Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George Drettakis. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. SIGGRAPH 2023 | |||
| Instant-NGP | 23.812 | 0.859 | 0.375 |
| Thomas Müller, Alex Evans, Christoph Schied, Alexander Keller. Instant Neural Graphics Primitives with a Multiresolution Hash Encoding. SIGGRAPH 2022 | |||
| Plenoxels | 22.549 | 0.836 | 0.407 |
| Alex Yu, Sara Fridovich-Keil, Matthew Tancik, Qinhong Chen, Benjamin Recht, Angjoo Kanazawa. Plenoxels: Radiance Fields without Neural Networks. CVPR 2022 | |||
Please refer to the submission instructions before making a submission
Submit results