Novel View Synthesis on DSLR Images

The novel view synthesis task is to render images from novel viewpoints given a dense RGB capture of the scene. The images are captured by a fisheye DSLR camera, and camera poses from COLMAP are provided for every training and test image.

We also provide the undistorted evaluation track: rendering undistorted perspective (pinhole) images of the given poses. The training images and the GT are generated from the raw fisheye images using the ScanNet++ Toolbox.

Novel view synthesis

Evaluation and Metrics

We evaluate the similarity beween the ground truth and generated RGB images. Our evaluation metrics are peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and learned perceptual image patch similarity (LPIPS). For each pair of generated and ground-truth images, we compute these three metrics, and the numbers reported in the table are the average over all the images across all the scenes.

Evaluation is carried out on GT images with resolution 1752 x 1168. Submitted images will be automatically resized if their resolutions differ from this.

Evaluation excludes the pixels which are anonymized. Anonymized pixels are specified in resized_anon_masks and original_anon_masks.

Results

Methods PSNR SSIM LPIPS
HW GTS - RPBG 24.355 0.873 0.280
Zizhuang Wei, Qingtian Zhu, et al. RPBG: Towards Robust Neural Point-based Graphics in the Wild. Under Review
FeatSplat 24.247 0.869 0.314
Tomas Berriel Martins, Javier Civera. Feature Splatting for Better Novel View Synthesis with Low Overlap. Under review
Nerfacto 24.049 0.861 0.342
Matthew Tancik, Ethan Weber, Evonne Ng, Ruilong Li, Brent Yi, Justin Kerr, Terrance Wang, Alexander Kristoffersen, Jake Austin, Kamyar Salahi, Abhik Ahuja, David McAllister, Angjoo Kanazawa. Nerfstudio: A Modular Framework for Neural Radiance Field Development. SIGGRAPH 2023
TensoRF 23.978 0.849 0.407
Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, Hao Su. TensoRF: Tensorial Radiance Fields. ECCV 2022
Gaussian Splatting 23.891 0.871 0.319
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George Drettakis. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. SIGGRAPH 2023
Instant-NGP 23.812 0.859 0.375
Thomas Müller, Alex Evans, Christoph Schied, Alexander Keller. Instant Neural Graphics Primitives with a Multiresolution Hash Encoding. SIGGRAPH 2022
Plenoxels 22.549 0.836 0.407
Alex Yu, Sara Fridovich-Keil, Matthew Tancik, Qinhong Chen, Benjamin Recht, Angjoo Kanazawa. Plenoxels: Radiance Fields without Neural Networks. CVPR 2022

Please refer to the submission instructions before making a submission

Submit results