The 2D semantic labeling task involves predicting a per-pixel semantic labeling of an image.

Evaluation and metrics

Our evaluation ranks all methods according to the PASCAL VOC intersection-over-union metric (IoU). IoU = TP/(TP+FP+FN), where TP, FP, and FN are the numbers of true positive, false positive, and false negative pixels, respectively.



This table lists the benchmark results for the 2D semantic label scenario.


Method Infoavg ioubathtubbedbookshelfcabinetchaircountercurtaindeskdoorfloorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwallwindow
sort bysort bysort bysorted bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort by
Virtual MVFusion (R)0.745 10.861 10.839 10.881 10.672 20.512 10.422 190.898 10.723 10.714 10.954 20.454 10.509 10.773 10.895 10.756 10.820 10.653 10.935 10.891 10.728 1
Abhijit Kundu, Xiaoqi Yin, Alireza Fathi, David Ross, Brian Brewington, Thomas Funkhouser, Caroline Pantofaru: Virtual Multi-view Fusion for 3D Semantic Segmentation. ECCV 2020
BPNet_2Dcopyleft0.670 20.822 30.795 30.836 20.659 30.481 20.451 150.769 50.656 30.567 40.931 30.395 60.390 60.700 40.534 40.689 110.770 20.574 30.865 110.831 30.675 6
Wenbo Hu, Hengshuang Zhao, Li Jiang, Jiaya Jia and Tien-Tsin Wong: Bidirectional Projection Network for Cross Dimension Scene Understanding. CVPR 2021 (Oral)
DMMF_3d0.605 70.651 100.744 110.782 30.637 50.387 40.536 50.732 100.590 80.540 60.856 230.359 120.306 170.596 160.539 30.627 220.706 40.497 80.785 230.757 210.476 24
MCA-Net0.595 90.533 220.756 90.746 40.590 100.334 100.506 90.670 170.587 90.500 130.905 110.366 110.352 100.601 150.506 90.669 170.648 100.501 70.839 170.769 170.516 23
DVEFormer0.626 50.616 120.764 60.690 50.583 110.322 140.540 30.809 30.593 70.502 120.900 140.374 90.433 30.660 90.528 50.665 190.663 60.491 90.871 100.810 90.705 4
RFBNet0.592 100.616 120.758 80.659 60.581 120.330 110.469 130.655 200.543 150.524 80.924 40.355 140.336 120.572 190.479 110.671 150.648 100.480 110.814 210.814 70.614 12
ScanNet (2d proj)permissive0.330 280.293 270.521 270.657 70.361 270.161 270.250 270.004 280.440 250.183 280.836 250.125 270.060 280.319 280.132 270.417 270.412 270.344 270.541 280.427 280.109 28
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nießner: ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. CVPR'17
SN_RN152pyrx8_RVCcopyleft0.546 190.572 190.663 230.638 80.518 200.298 180.366 260.633 230.510 190.446 210.864 210.296 220.267 210.542 210.346 230.704 80.575 200.431 210.853 150.766 190.630 10
UDSSEG_RVC0.545 200.610 150.661 240.588 90.556 170.268 230.482 110.642 220.572 110.475 180.836 250.312 200.367 80.630 110.189 250.639 210.495 250.452 170.826 190.756 220.541 19
3DMV (2d proj)0.498 240.481 260.612 250.579 100.456 240.343 70.384 230.623 240.525 180.381 250.845 240.254 240.264 230.557 200.182 260.581 260.598 170.429 220.760 250.661 270.446 26
Angela Dai, Matthias Niessner: 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation. ECCV'18
EMSAFormer0.564 180.581 180.736 120.564 110.546 180.219 250.517 70.675 160.486 210.427 230.904 120.352 150.320 150.589 170.528 50.708 70.464 260.413 240.847 160.786 130.611 13
DCRedNet0.583 130.682 80.723 140.542 120.510 220.310 160.451 150.668 180.549 140.520 90.920 80.375 70.446 20.528 220.417 160.670 160.577 190.478 120.862 120.806 110.628 11
CMX0.613 60.681 90.725 130.502 130.634 60.297 190.478 120.830 20.651 40.537 70.924 40.375 70.315 160.686 70.451 150.714 50.543 230.504 60.894 70.823 50.688 5
Enet (reimpl)0.376 270.264 280.452 280.452 140.365 260.181 260.143 280.456 270.409 270.346 270.769 280.164 260.218 260.359 270.123 280.403 280.381 280.313 280.571 270.685 260.472 25
Re-implementation of Adam Paszke, Abhishek Chaurasia, Sangpil Kim, Eugenio Culurciello: ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation.
SSMAcopyleft0.577 150.695 60.716 160.439 150.563 160.314 150.444 170.719 110.551 130.503 100.887 170.346 180.348 110.603 140.353 220.709 60.600 160.457 160.901 30.786 130.599 15
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. International Journal of Computer Vision, 2019
UNIV_CNP_RVC_UE0.566 170.569 210.686 210.435 160.524 190.294 200.421 200.712 140.543 150.463 190.872 190.320 190.363 90.611 130.477 120.686 120.627 130.443 190.862 120.775 160.639 8
MVF-GNN(2D)0.636 30.606 160.794 40.434 170.688 10.337 80.464 140.798 40.632 50.589 30.908 90.420 20.329 140.743 20.594 20.738 20.676 50.527 40.906 20.818 60.715 3
WSGFormer0.585 120.706 50.708 180.434 170.574 140.283 220.538 40.759 60.542 170.482 170.924 40.351 160.333 130.614 120.393 180.692 100.551 220.461 150.874 90.809 100.673 7
AdapNet++copyleft0.503 230.613 140.722 150.418 190.358 280.337 80.370 250.479 260.443 240.368 260.907 100.207 250.213 270.464 260.525 70.618 240.657 90.450 180.788 220.721 250.408 27
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. International Journal of Computer Vision, 2019
EMSANet0.600 80.716 40.746 100.395 200.614 90.382 50.523 60.713 130.571 120.503 100.922 70.404 50.397 50.655 100.400 170.626 230.663 60.469 140.900 40.827 40.577 16
Seichter, Daniel and Fischedick, Söhnke and Köhler, Mona and Gross, Horst-Michael: EMSANet: Efficient Multi-Task RGB-D Scene Analysis for Indoor Environments. IJCNN 2022
ILC-PSPNet0.475 260.490 250.581 260.289 210.507 230.067 280.379 240.610 250.417 260.435 220.822 270.278 230.267 210.503 240.228 240.616 250.533 240.375 250.820 200.729 230.560 18
DMMF0.567 160.623 110.767 50.238 220.571 150.347 60.413 210.719 110.472 220.418 240.895 150.357 130.260 240.696 50.523 80.666 180.642 120.437 200.895 60.793 120.603 14
MIX6D_RVC0.582 140.695 60.687 190.225 230.632 70.328 130.550 10.748 80.623 60.494 160.890 160.350 170.254 250.688 60.454 130.716 40.597 180.489 100.881 80.768 180.575 17
FuseNetpermissive0.535 220.570 200.681 220.182 240.512 210.290 210.431 180.659 190.504 200.495 150.903 130.308 210.428 40.523 230.365 210.676 130.621 150.470 130.762 240.779 150.541 19
Caner Hazirbas, Lingni Ma, Csaba Domokos, Daniel Cremers: FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-based CNN Architecture. ACCV 2016
CU-Hybrid-2D Net0.636 30.825 20.820 20.179 250.648 40.463 30.549 20.742 90.676 20.628 20.961 10.420 20.379 70.684 80.381 200.732 30.723 30.599 20.827 180.851 20.634 9
segfomer with 6d0.542 210.594 170.687 190.146 260.579 130.308 170.515 80.703 150.472 220.498 140.868 200.369 100.282 190.589 170.390 190.701 90.556 210.416 230.860 140.759 200.539 21
MSeg1080_RVCpermissive0.485 250.505 240.709 170.092 270.427 250.241 240.411 220.654 210.385 280.457 200.861 220.053 280.279 200.503 240.481 100.645 200.626 140.365 260.748 260.725 240.529 22
John Lambert*, Zhuang Liu*, Ozan Sener, James Hays, Vladlen Koltun: MSeg: A Composite Dataset for Multi-domain Semantic Segmentation. CVPR 2020
FAN_NV_RVC0.586 110.510 230.764 60.079 280.620 80.330 110.494 100.753 70.573 100.556 50.884 180.405 40.303 180.718 30.452 140.672 140.658 80.509 50.898 50.813 80.727 2