The 2D semantic labeling task involves predicting a per-pixel semantic labeling of an image.

Evaluation and metrics

Our evaluation ranks all methods according to the PASCAL VOC intersection-over-union metric (IoU). IoU = TP/(TP+FP+FN), where TP, FP, and FN are the numbers of true positive, false positive, and false negative pixels, respectively.



This table lists the benchmark results for the 2D semantic label scenario.


Method Infoavg ioubathtubbedbookshelfcabinetchaircountercurtaindeskdoorfloorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwallwindow
sort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysorted bysort bysort bysort bysort bysort by
Virtual MVFusion (R)0.745 10.861 10.839 10.881 10.672 20.512 10.422 170.898 10.723 10.714 10.954 20.454 10.509 10.773 10.895 10.756 10.820 10.653 10.935 10.891 10.728 1
Abhijit Kundu, Xiaoqi Yin, Alireza Fathi, David Ross, Brian Brewington, Thomas Funkhouser, Caroline Pantofaru: Virtual Multi-view Fusion for 3D Semantic Segmentation. ECCV 2020
MVF-GNN(2D)0.636 30.606 140.794 40.434 160.688 10.337 80.464 120.798 30.632 50.589 30.908 80.420 20.329 120.743 20.594 20.738 20.676 50.527 40.906 20.818 60.715 3
CU-Hybrid-2D Net0.636 30.825 20.820 20.179 230.648 40.463 30.549 20.742 70.676 20.628 20.961 10.420 20.379 60.684 80.381 180.732 30.723 30.599 20.827 160.851 20.634 7
MIX6D_RVC0.582 120.695 50.687 170.225 210.632 70.328 130.550 10.748 60.623 60.494 150.890 140.350 150.254 230.688 60.454 120.716 40.597 170.489 90.881 80.768 160.575 15
CMX0.613 50.681 80.725 120.502 120.634 60.297 180.478 100.830 20.651 40.537 70.924 40.375 70.315 140.686 70.451 140.714 50.543 210.504 60.894 70.823 50.688 4
SSMAcopyleft0.577 130.695 50.716 150.439 140.563 140.314 140.444 150.719 90.551 120.503 100.887 150.346 160.348 100.603 120.353 200.709 60.600 150.457 140.901 30.786 110.599 13
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. International Journal of Computer Vision, 2019
EMSAFormer0.564 160.581 160.736 110.564 100.546 160.219 230.517 50.675 140.486 190.427 210.904 110.352 140.320 130.589 150.528 50.708 70.464 240.413 220.847 140.786 110.611 11
SN_RN152pyrx8_RVCcopyleft0.546 170.572 170.663 210.638 70.518 180.298 170.366 240.633 210.510 170.446 190.864 190.296 200.267 190.542 190.346 210.704 80.575 190.431 190.853 130.766 170.630 8
segfomer with 6d0.542 190.594 150.687 170.146 240.579 120.308 160.515 60.703 130.472 200.498 130.868 180.369 90.282 170.589 150.390 170.701 90.556 200.416 210.860 120.759 180.539 19
BPNet_2Dcopyleft0.670 20.822 30.795 30.836 20.659 30.481 20.451 130.769 40.656 30.567 40.931 30.395 60.390 50.700 40.534 40.689 100.770 20.574 30.865 90.831 30.675 5
Wenbo Hu, Hengshuang Zhao, Li Jiang, Jiaya Jia and Tien-Tsin Wong: Bidirectional Projection Network for Cross Dimension Scene Understanding. CVPR 2021 (Oral)
UNIV_CNP_RVC_UE0.566 150.569 190.686 190.435 150.524 170.294 190.421 180.712 120.543 140.463 170.872 170.320 170.363 80.611 110.477 110.686 110.627 120.443 170.862 100.775 140.639 6
FuseNetpermissive0.535 200.570 180.681 200.182 220.512 190.290 200.431 160.659 170.504 180.495 140.903 120.308 190.428 30.523 210.365 190.676 120.621 140.470 120.762 220.779 130.541 17
Caner Hazirbas, Lingni Ma, Csaba Domokos, Daniel Cremers: FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-based CNN Architecture. ACCV 2016
FAN_NV_RVC0.586 100.510 210.764 60.079 260.620 80.330 110.494 80.753 50.573 90.556 50.884 160.405 40.303 160.718 30.452 130.672 130.658 70.509 50.898 50.813 80.727 2
RFBNet0.592 90.616 110.758 70.659 50.581 110.330 110.469 110.655 180.543 140.524 80.924 40.355 130.336 110.572 170.479 100.671 140.648 90.480 100.814 190.814 70.614 10
DCRedNet0.583 110.682 70.723 130.542 110.510 200.310 150.451 130.668 160.549 130.520 90.920 70.375 70.446 20.528 200.417 150.670 150.577 180.478 110.862 100.806 90.628 9
MCA-Net0.595 80.533 200.756 80.746 40.590 100.334 100.506 70.670 150.587 80.500 120.905 100.366 100.352 90.601 130.506 80.669 160.648 90.501 70.839 150.769 150.516 21
DMMF0.567 140.623 100.767 50.238 200.571 130.347 60.413 190.719 90.472 200.418 220.895 130.357 120.260 220.696 50.523 70.666 170.642 110.437 180.895 60.793 100.603 12
MSeg1080_RVCpermissive0.485 230.505 220.709 160.092 250.427 230.241 220.411 200.654 190.385 260.457 180.861 200.053 260.279 180.503 220.481 90.645 180.626 130.365 240.748 240.725 220.529 20
John Lambert*, Zhuang Liu*, Ozan Sener, James Hays, Vladlen Koltun: MSeg: A Composite Dataset for Multi-domain Semantic Segmentation. CVPR 2020
UDSSEG_RVC0.545 180.610 130.661 220.588 80.556 150.268 210.482 90.642 200.572 100.475 160.836 230.312 180.367 70.630 100.189 230.639 190.495 230.452 150.826 170.756 200.541 17
DMMF_3d0.605 60.651 90.744 100.782 30.637 50.387 40.536 30.732 80.590 70.540 60.856 210.359 110.306 150.596 140.539 30.627 200.706 40.497 80.785 210.757 190.476 22
EMSANet0.600 70.716 40.746 90.395 180.614 90.382 50.523 40.713 110.571 110.503 100.922 60.404 50.397 40.655 90.400 160.626 210.663 60.469 130.900 40.827 40.577 14
Seichter, Daniel and Fischedick, Söhnke and Köhler, Mona and Gross, Horst-Michael: EMSANet: Efficient Multi-Task RGB-D Scene Analysis for Indoor Environments. IJCNN 2022
AdapNet++copyleft0.503 210.613 120.722 140.418 170.358 260.337 80.370 230.479 240.443 220.368 240.907 90.207 230.213 250.464 240.525 60.618 220.657 80.450 160.788 200.721 230.408 25
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. International Journal of Computer Vision, 2019
ILC-PSPNet0.475 240.490 230.581 240.289 190.507 210.067 260.379 220.610 230.417 240.435 200.822 250.278 210.267 190.503 220.228 220.616 230.533 220.375 230.820 180.729 210.560 16
3DMV (2d proj)0.498 220.481 240.612 230.579 90.456 220.343 70.384 210.623 220.525 160.381 230.845 220.254 220.264 210.557 180.182 240.581 240.598 160.429 200.760 230.661 250.446 24
Angela Dai, Matthias Niessner: 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation. ECCV'18
ScanNet (2d proj)permissive0.330 260.293 250.521 250.657 60.361 250.161 250.250 250.004 260.440 230.183 260.836 230.125 250.060 260.319 260.132 250.417 250.412 250.344 250.541 260.427 260.109 26
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nießner: ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. CVPR'17
Enet (reimpl)0.376 250.264 260.452 260.452 130.365 240.181 240.143 260.456 250.409 250.346 250.769 260.164 240.218 240.359 250.123 260.403 260.381 260.313 260.571 250.685 240.472 23
Re-implementation of Adam Paszke, Abhishek Chaurasia, Sangpil Kim, Eugenio Culurciello: ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation.