Benchmark Results - ScanNet Benchmark

This table lists the benchmark results for the 3D semantic label scenario.

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

MinkowskiNet	0.736 1	0.859 2	0.818 2	0.832 2	0.709 2	0.840 2	0.521 2	0.853 1	0.660 1	0.643 1	0.951 3	0.544 2	0.286 7	0.731 1	0.893 1	0.675 5	0.772 4	0.683 1	0.874 8	0.852 2	0.727 1
C. Choy, J. Gwak, S. Savarese: 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks. CVPR 2019
SparseConvNet	0.725 2	0.647 18	0.821 1	0.846 1	0.721 1	0.869 1	0.533 1	0.754 6	0.603 4	0.614 2	0.955 1	0.572 1	0.325 1	0.710 2	0.870 2	0.724 2	0.823 1	0.628 3	0.934 1	0.865 1	0.683 2

KP-FCNN	0.684 4	0.847 4	0.758 7	0.784 6	0.647 4	0.814 5	0.473 5	0.772 5	0.605 3	0.594 3	0.935 17	0.450 8	0.181 22	0.587 6	0.805 5	0.690 4	0.785 3	0.614 4	0.882 5	0.819 4	0.632 6
H. Thomas, C. Qi, J. Deschaud, B. Marcotegui, F. Goulette, L. Guibas.: KPConv: Flexible and Deformable Convolution for Point Clouds. ICCV 2019
CU-Hybrid Net	0.693 3	0.596 21	0.789 3	0.803 3	0.677 3	0.800 8	0.469 6	0.846 2	0.554 10	0.591 4	0.948 7	0.500 3	0.316 2	0.609 4	0.847 3	0.732 1	0.808 2	0.593 5	0.894 4	0.839 3	0.652 3

joint point-based	0.634 8	0.614 20	0.778 4	0.667 15	0.633 7	0.825 3	0.420 9	0.804 3	0.467 20	0.561 5	0.951 3	0.494 4	0.291 4	0.566 9	0.458 17	0.579 16	0.764 5	0.559 13	0.838 13	0.814 5	0.598 13
Hung-Yueh Chiang, Yen-Liang Lin, Yueh-Cheng Liu, Winston H. Hsu: A Unified Point-Based Framework for 3D Segmentation. 3DV 2019
MVPNet	0.641 7	0.831 5	0.715 11	0.671 13	0.590 11	0.781 10	0.394 15	0.679 14	0.642 2	0.553 6	0.937 16	0.462 6	0.256 9	0.649 3	0.406 20	0.626 10	0.691 10	0.666 2	0.877 6	0.792 11	0.608 10
Maximilian Jaritz, Jiayuan Gu, Hao Su: Multi-view PointNet for 3D Scene Understanding. GMDL Workshop, ICCV 2019
PointASNL	0.630 11	0.738 10	0.729 10	0.764 9	0.637 6	0.779 11	0.416 11	0.626 18	0.518 12	0.530 7	0.951 3	0.398 14	0.260 8	0.518 11	0.576 13	0.590 14	0.687 11	0.568 9	0.872 9	0.810 7	0.631 7

HPEIN	0.618 12	0.729 12	0.668 17	0.647 16	0.597 10	0.766 14	0.414 12	0.680 12	0.520 11	0.525 8	0.946 8	0.432 10	0.215 14	0.493 16	0.599 11	0.638 8	0.617 21	0.570 8	0.897 3	0.806 8	0.605 11
Li Jiang, Hengshuang Zhao, Shu Liu, Xiaoyong Shen, Chi-Wing Fu, Jiaya Jia: Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentation. ICCV 2019
DMC-Net	0.653 6	0.771 7	0.701 12	0.801 4	0.619 8	0.807 7	0.463 7	0.680 12	0.495 16	0.520 9	0.940 13	0.452 7	0.301 3	0.496 15	0.816 4	0.664 6	0.719 7	0.563 11	0.822 15	0.799 9	0.638 5

SPH3D-GCN	0.610 13	0.858 3	0.772 5	0.489 27	0.532 15	0.792 9	0.404 14	0.643 17	0.570 7	0.507 10	0.935 17	0.414 12	0.046 32	0.510 13	0.702 7	0.602 12	0.705 8	0.549 14	0.859 11	0.773 14	0.534 18
Huan Lei, Naveed Akhtar, and Ajmal Mian: Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds.
PointConv	0.666 5	0.781 6	0.759 6	0.699 10	0.644 5	0.822 4	0.475 4	0.779 4	0.564 9	0.504 11	0.953 2	0.428 11	0.203 18	0.586 7	0.754 6	0.661 7	0.753 6	0.588 6	0.902 2	0.813 6	0.642 4
Wenxuan Wu, Zhongang Qi, Li Fuxin: PointConv: Deep Convolutional Networks on 3D Point Clouds. CVPR 2019
MCCNN	0.633 9	0.866 1	0.731 9	0.771 7	0.576 13	0.809 6	0.410 13	0.684 11	0.497 15	0.491 12	0.949 6	0.466 5	0.105 28	0.581 8	0.646 9	0.620 11	0.680 13	0.542 16	0.817 16	0.795 10	0.618 8
P. Hermosilla, T. Ritschel, P.P. Vazquez, A. Vinacua, T. Ropinski: Monte Carlo Convolution for Learning on Non-Uniformly Sampled Point Clouds. SIGGRAPH Asia 2018
3DSM_DMMF	0.631 10	0.626 19	0.745 8	0.801 4	0.607 9	0.751 16	0.506 3	0.729 8	0.565 8	0.491 12	0.866 30	0.434 9	0.197 20	0.595 5	0.630 10	0.709 3	0.705 8	0.560 12	0.875 7	0.740 20	0.491 23

CCRFNet	0.589 17	0.766 9	0.659 20	0.683 12	0.470 20	0.740 18	0.387 18	0.620 19	0.490 17	0.476 14	0.922 22	0.355 21	0.245 10	0.511 12	0.511 15	0.571 17	0.643 17	0.493 20	0.872 9	0.762 15	0.600 12

DVVNet	0.562 19	0.648 17	0.700 13	0.770 8	0.586 12	0.687 23	0.333 21	0.650 16	0.514 13	0.475 15	0.906 27	0.359 19	0.223 13	0.340 24	0.442 19	0.422 27	0.668 14	0.501 18	0.708 25	0.779 12	0.534 18

SIConv	0.594 14	0.768 8	0.639 23	0.616 19	0.544 14	0.768 13	0.419 10	0.601 20	0.513 14	0.474 16	0.946 8	0.402 13	0.213 16	0.387 22	0.581 12	0.633 9	0.683 12	0.549 14	0.843 12	0.774 13	0.521 20

3DMV, FTSDF	0.501 22	0.558 25	0.608 26	0.424 31	0.478 19	0.690 22	0.246 28	0.586 21	0.468 19	0.450 17	0.911 25	0.394 15	0.160 24	0.438 17	0.212 28	0.432 26	0.541 26	0.475 21	0.742 23	0.727 21	0.477 24

LAP-D	0.594 14	0.720 13	0.692 15	0.637 18	0.456 21	0.773 12	0.391 17	0.730 7	0.587 5	0.445 18	0.940 13	0.381 16	0.288 5	0.434 18	0.453 18	0.591 13	0.649 15	0.581 7	0.777 20	0.749 19	0.610 9

DPC	0.592 16	0.720 13	0.700 13	0.602 22	0.480 18	0.762 15	0.380 19	0.713 9	0.585 6	0.437 19	0.940 13	0.369 18	0.288 5	0.434 18	0.509 16	0.590 14	0.639 19	0.567 10	0.772 21	0.755 17	0.592 14
Francis Engelmann, Theodora Kontogianni, Bastian Leibe: Dilated Point Convolutions. arXiv
TextureNet	0.566 18	0.672 16	0.664 18	0.671 13	0.494 16	0.719 19	0.445 8	0.678 15	0.411 25	0.396 20	0.935 17	0.356 20	0.225 12	0.412 20	0.535 14	0.565 18	0.636 20	0.464 22	0.794 19	0.680 25	0.568 15
Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkerhouser, Matthias Niessner, Leonidas Guibas: TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshes. CVPR
3DMV	0.484 24	0.484 29	0.538 29	0.643 17	0.424 22	0.606 31	0.310 22	0.574 22	0.433 24	0.378 21	0.796 32	0.301 24	0.214 15	0.537 10	0.208 29	0.472 25	0.507 30	0.413 29	0.693 26	0.602 30	0.539 17
Angela Dai, Matthias Niessner: 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation. ECCV'18
Pointnet++ & Feature	0.557 20	0.735 11	0.661 19	0.686 11	0.491 17	0.744 17	0.392 16	0.539 23	0.451 21	0.375 22	0.946 8	0.376 17	0.205 17	0.403 21	0.356 22	0.553 19	0.643 17	0.497 19	0.824 14	0.756 16	0.515 21

PCNN	0.498 23	0.559 24	0.644 22	0.560 25	0.420 23	0.711 21	0.229 30	0.414 24	0.436 22	0.352 23	0.941 12	0.324 23	0.155 25	0.238 28	0.387 21	0.493 21	0.529 27	0.509 17	0.813 17	0.751 18	0.504 22

PointCNN with RGB	0.458 25	0.577 23	0.611 25	0.356 33	0.321 30	0.715 20	0.299 24	0.376 27	0.328 30	0.319 24	0.944 11	0.285 26	0.164 23	0.216 31	0.229 27	0.484 23	0.545 25	0.456 24	0.755 22	0.709 22	0.475 25
Yangyan Li, Rui Bu, Mingchao Sun, Baoquan Chen: PointCNN. NeurIPS 2018
PNET2	0.442 27	0.548 26	0.548 28	0.597 23	0.363 27	0.628 29	0.300 23	0.292 28	0.374 27	0.307 25	0.881 29	0.268 27	0.186 21	0.238 28	0.204 30	0.407 28	0.506 31	0.449 25	0.667 27	0.620 29	0.462 26

PanopticFusion-label	0.529 21	0.491 28	0.688 16	0.604 21	0.386 24	0.632 28	0.225 32	0.705 10	0.434 23	0.293 26	0.815 31	0.348 22	0.241 11	0.499 14	0.669 8	0.507 20	0.649 15	0.442 26	0.796 18	0.602 30	0.561 16
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
Tangent Convolutions	0.438 29	0.437 31	0.646 21	0.474 28	0.369 26	0.645 27	0.353 20	0.258 30	0.282 32	0.279 27	0.918 24	0.298 25	0.147 26	0.283 25	0.294 24	0.487 22	0.562 23	0.427 28	0.619 29	0.633 28	0.352 29
Maxim Tatarchenko, Jaesik Park, Vladlen Koltun, Qian-Yi Zhou: Tangent convolutions for dense prediction in 3d. CVPR 2018
SurfaceConvPF	0.442 27	0.505 27	0.622 24	0.380 32	0.342 29	0.654 26	0.227 31	0.397 26	0.367 28	0.276 28	0.924 21	0.240 29	0.198 19	0.359 23	0.262 25	0.366 29	0.581 22	0.435 27	0.640 28	0.668 26	0.398 27
Hao Pan, Shilin Liu, Yang Liu, Xin Tong: Convolutional Neural Networks on 3D Surfaces Using Parallel Frames.
ScanNet+FTSDF	0.383 31	0.297 33	0.491 31	0.432 30	0.358 28	0.612 30	0.274 26	0.116 32	0.411 25	0.265 29	0.904 28	0.229 30	0.079 30	0.250 26	0.185 31	0.320 32	0.510 28	0.385 30	0.548 31	0.597 32	0.394 28

PointNet++	0.339 32	0.584 22	0.478 32	0.458 29	0.256 33	0.360 34	0.250 27	0.247 31	0.278 33	0.261 30	0.677 34	0.183 32	0.117 27	0.212 32	0.145 33	0.364 30	0.346 34	0.232 34	0.548 31	0.523 33	0.252 32
Charles R. Qi, Li Yi, Hao Su, Leonidas J. Guibas: pointnet++: deep hierarchical feature learning on point sets in a metric space.
FCPN	0.447 26	0.679 15	0.604 27	0.578 24	0.380 25	0.682 24	0.291 25	0.106 33	0.483 18	0.258 31	0.920 23	0.258 28	0.025 33	0.231 30	0.325 23	0.480 24	0.560 24	0.463 23	0.725 24	0.666 27	0.231 33
Dario Rethage, Johanna Wald, Jürgen Sturm, Nassir Navab, Federico Tombari: Fully-Convolutional Point Networks for Large-Scale Point Clouds. ECCV 2018
SPLAT Net	0.393 30	0.472 30	0.511 30	0.606 20	0.311 31	0.656 25	0.245 29	0.405 25	0.328 30	0.197 32	0.927 20	0.227 31	0.000 35	0.001 35	0.249 26	0.271 34	0.510 28	0.383 31	0.593 30	0.699 23	0.267 31
Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, Jan Kautz: SPLATNet: Sparse Lattice Networks for Point Cloud Processing. CVPR 2018
ScanNet	0.306 34	0.203 34	0.366 33	0.501 26	0.311 31	0.524 33	0.211 33	0.002 35	0.342 29	0.189 33	0.786 33	0.145 34	0.102 29	0.245 27	0.152 32	0.318 33	0.348 33	0.300 33	0.460 34	0.437 34	0.182 34
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nießner: ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. CVPR'17
SSC-UNet	0.308 33	0.353 32	0.290 34	0.278 34	0.166 34	0.553 32	0.169 34	0.286 29	0.147 34	0.148 34	0.908 26	0.182 33	0.064 31	0.023 34	0.018 35	0.354 31	0.363 32	0.345 32	0.546 33	0.685 24	0.278 30

ERROR	0.054 35	0.000 35	0.041 35	0.172 35	0.030 35	0.062 35	0.001 35	0.035 34	0.004 35	0.051 35	0.143 35	0.019 35	0.003 34	0.041 33	0.050 34	0.003 35	0.054 35	0.018 35	0.005 35	0.264 35	0.082 35

This table lists the benchmark results for the 3D semantic instance scenario.

Method	avg ap 50%	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	window

OccuSeg	0.634 3	1.000 1	0.902 1	0.771 2	0.461 5	0.814 1	0.282 1	0.583 9	0.328 5	0.472 1	0.471 2	0.295 7	0.600 3	1.000 1	0.650 2	0.664 7	0.587 4	1.000 1	0.537 1

GICN	0.638 1	1.000 1	0.895 2	0.800 1	0.480 4	0.676 5	0.144 2	0.737 2	0.354 3	0.447 2	0.400 8	0.365 4	0.700 1	1.000 1	0.569 3	0.836 1	0.599 2	1.000 1	0.473 3

PointGroup	0.636 2	1.000 1	0.765 5	0.624 4	0.505 3	0.797 2	0.116 6	0.696 4	0.384 2	0.441 3	0.559 1	0.476 1	0.596 5	1.000 1	0.666 1	0.756 4	0.556 6	0.997 5	0.513 2

SSEN	0.568 5	1.000 1	0.747 6	0.449 9	0.371 7	0.760 3	0.143 3	0.706 3	0.336 4	0.439 4	0.430 6	0.306 6	0.600 3	0.857 5	0.407 8	0.831 2	0.611 1	0.944 7	0.283 10

MPA	0.610 4	1.000 1	0.833 3	0.765 3	0.546 1	0.750 4	0.140 4	0.588 8	0.478 1	0.433 5	0.454 3	0.376 3	0.650 2	0.857 5	0.429 6	0.765 3	0.537 8	1.000 1	0.378 7

MTML	0.549 6	1.000 1	0.807 4	0.588 7	0.327 9	0.647 6	0.004 15	0.815 1	0.180 9	0.418 6	0.364 10	0.182 9	0.445 7	1.000 1	0.442 5	0.688 6	0.571 5	1.000 1	0.396 5
Jean Lahoud, Bernard Ghanem, Marc Pollefeys, Martin R. Oswald: 3D Instance Segmentation via Multi-task Metric Learning. ICCV 2019 [oral]
ResNet-backbone	0.459 10	1.000 1	0.737 7	0.159 16	0.259 11	0.587 10	0.138 5	0.475 11	0.217 8	0.416 7	0.408 7	0.128 11	0.315 12	0.714 11	0.411 7	0.536 13	0.590 3	0.873 13	0.304 9

MASC	0.447 11	0.528 14	0.555 13	0.381 10	0.382 6	0.633 7	0.002 16	0.509 10	0.260 7	0.361 8	0.432 5	0.327 5	0.451 6	0.571 12	0.367 11	0.639 8	0.386 10	0.980 6	0.276 11
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
Occipital-SCS	0.512 7	1.000 1	0.716 8	0.509 8	0.506 2	0.611 8	0.092 8	0.602 7	0.177 10	0.346 9	0.383 9	0.165 10	0.442 8	0.850 9	0.386 10	0.618 9	0.543 7	0.889 11	0.389 6

3D-BoNet	0.488 8	1.000 1	0.672 11	0.590 6	0.301 10	0.484 13	0.098 7	0.620 5	0.306 6	0.341 10	0.259 12	0.125 12	0.434 9	0.796 10	0.402 9	0.499 14	0.513 9	0.909 10	0.439 4
Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, Niki Trigoni: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds. NeurIPS 2019 Spotlight
3D-SIS	0.382 12	1.000 1	0.432 14	0.245 13	0.190 13	0.577 11	0.013 13	0.263 13	0.033 16	0.320 11	0.240 13	0.075 14	0.422 10	0.857 5	0.117 15	0.699 5	0.271 14	0.883 12	0.235 13
Ji Hou, Angela Dai, Matthias Niessner: 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans. CVPR 2019
R-PointNet	0.306 14	0.500 15	0.405 15	0.311 11	0.348 8	0.589 9	0.054 9	0.068 16	0.126 12	0.283 12	0.290 11	0.028 15	0.219 13	0.214 16	0.331 12	0.396 15	0.275 13	0.821 15	0.245 12

PanopticFusion-inst	0.478 9	0.667 11	0.712 10	0.595 5	0.259 12	0.550 12	0.000 18	0.613 6	0.175 11	0.250 13	0.434 4	0.437 2	0.411 11	0.857 5	0.485 4	0.591 12	0.267 15	0.944 7	0.359 8
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
UNet-backbone	0.319 13	0.667 11	0.715 9	0.233 14	0.189 14	0.479 14	0.008 14	0.218 14	0.067 15	0.201 14	0.173 14	0.107 13	0.123 14	0.438 13	0.150 13	0.615 10	0.355 11	0.916 9	0.093 17

Seg-Cluster	0.215 16	0.370 16	0.337 17	0.285 12	0.105 15	0.325 16	0.025 12	0.282 12	0.085 14	0.105 15	0.107 15	0.007 18	0.079 16	0.317 15	0.114 16	0.309 17	0.304 12	0.587 16	0.123 16

3D-BEVIS	0.248 15	0.667 11	0.566 12	0.076 17	0.035 18	0.394 15	0.027 11	0.035 17	0.098 13	0.099 16	0.030 17	0.025 16	0.098 15	0.375 14	0.126 14	0.604 11	0.181 16	0.854 14	0.171 14
Cathrin Elich, Francis Engelmann, Jonas Schult, Theodora Kontogianni, Bastian Leibe: 3D-BEVIS: Birds-Eye-View Instance Segmentation.
Sgpn_scannet	0.143 17	0.208 18	0.390 16	0.169 15	0.065 16	0.275 17	0.029 10	0.069 15	0.000 17	0.087 17	0.043 16	0.014 17	0.027 18	0.000 17	0.112 17	0.351 16	0.168 17	0.438 17	0.138 15

MaskRCNN 2d->3d Proj	0.058 18	0.333 17	0.002 18	0.000 18	0.053 17	0.002 18	0.002 17	0.021 18	0.000 17	0.045 18	0.024 18	0.238 8	0.065 17	0.000 17	0.014 18	0.107 18	0.020 18	0.110 18	0.006 18

This table lists the benchmark results for the 2D semantic label scenario.

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

CU-Hybrid-2D Net	0.636 1	0.825 1	0.820 1	0.179 13	0.648 1	0.463 1	0.549 1	0.742 1	0.676 1	0.628 1	0.961 1	0.420 1	0.379 3	0.684 1	0.381 7	0.732 1	0.723 1	0.599 1	0.827 5	0.851 1	0.634 1

DMMF_3d	0.605 2	0.651 4	0.744 5	0.782 1	0.637 2	0.387 2	0.536 2	0.732 2	0.590 3	0.540 2	0.856 9	0.359 4	0.306 8	0.596 4	0.539 1	0.627 8	0.706 2	0.497 4	0.785 9	0.757 8	0.476 9

RFBNet	0.592 5	0.616 5	0.758 2	0.659 4	0.581 5	0.330 7	0.469 5	0.655 8	0.543 7	0.524 3	0.924 2	0.355 5	0.336 6	0.572 6	0.479 5	0.671 5	0.648 4	0.480 5	0.814 7	0.814 2	0.614 3

DCRedNet	0.583 6	0.682 3	0.723 6	0.542 7	0.510 8	0.310 9	0.451 6	0.668 6	0.549 6	0.520 4	0.920 3	0.375 2	0.446 1	0.528 8	0.417 6	0.670 6	0.577 10	0.478 6	0.862 2	0.806 3	0.628 2

SSMA	0.577 7	0.695 2	0.716 8	0.439 9	0.563 6	0.314 8	0.444 7	0.719 3	0.551 5	0.503 5	0.887 8	0.346 7	0.348 5	0.603 2	0.353 9	0.709 2	0.600 8	0.457 8	0.901 1	0.786 4	0.599 4
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. International Journal of Computer Vision, 2019
MCA-Net	0.595 4	0.533 9	0.756 3	0.746 3	0.590 3	0.334 6	0.506 3	0.670 5	0.587 4	0.500 6	0.905 6	0.366 3	0.352 4	0.601 3	0.506 4	0.669 7	0.648 4	0.501 3	0.839 4	0.769 7	0.516 8

FuseNet	0.535 8	0.570 7	0.681 9	0.182 12	0.512 7	0.290 10	0.431 8	0.659 7	0.504 9	0.495 7	0.903 7	0.308 8	0.428 2	0.523 9	0.365 8	0.676 3	0.621 7	0.470 7	0.762 10	0.779 5	0.541 6
Caner Hazirbas, Lingni Ma, Csaba Domokos, Daniel Cremers: FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-based CNN Architecture. ACCV 2016
DMMF	0.597 3	0.543 8	0.755 4	0.749 2	0.585 4	0.338 4	0.494 4	0.704 4	0.598 2	0.494 8	0.911 4	0.347 6	0.327 7	0.593 5	0.527 2	0.675 4	0.646 6	0.513 2	0.842 3	0.774 6	0.527 7

ILC-PSPNet	0.475 11	0.490 10	0.581 11	0.289 11	0.507 9	0.067 13	0.379 10	0.610 10	0.417 12	0.435 9	0.822 12	0.278 9	0.267 9	0.503 10	0.228 10	0.616 10	0.533 11	0.375 11	0.820 6	0.729 9	0.560 5

3DMV (2d proj)	0.498 10	0.481 11	0.612 10	0.579 6	0.456 10	0.343 3	0.384 9	0.623 9	0.525 8	0.381 10	0.845 10	0.254 10	0.264 10	0.557 7	0.182 11	0.581 11	0.598 9	0.429 10	0.760 11	0.661 12	0.446 11
Angela Dai, Matthias Niessner: 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation. ECCV'18
AdapNet++	0.503 9	0.613 6	0.722 7	0.418 10	0.358 13	0.337 5	0.370 11	0.479 11	0.443 10	0.368 11	0.907 5	0.207 11	0.213 12	0.464 11	0.525 3	0.618 9	0.657 3	0.450 9	0.788 8	0.721 10	0.408 12
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. International Journal of Computer Vision, 2019
Enet (reimpl)	0.376 12	0.264 13	0.452 13	0.452 8	0.365 11	0.181 11	0.143 13	0.456 12	0.409 13	0.346 12	0.769 13	0.164 12	0.218 11	0.359 12	0.123 13	0.403 13	0.381 13	0.313 13	0.571 12	0.685 11	0.472 10
Re-implementation of Adam Paszke, Abhishek Chaurasia, Sangpil Kim, Eugenio Culurciello: ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation.
ScanNet (2d proj)	0.330 13	0.293 12	0.521 12	0.657 5	0.361 12	0.161 12	0.250 12	0.004 13	0.440 11	0.183 13	0.836 11	0.125 13	0.060 13	0.319 13	0.132 12	0.417 12	0.412 12	0.344 12	0.541 13	0.427 13	0.109 13
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nießner: ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. CVPR'17

This table lists the benchmark results for the 2D semantic instance scenario.

Method	avg ap	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	window

MaskRCNN_ScanNet	0.119 1	0.129 1	0.212 1	0.002 1	0.112 1	0.148 1	0.014 1	0.205 1	0.044 1	0.066 1	0.078 1	0.095 1	0.142 1	0.030 1	0.128 1	0.139 1	0.080 1	0.459 1	0.057 1
Re-implementation of Kaiming He, Georgia Gkioxari, Piotr Dollár, Ross Girshick: Mask R-CNN. ICCV'17

This table lists the benchmark results for the scene type classification scenario.

Method	avg recall	apartment	bathroom	bedroom / hotel	bookstore / library	conference room	copy/mail room	hallway	kitchen	laundry room	living room / lounge	misc	office	storage / basement / garage

MTL	0.700 1	0.500 1	1.000 1	0.882 2	0.500 1	1.000 1	1.000 1	0.500 1	1.000 1	1.000 1	0.778 1	0.000 2	0.938 1	0.000 1

SE-ResNeXt-SSMA	0.498 2	0.000 3	0.812 2	0.941 1	0.500 1	0.500 2	0.500 2	0.500 1	0.429 3	0.500 2	0.667 2	0.500 1	0.625 2	0.000 1
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. arXiv
resnet50_scannet	0.353 3	0.250 2	0.812 2	0.529 3	0.500 1	0.500 2	0.000 3	0.500 1	0.571 2	0.000 3	0.556 3	0.000 2	0.375 3	0.000 1

3D Semantic label benchmark

3D Semantic instance benchmark

2D Semantic label benchmark

2D Semantic instance benchmark

Scene type classification benchmark