Benchmark Results - ScanNet Benchmark

This table lists the benchmark results for the 3D semantic label scenario.

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

MinkowskiNet	0.736 1	0.859 2	0.818 2	0.832 2	0.709 2	0.840 2	0.521 2	0.853 1	0.660 1	0.643 1	0.951 3	0.544 2	0.286 6	0.731 1	0.893 1	0.675 4	0.772 3	0.683 1	0.874 7	0.852 2	0.727 1
C. Choy, J. Gwak, S. Savarese: 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks. CVPR 2019
SparseConvNet	0.725 2	0.647 18	0.821 1	0.846 1	0.721 1	0.869 1	0.533 1	0.754 5	0.603 4	0.614 2	0.955 1	0.572 1	0.325 1	0.710 2	0.870 2	0.724 1	0.823 1	0.628 3	0.934 1	0.865 1	0.683 2

KP-FCNN	0.684 3	0.847 4	0.758 6	0.784 4	0.647 3	0.814 5	0.473 5	0.772 4	0.605 3	0.594 3	0.935 16	0.450 7	0.181 21	0.587 4	0.805 4	0.690 3	0.785 2	0.614 4	0.882 5	0.819 3	0.632 5
H. Thomas, C. Qi, J. Deschaud, B. Marcotegui, F. Goulette, L. Guibas.: KPConv: Flexible and Deformable Convolution for Point Clouds. ICCV 2019
joint point-based	0.634 7	0.614 19	0.778 3	0.667 14	0.633 6	0.825 3	0.420 8	0.804 2	0.467 19	0.561 4	0.951 3	0.494 3	0.291 3	0.566 8	0.458 16	0.579 15	0.764 4	0.559 12	0.838 12	0.814 4	0.598 12
Hung-Yueh Chiang, Yen-Liang Lin, Yueh-Cheng Liu, Winston H. Hsu: A Unified Point-Based Framework for 3D Segmentation. 3DV 2019
MVPNet	0.641 6	0.831 5	0.715 10	0.671 12	0.590 10	0.781 9	0.394 14	0.679 13	0.642 2	0.553 5	0.937 15	0.462 5	0.256 8	0.649 3	0.406 19	0.626 9	0.691 9	0.666 2	0.877 6	0.792 10	0.608 9
Maximilian Jaritz, Jiayuan Gu, Hao Su: Multi-view PointNet for 3D Scene Understanding. GMDL Workshop, ICCV 2019
PointASNL	0.630 9	0.738 10	0.729 9	0.764 8	0.637 5	0.779 10	0.416 10	0.626 17	0.518 11	0.530 6	0.951 3	0.398 13	0.260 7	0.518 10	0.576 12	0.590 13	0.687 10	0.568 8	0.872 8	0.810 6	0.631 6

HPEIN	0.618 11	0.729 12	0.668 16	0.647 15	0.597 9	0.766 13	0.414 11	0.680 11	0.520 10	0.525 7	0.946 7	0.432 8	0.215 13	0.493 15	0.599 9	0.638 7	0.617 20	0.570 7	0.897 3	0.806 7	0.605 10
Li Jiang, Hengshuang Zhao, Shu Liu, Xiaoyong Shen, Chi-Wing Fu, Jiaya Jia: Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentation. ICCV 2019
DMC-Net	0.653 5	0.771 7	0.701 11	0.801 3	0.619 7	0.807 7	0.463 6	0.680 11	0.495 15	0.520 8	0.940 12	0.452 6	0.301 2	0.496 14	0.816 3	0.664 5	0.719 6	0.563 11	0.822 14	0.799 8	0.638 4

SPH3D-GCN	0.610 12	0.858 3	0.772 4	0.489 26	0.532 14	0.792 8	0.404 13	0.643 16	0.570 7	0.507 9	0.935 16	0.414 11	0.046 31	0.510 12	0.702 6	0.602 11	0.705 8	0.549 13	0.859 10	0.773 13	0.534 17
Huan Lei, Naveed Akhtar, and Ajmal Mian: Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds.
PointConv	0.666 4	0.781 6	0.759 5	0.699 9	0.644 4	0.822 4	0.475 4	0.779 3	0.564 8	0.504 10	0.953 2	0.428 9	0.203 17	0.586 5	0.754 5	0.661 6	0.753 5	0.588 5	0.902 2	0.813 5	0.642 3
Wenxuan Wu, Zhongang Qi, Li Fuxin: PointConv: Deep Convolutional Networks on 3D Point Clouds. CVPR 2019
CDF-SM3D	0.626 10	0.592 20	0.746 7	0.767 7	0.607 8	0.761 15	0.501 3	0.738 6	0.546 9	0.503 11	0.864 29	0.421 10	0.198 18	0.584 6	0.579 11	0.694 2	0.706 7	0.566 10	0.885 4	0.745 19	0.523 19

MCCNN	0.633 8	0.866 1	0.731 8	0.771 5	0.576 12	0.809 6	0.410 12	0.684 10	0.497 14	0.491 12	0.949 6	0.466 4	0.105 27	0.581 7	0.646 8	0.620 10	0.680 12	0.542 15	0.817 15	0.795 9	0.618 7
P. Hermosilla, T. Ritschel, P.P. Vazquez, A. Vinacua, T. Ropinski: Monte Carlo Convolution for Learning on Non-Uniformly Sampled Point Clouds. SIGGRAPH Asia 2018
CCRFNet	0.589 16	0.766 9	0.659 19	0.683 11	0.470 19	0.740 17	0.387 17	0.620 18	0.490 16	0.476 13	0.922 21	0.355 20	0.245 9	0.511 11	0.511 14	0.571 16	0.643 16	0.493 19	0.872 8	0.762 14	0.600 11

DVVNet	0.562 18	0.648 17	0.700 12	0.770 6	0.586 11	0.687 22	0.333 20	0.650 15	0.514 12	0.475 14	0.906 26	0.359 18	0.223 12	0.340 23	0.442 18	0.422 26	0.668 13	0.501 17	0.708 24	0.779 11	0.534 17

SIConv	0.594 13	0.768 8	0.639 22	0.616 18	0.544 13	0.768 12	0.419 9	0.601 19	0.513 13	0.474 15	0.946 7	0.402 12	0.213 15	0.387 21	0.581 10	0.633 8	0.683 11	0.549 13	0.843 11	0.774 12	0.521 20

3DMV, FTSDF	0.501 21	0.558 24	0.608 25	0.424 30	0.478 18	0.690 21	0.246 27	0.586 20	0.468 18	0.450 16	0.911 24	0.394 14	0.160 23	0.438 16	0.212 27	0.432 25	0.541 25	0.475 20	0.742 22	0.727 20	0.477 23

LAP-D	0.594 13	0.720 13	0.692 14	0.637 17	0.456 20	0.773 11	0.391 16	0.730 7	0.587 5	0.445 17	0.940 12	0.381 15	0.288 4	0.434 17	0.453 17	0.591 12	0.649 14	0.581 6	0.777 19	0.749 18	0.610 8

DPC	0.592 15	0.720 13	0.700 12	0.602 21	0.480 17	0.762 14	0.380 18	0.713 8	0.585 6	0.437 18	0.940 12	0.369 17	0.288 4	0.434 17	0.509 15	0.590 13	0.639 18	0.567 9	0.772 20	0.755 16	0.592 13
Francis Engelmann, Theodora Kontogianni, Bastian Leibe: Dilated Point Convolutions. arXiv
TextureNet	0.566 17	0.672 16	0.664 17	0.671 12	0.494 15	0.719 18	0.445 7	0.678 14	0.411 24	0.396 19	0.935 16	0.356 19	0.225 11	0.412 19	0.535 13	0.565 17	0.636 19	0.464 21	0.794 18	0.680 24	0.568 14
Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkerhouser, Matthias Niessner, Leonidas Guibas: TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshes. CVPR
3DMV	0.484 23	0.484 28	0.538 28	0.643 16	0.424 21	0.606 30	0.310 21	0.574 21	0.433 23	0.378 20	0.796 31	0.301 23	0.214 14	0.537 9	0.208 28	0.472 24	0.507 29	0.413 28	0.693 25	0.602 29	0.539 16
Angela Dai, Matthias Niessner: 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation. ECCV'18
Pointnet++ & Feature	0.557 19	0.735 11	0.661 18	0.686 10	0.491 16	0.744 16	0.392 15	0.539 22	0.451 20	0.375 21	0.946 7	0.376 16	0.205 16	0.403 20	0.356 21	0.553 18	0.643 16	0.497 18	0.824 13	0.756 15	0.515 21

PCNN	0.498 22	0.559 23	0.644 21	0.560 24	0.420 22	0.711 20	0.229 29	0.414 23	0.436 21	0.352 22	0.941 11	0.324 22	0.155 24	0.238 27	0.387 20	0.493 20	0.529 26	0.509 16	0.813 16	0.751 17	0.504 22

PointCNN with RGB	0.458 24	0.577 22	0.611 24	0.356 32	0.321 29	0.715 19	0.299 23	0.376 26	0.328 29	0.319 23	0.944 10	0.285 25	0.164 22	0.216 30	0.229 26	0.484 22	0.545 24	0.456 23	0.755 21	0.709 21	0.475 24
Yangyan Li, Rui Bu, Mingchao Sun, Baoquan Chen: PointCNN. NeurIPS 2018
PNET2	0.442 26	0.548 25	0.548 27	0.597 22	0.363 26	0.628 28	0.300 22	0.292 27	0.374 26	0.307 24	0.881 28	0.268 26	0.186 20	0.238 27	0.204 29	0.407 27	0.506 30	0.449 24	0.667 26	0.620 28	0.462 25

PanopticFusion-label	0.529 20	0.491 27	0.688 15	0.604 20	0.386 23	0.632 27	0.225 31	0.705 9	0.434 22	0.293 25	0.815 30	0.348 21	0.241 10	0.499 13	0.669 7	0.507 19	0.649 14	0.442 25	0.796 17	0.602 29	0.561 15
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
Tangent Convolutions	0.438 28	0.437 30	0.646 20	0.474 27	0.369 25	0.645 26	0.353 19	0.258 29	0.282 31	0.279 26	0.918 23	0.298 24	0.147 25	0.283 24	0.294 23	0.487 21	0.562 22	0.427 27	0.619 28	0.633 27	0.352 28
Maxim Tatarchenko, Jaesik Park, Vladlen Koltun, Qian-Yi Zhou: Tangent convolutions for dense prediction in 3d. CVPR 2018
SurfaceConvPF	0.442 26	0.505 26	0.622 23	0.380 31	0.342 28	0.654 25	0.227 30	0.397 25	0.367 27	0.276 27	0.924 20	0.240 28	0.198 18	0.359 22	0.262 24	0.366 28	0.581 21	0.435 26	0.640 27	0.668 25	0.398 26
Hao Pan, Shilin Liu, Yang Liu, Xin Tong: Convolutional Neural Networks on 3D Surfaces Using Parallel Frames.
ScanNet+FTSDF	0.383 30	0.297 32	0.491 30	0.432 29	0.358 27	0.612 29	0.274 25	0.116 31	0.411 24	0.265 28	0.904 27	0.229 29	0.079 29	0.250 25	0.185 30	0.320 31	0.510 27	0.385 29	0.548 30	0.597 31	0.394 27

PointNet++	0.339 31	0.584 21	0.478 31	0.458 28	0.256 32	0.360 33	0.250 26	0.247 30	0.278 32	0.261 29	0.677 33	0.183 31	0.117 26	0.212 31	0.145 32	0.364 29	0.346 33	0.232 33	0.548 30	0.523 32	0.252 31
Charles R. Qi, Li Yi, Hao Su, Leonidas J. Guibas: pointnet++: deep hierarchical feature learning on point sets in a metric space.
FCPN	0.447 25	0.679 15	0.604 26	0.578 23	0.380 24	0.682 23	0.291 24	0.106 32	0.483 17	0.258 30	0.920 22	0.258 27	0.025 32	0.231 29	0.325 22	0.480 23	0.560 23	0.463 22	0.725 23	0.666 26	0.231 32
Dario Rethage, Johanna Wald, Jürgen Sturm, Nassir Navab, Federico Tombari: Fully-Convolutional Point Networks for Large-Scale Point Clouds. ECCV 2018
SPLAT Net	0.393 29	0.472 29	0.511 29	0.606 19	0.311 30	0.656 24	0.245 28	0.405 24	0.328 29	0.197 31	0.927 19	0.227 30	0.000 34	0.001 34	0.249 25	0.271 33	0.510 27	0.383 30	0.593 29	0.699 22	0.267 30
Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, Jan Kautz: SPLATNet: Sparse Lattice Networks for Point Cloud Processing. CVPR 2018
ScanNet	0.306 33	0.203 33	0.366 32	0.501 25	0.311 30	0.524 32	0.211 32	0.002 34	0.342 28	0.189 32	0.786 32	0.145 33	0.102 28	0.245 26	0.152 31	0.318 32	0.348 32	0.300 32	0.460 33	0.437 33	0.182 33
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nießner: ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. CVPR'17
SSC-UNet	0.308 32	0.353 31	0.290 33	0.278 33	0.166 33	0.553 31	0.169 33	0.286 28	0.147 33	0.148 33	0.908 25	0.182 32	0.064 30	0.023 33	0.018 34	0.354 30	0.363 31	0.345 31	0.546 32	0.685 23	0.278 29

ERROR	0.054 34	0.000 34	0.041 34	0.172 34	0.030 34	0.062 34	0.001 34	0.035 33	0.004 34	0.051 34	0.143 34	0.019 34	0.003 33	0.041 32	0.050 33	0.003 34	0.054 34	0.018 34	0.005 34	0.264 34	0.082 34

This table lists the benchmark results for the 3D semantic instance scenario.

Method	avg ap 50%	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	window

OccuSeg	0.634 2	1.000 1	0.902 1	0.771 1	0.461 4	0.814 1	0.282 1	0.583 7	0.328 3	0.472 1	0.471 2	0.295 6	0.600 1	1.000 1	0.650 2	0.664 6	0.587 2	1.000 1	0.537 1

PointGroup	0.636 1	1.000 1	0.765 4	0.624 4	0.505 3	0.797 2	0.116 5	0.696 3	0.384 2	0.441 2	0.559 1	0.476 1	0.596 2	1.000 1	0.666 1	0.756 2	0.556 5	0.997 5	0.513 2

MPA	0.595 3	1.000 1	0.833 2	0.668 3	0.517 1	0.748 3	0.131 4	0.539 8	0.466 1	0.423 3	0.420 5	0.362 3	0.589 3	0.857 5	0.424 6	0.755 3	0.558 4	1.000 1	0.420 4

MTML	0.549 5	1.000 1	0.807 3	0.588 7	0.327 8	0.647 4	0.004 14	0.815 1	0.180 8	0.418 4	0.364 8	0.182 8	0.445 6	1.000 1	0.442 5	0.688 5	0.571 3	1.000 1	0.396 6
Jean Lahoud, Bernard Ghanem, Marc Pollefeys, Martin R. Oswald: 3D Instance Segmentation via Multi-task Metric Learning. ICCV 2019 [oral]
ResNet-backbone	0.459 9	1.000 1	0.737 5	0.159 15	0.259 10	0.587 9	0.138 3	0.475 10	0.217 7	0.416 5	0.408 6	0.128 10	0.315 11	0.714 10	0.411 7	0.536 12	0.590 1	0.873 12	0.304 9

GICN	0.586 4	1.000 1	0.716 6	0.769 2	0.438 5	0.644 5	0.140 2	0.714 2	0.292 5	0.371 6	0.305 9	0.353 4	0.550 4	1.000 1	0.525 3	0.789 1	0.531 7	1.000 1	0.412 5

MASC	0.447 10	0.528 13	0.555 12	0.381 9	0.382 6	0.633 6	0.002 15	0.509 9	0.260 6	0.361 7	0.432 4	0.327 5	0.451 5	0.571 11	0.367 10	0.639 7	0.386 9	0.980 6	0.276 10
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
Occipital-SCS	0.512 6	1.000 1	0.716 6	0.509 8	0.506 2	0.611 7	0.092 7	0.602 6	0.177 9	0.346 8	0.383 7	0.165 9	0.442 7	0.850 8	0.386 9	0.618 8	0.543 6	0.889 10	0.389 7

3D-BoNet	0.488 7	1.000 1	0.672 10	0.590 6	0.301 9	0.484 12	0.098 6	0.620 4	0.306 4	0.341 9	0.259 11	0.125 11	0.434 8	0.796 9	0.402 8	0.499 13	0.513 8	0.909 9	0.439 3
Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, Niki Trigoni: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds. NeurIPS 2019 Spotlight
3D-SIS	0.382 11	1.000 1	0.432 13	0.245 12	0.190 12	0.577 10	0.013 12	0.263 12	0.033 15	0.320 10	0.240 12	0.075 13	0.422 9	0.857 5	0.117 14	0.699 4	0.271 13	0.883 11	0.235 12
Ji Hou, Angela Dai, Matthias Niessner: 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans. CVPR 2019
R-PointNet	0.306 13	0.500 14	0.405 14	0.311 10	0.348 7	0.589 8	0.054 8	0.068 15	0.126 11	0.283 11	0.290 10	0.028 14	0.219 12	0.214 15	0.331 11	0.396 14	0.275 12	0.821 14	0.245 11

PanopticFusion-inst	0.478 8	0.667 10	0.712 9	0.595 5	0.259 11	0.550 11	0.000 17	0.613 5	0.175 10	0.250 12	0.434 3	0.437 2	0.411 10	0.857 5	0.485 4	0.591 11	0.267 14	0.944 7	0.359 8
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
UNet-backbone	0.319 12	0.667 10	0.715 8	0.233 13	0.189 13	0.479 13	0.008 13	0.218 13	0.067 14	0.201 13	0.173 13	0.107 12	0.123 13	0.438 12	0.150 12	0.615 9	0.355 10	0.916 8	0.093 16

Seg-Cluster	0.215 15	0.370 15	0.337 16	0.285 11	0.105 14	0.325 15	0.025 11	0.282 11	0.085 13	0.105 14	0.107 14	0.007 17	0.079 15	0.317 14	0.114 15	0.309 16	0.304 11	0.587 15	0.123 15

3D-BEVIS	0.248 14	0.667 10	0.566 11	0.076 16	0.035 17	0.394 14	0.027 10	0.035 16	0.098 12	0.099 15	0.030 16	0.025 15	0.098 14	0.375 13	0.126 13	0.604 10	0.181 15	0.854 13	0.171 13
Cathrin Elich, Francis Engelmann, Jonas Schult, Theodora Kontogianni, Bastian Leibe: 3D-BEVIS: Birds-Eye-View Instance Segmentation.
Sgpn_scannet	0.143 16	0.208 17	0.390 15	0.169 14	0.065 15	0.275 16	0.029 9	0.069 14	0.000 16	0.087 16	0.043 15	0.014 16	0.027 17	0.000 16	0.112 16	0.351 15	0.168 16	0.438 16	0.138 14

MaskRCNN 2d->3d Proj	0.058 17	0.333 16	0.002 17	0.000 17	0.053 16	0.002 17	0.002 16	0.021 17	0.000 16	0.045 17	0.024 17	0.238 7	0.065 16	0.000 16	0.014 17	0.107 17	0.020 17	0.110 17	0.006 17

This table lists the benchmark results for the 2D semantic label scenario.

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

DMMF_3d	0.605 1	0.651 3	0.744 4	0.782 1	0.637 1	0.387 1	0.536 1	0.732 1	0.590 2	0.540 1	0.856 8	0.359 3	0.306 7	0.596 3	0.539 1	0.627 7	0.706 1	0.497 3	0.785 8	0.757 7	0.476 8

RFBNet	0.592 4	0.616 4	0.758 1	0.659 4	0.581 4	0.330 6	0.469 4	0.655 7	0.543 6	0.524 2	0.924 1	0.355 4	0.336 5	0.572 5	0.479 5	0.671 4	0.648 3	0.480 4	0.814 6	0.814 1	0.614 2

DCRedNet	0.583 5	0.682 2	0.723 5	0.542 7	0.510 7	0.310 8	0.451 5	0.668 5	0.549 5	0.520 3	0.920 2	0.375 1	0.446 1	0.528 7	0.417 6	0.670 5	0.577 9	0.478 5	0.862 2	0.806 2	0.628 1

SSMA	0.577 6	0.695 1	0.716 7	0.439 9	0.563 5	0.314 7	0.444 6	0.719 2	0.551 4	0.503 4	0.887 7	0.346 6	0.348 4	0.603 1	0.353 8	0.709 1	0.600 7	0.457 7	0.901 1	0.786 3	0.599 3
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. International Journal of Computer Vision, 2019
MCA-Net	0.595 3	0.533 8	0.756 2	0.746 3	0.590 2	0.334 5	0.506 2	0.670 4	0.587 3	0.500 5	0.905 5	0.366 2	0.352 3	0.601 2	0.506 4	0.669 6	0.648 3	0.501 2	0.839 4	0.769 6	0.516 7

FuseNet	0.535 7	0.570 6	0.681 8	0.182 12	0.512 6	0.290 9	0.431 7	0.659 6	0.504 8	0.495 6	0.903 6	0.308 7	0.428 2	0.523 8	0.365 7	0.676 2	0.621 6	0.470 6	0.762 9	0.779 4	0.541 5
Caner Hazirbas, Lingni Ma, Csaba Domokos, Daniel Cremers: FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-based CNN Architecture. ACCV 2016
DMMF	0.597 2	0.543 7	0.755 3	0.749 2	0.585 3	0.338 3	0.494 3	0.704 3	0.598 1	0.494 7	0.911 3	0.347 5	0.327 6	0.593 4	0.527 2	0.675 3	0.646 5	0.513 1	0.842 3	0.774 5	0.527 6

ILC-PSPNet	0.475 10	0.490 9	0.581 10	0.289 11	0.507 8	0.067 12	0.379 9	0.610 9	0.417 11	0.435 8	0.822 11	0.278 8	0.267 8	0.503 9	0.228 9	0.616 9	0.533 10	0.375 10	0.820 5	0.729 8	0.560 4

3DMV (2d proj)	0.498 9	0.481 10	0.612 9	0.579 6	0.456 9	0.343 2	0.384 8	0.623 8	0.525 7	0.381 9	0.845 9	0.254 9	0.264 9	0.557 6	0.182 10	0.581 10	0.598 8	0.429 9	0.760 10	0.661 11	0.446 10
Angela Dai, Matthias Niessner: 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation. ECCV'18
AdapNet++	0.503 8	0.613 5	0.722 6	0.418 10	0.358 12	0.337 4	0.370 10	0.479 10	0.443 9	0.368 10	0.907 4	0.207 10	0.213 11	0.464 10	0.525 3	0.618 8	0.657 2	0.450 8	0.788 7	0.721 9	0.408 11
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. International Journal of Computer Vision, 2019
Enet (reimpl)	0.376 11	0.264 12	0.452 12	0.452 8	0.365 10	0.181 10	0.143 12	0.456 11	0.409 12	0.346 11	0.769 12	0.164 11	0.218 10	0.359 11	0.123 12	0.403 12	0.381 12	0.313 12	0.571 11	0.685 10	0.472 9
Re-implementation of Adam Paszke, Abhishek Chaurasia, Sangpil Kim, Eugenio Culurciello: ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation.
ScanNet (2d proj)	0.330 12	0.293 11	0.521 11	0.657 5	0.361 11	0.161 11	0.250 11	0.004 12	0.440 10	0.183 12	0.836 10	0.125 12	0.060 12	0.319 12	0.132 11	0.417 11	0.412 11	0.344 11	0.541 12	0.427 12	0.109 12
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nießner: ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. CVPR'17

This table lists the benchmark results for the 2D semantic instance scenario.

Method	avg ap	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	window

MaskRCNN_ScanNet	0.119 1	0.129 1	0.212 1	0.002 1	0.112 1	0.148 1	0.014 1	0.205 1	0.044 1	0.066 1	0.078 1	0.095 1	0.142 1	0.030 1	0.128 1	0.139 1	0.080 1	0.459 1	0.057 1
Re-implementation of Kaiming He, Georgia Gkioxari, Piotr Dollár, Ross Girshick: Mask R-CNN. ICCV'17

This table lists the benchmark results for the scene type classification scenario.

Method	avg recall	apartment	bathroom	bedroom / hotel	bookstore / library	conference room	copy/mail room	hallway	kitchen	laundry room	living room / lounge	misc	office	storage / basement / garage

MTL	0.700 1	0.500 1	1.000 1	0.882 2	0.500 1	1.000 1	1.000 1	0.500 1	1.000 1	1.000 1	0.778 1	0.000 2	0.938 1	0.000 1

SE-ResNeXt-SSMA	0.498 2	0.000 3	0.812 2	0.941 1	0.500 1	0.500 2	0.500 2	0.500 1	0.429 3	0.500 2	0.667 2	0.500 1	0.625 2	0.000 1
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. arXiv
resnet50_scannet	0.353 3	0.250 2	0.812 2	0.529 3	0.500 1	0.500 2	0.000 3	0.500 1	0.571 2	0.000 3	0.556 3	0.000 2	0.375 3	0.000 1

3D Semantic label benchmark

3D Semantic instance benchmark

2D Semantic label benchmark

2D Semantic instance benchmark

Scene type classification benchmark