Benchmark Results - ScanNet Benchmark

This table lists the benchmark results for the 3D semantic label scenario.

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

BPNet	0.749 2	0.909 1	0.818 3	0.811 7	0.752 1	0.839 5	0.485 11	0.842 6	0.673 3	0.644 3	0.957 2	0.528 5	0.305 7	0.773 2	0.859 3	0.788 2	0.818 3	0.693 2	0.916 5	0.856 4	0.723 4

OccuSeg+Semantic	0.764 1	0.758 22	0.796 6	0.839 3	0.746 2	0.907 1	0.562 1	0.850 4	0.680 2	0.672 1	0.978 1	0.610 1	0.335 2	0.777 1	0.819 11	0.847 1	0.830 1	0.691 3	0.972 1	0.885 1	0.727 2

MatchingNet	0.724 6	0.812 14	0.812 5	0.810 8	0.735 3	0.834 6	0.495 9	0.860 2	0.572 18	0.602 8	0.954 4	0.512 7	0.280 13	0.757 3	0.845 9	0.725 5	0.780 7	0.606 15	0.937 2	0.851 6	0.700 6

SparseConvNet	0.725 5	0.647 37	0.821 1	0.846 2	0.721 4	0.869 2	0.533 2	0.754 15	0.603 13	0.614 5	0.955 3	0.572 3	0.325 4	0.710 6	0.870 2	0.724 6	0.823 2	0.628 8	0.934 4	0.865 3	0.683 7

MinkowskiNet	0.736 4	0.859 5	0.818 3	0.832 4	0.709 5	0.840 4	0.521 4	0.853 3	0.660 4	0.643 4	0.951 8	0.544 4	0.286 12	0.731 5	0.893 1	0.675 14	0.772 9	0.683 4	0.874 19	0.852 5	0.727 2
C. Choy, J. Gwak, S. Savarese: 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks. CVPR 2019
Virtual MVFusion	0.746 3	0.771 18	0.819 2	0.848 1	0.702 6	0.865 3	0.397 34	0.899 1	0.699 1	0.664 2	0.948 18	0.588 2	0.330 3	0.746 4	0.851 6	0.764 3	0.796 5	0.704 1	0.935 3	0.866 2	0.728 1
Abhijit Kundu, Xiaoqi Yin, Alireza Fathi, David Ross, Brian Brewington, Thomas Funkhouser, Caroline Pantofaru: Virtual Multi-view Fusion for 3D Semantic Segmentation. ECCV 2020
CU-Hybrid Net	0.693 9	0.596 42	0.789 7	0.803 10	0.677 7	0.800 19	0.469 15	0.846 5	0.554 26	0.591 10	0.948 18	0.500 8	0.316 6	0.609 14	0.847 8	0.732 4	0.808 4	0.593 16	0.894 10	0.839 7	0.652 10

RFCR	0.702 7	0.889 2	0.745 16	0.813 6	0.672 8	0.818 12	0.493 10	0.815 7	0.623 8	0.610 6	0.947 21	0.470 15	0.249 20	0.594 16	0.848 7	0.705 9	0.779 8	0.646 7	0.892 12	0.823 9	0.611 17

JSENet	0.699 8	0.881 3	0.762 12	0.821 5	0.667 9	0.800 19	0.522 3	0.792 9	0.613 9	0.607 7	0.935 34	0.492 10	0.205 28	0.576 22	0.853 5	0.691 10	0.758 12	0.652 6	0.872 22	0.828 8	0.649 11
Zeyu HU, Mingmin Zhen, Xuyang BAI, Hongbo Fu, Chiew-lan Tai: JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds. ECCV 2020
FusionNet	0.688 10	0.704 30	0.741 19	0.754 18	0.656 10	0.829 8	0.501 7	0.741 17	0.609 11	0.548 14	0.950 12	0.522 6	0.371 1	0.633 11	0.756 15	0.715 7	0.771 10	0.623 9	0.861 28	0.814 11	0.658 9
Feihu Zhang, Jin Fang, Benjamin Wah, Philip Torr: Deep FusionNet for Point Cloud Semantic Segmentation. ECCV 2020
PointASNL	0.666 13	0.703 31	0.781 8	0.751 20	0.655 11	0.830 7	0.471 14	0.769 12	0.474 39	0.537 16	0.951 8	0.475 13	0.279 14	0.635 9	0.698 24	0.675 14	0.751 14	0.553 27	0.816 36	0.806 14	0.703 5
Xu Yan, Chaoda Zheng, Zhen Li, Sheng Wang, Shuguang Cui: PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Sampling. CVPR 2020
SALANet	0.670 12	0.816 13	0.770 11	0.768 15	0.652 12	0.807 16	0.451 19	0.747 16	0.659 5	0.545 15	0.924 40	0.473 14	0.149 44	0.571 24	0.811 12	0.635 24	0.746 15	0.623 9	0.892 12	0.794 23	0.570 31

KP-FCNN	0.684 11	0.847 7	0.758 15	0.784 12	0.647 13	0.814 13	0.473 13	0.772 11	0.605 12	0.594 9	0.935 34	0.450 22	0.181 36	0.587 17	0.805 13	0.690 11	0.785 6	0.614 11	0.882 15	0.819 10	0.632 14
H. Thomas, C. Qi, J. Deschaud, B. Marcotegui, F. Goulette, L. Guibas.: KPConv: Flexible and Deformable Convolution for Point Clouds. ICCV 2019
PointConv	0.666 13	0.781 16	0.759 14	0.699 23	0.644 14	0.822 10	0.475 12	0.779 10	0.564 22	0.504 26	0.953 5	0.428 30	0.203 30	0.586 19	0.754 16	0.661 18	0.753 13	0.588 17	0.902 7	0.813 13	0.642 12
Wenxuan Wu, Zhongang Qi, Li Fuxin: PointConv: Deep Convolutional Networks on 3D Point Clouds. CVPR 2019
joint point-based	0.634 23	0.614 40	0.778 9	0.667 34	0.633 15	0.825 9	0.420 28	0.804 8	0.467 41	0.561 12	0.951 8	0.494 9	0.291 9	0.566 25	0.458 37	0.579 37	0.764 11	0.559 25	0.838 32	0.814 11	0.598 23
Hung-Yueh Chiang, Yen-Liang Lin, Yueh-Cheng Liu, Winston H. Hsu: A Unified Point-Based Framework for 3D Segmentation. 3DV 2019
DCM-Net	0.658 15	0.778 17	0.702 26	0.806 9	0.619 16	0.813 14	0.468 16	0.693 26	0.494 36	0.524 21	0.941 28	0.449 23	0.298 8	0.510 31	0.821 10	0.675 14	0.727 19	0.568 21	0.826 33	0.803 16	0.637 13
Jonas Schult, Francis Engelmann, Theodora Kontogianni, Bastian Leibe: DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshes. CVPR 2020 [Oral]
Supervoxel-CNN	0.635 22	0.656 35	0.711 24	0.719 21	0.613 17	0.757 35	0.444 24	0.765 13	0.534 28	0.566 11	0.928 38	0.478 12	0.272 15	0.636 8	0.531 33	0.664 17	0.645 37	0.508 38	0.864 27	0.792 25	0.611 17

3DSM_DMMF	0.631 26	0.626 39	0.745 16	0.801 11	0.607 18	0.751 36	0.506 5	0.729 20	0.565 21	0.491 29	0.866 51	0.434 26	0.197 32	0.595 15	0.630 29	0.709 8	0.705 23	0.560 24	0.875 18	0.740 41	0.491 43

FPConv	0.639 19	0.785 15	0.760 13	0.713 22	0.603 19	0.798 21	0.392 36	0.534 44	0.603 13	0.524 21	0.948 18	0.457 19	0.250 19	0.538 27	0.723 17	0.598 33	0.696 26	0.614 11	0.872 22	0.799 18	0.567 33
Yiqun Lin, Zizheng Yan, Haibin Huang, Dong Du, Ligang Liu, Shuguang Cui, Xiaoguang Han: FPConv: Learning Local Flattening for Point Convolution. CVPR 2020
HPEIN	0.618 32	0.729 26	0.668 38	0.647 37	0.597 20	0.766 33	0.414 29	0.680 28	0.520 31	0.525 20	0.946 23	0.432 27	0.215 26	0.493 37	0.599 31	0.638 22	0.617 42	0.570 19	0.897 8	0.806 14	0.605 21
Li Jiang, Hengshuang Zhao, Shu Liu, Xiaoyong Shen, Chi-Wing Fu, Jiaya Jia: Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentation. ICCV 2019
VACNN++	0.638 20	0.820 12	0.701 27	0.687 27	0.594 21	0.791 26	0.430 27	0.587 39	0.569 20	0.529 18	0.950 12	0.467 16	0.253 18	0.524 29	0.722 18	0.618 28	0.694 27	0.570 19	0.793 40	0.802 17	0.659 8

PointMTL	0.632 25	0.731 25	0.688 34	0.675 30	0.591 22	0.784 28	0.444 24	0.565 42	0.610 10	0.492 28	0.949 15	0.456 20	0.254 17	0.587 17	0.706 21	0.599 32	0.665 34	0.612 14	0.868 26	0.791 27	0.579 28

FusionAwareConv	0.630 28	0.604 41	0.741 19	0.766 16	0.590 23	0.747 37	0.501 7	0.734 18	0.503 34	0.527 19	0.919 44	0.454 21	0.323 5	0.550 26	0.420 41	0.678 13	0.688 29	0.544 30	0.896 9	0.795 21	0.627 15
Jiazhao Zhang, Chenyang Zhu, Lintao Zheng, Kai Xu: Fusion-Aware Point Convolution for Online Semantic 3D Scene Segmentation. CVPR 2020
MVPNet	0.641 17	0.831 9	0.715 23	0.671 32	0.590 23	0.781 29	0.394 35	0.679 29	0.642 6	0.553 13	0.937 32	0.462 18	0.256 16	0.649 7	0.406 42	0.626 25	0.691 28	0.666 5	0.877 16	0.792 25	0.608 20
Maximilian Jaritz, Jiayuan Gu, Hao Su: Multi-view PointNet for 3D Scene Understanding. GMDL Workshop, ICCV 2019
DVVNet	0.562 40	0.648 36	0.700 29	0.770 14	0.586 25	0.687 44	0.333 42	0.650 34	0.514 33	0.475 34	0.906 48	0.359 40	0.223 25	0.340 45	0.442 40	0.422 48	0.668 33	0.501 39	0.708 46	0.779 30	0.534 38

PointMRNet-lite	0.625 29	0.643 38	0.711 24	0.697 25	0.581 26	0.801 17	0.408 31	0.670 31	0.558 25	0.497 27	0.944 26	0.436 25	0.152 43	0.617 12	0.708 20	0.603 30	0.743 16	0.532 36	0.870 25	0.784 28	0.545 35

MCCNN	0.633 24	0.866 4	0.731 21	0.771 13	0.576 27	0.809 15	0.410 30	0.684 27	0.497 35	0.491 29	0.949 15	0.466 17	0.105 49	0.581 20	0.646 28	0.620 26	0.680 31	0.542 32	0.817 35	0.795 21	0.618 16
P. Hermosilla, T. Ritschel, P.P. Vazquez, A. Vinacua, T. Ropinski: Monte Carlo Convolution for Learning on Non-Uniformly Sampled Point Clouds. SIGGRAPH Asia 2018
PointMRNet	0.640 18	0.717 29	0.701 27	0.692 26	0.576 27	0.801 17	0.467 17	0.716 22	0.563 23	0.459 35	0.953 5	0.429 29	0.169 38	0.581 20	0.854 4	0.605 29	0.710 21	0.550 28	0.894 10	0.793 24	0.575 29

SConv	0.636 21	0.830 10	0.697 31	0.752 19	0.572 29	0.780 30	0.445 22	0.716 22	0.529 29	0.530 17	0.951 8	0.446 24	0.170 37	0.507 33	0.666 27	0.636 23	0.682 30	0.541 33	0.886 14	0.799 18	0.594 25

HPGCNN	0.656 16	0.698 32	0.743 18	0.650 35	0.564 30	0.820 11	0.505 6	0.758 14	0.631 7	0.479 32	0.945 25	0.480 11	0.226 23	0.572 23	0.774 14	0.690 11	0.735 17	0.614 11	0.853 31	0.776 32	0.597 24
Jisheng Dang, Qingyong Hu, Yulan Guo, Jun Yang: HPGCNN.
SIConv	0.625 29	0.830 10	0.694 32	0.757 17	0.563 31	0.772 32	0.448 20	0.647 36	0.520 31	0.509 23	0.949 15	0.431 28	0.191 33	0.496 36	0.614 30	0.647 21	0.672 32	0.535 35	0.876 17	0.783 29	0.571 30

APCF-Net	0.631 26	0.742 23	0.687 36	0.672 31	0.557 32	0.792 24	0.408 31	0.665 32	0.545 27	0.508 24	0.952 7	0.428 30	0.186 34	0.634 10	0.702 22	0.620 26	0.706 22	0.555 26	0.873 21	0.798 20	0.581 27

PointSPNet	0.625 29	0.770 19	0.684 37	0.698 24	0.552 33	0.793 22	0.434 26	0.662 33	0.560 24	0.457 36	0.950 12	0.416 32	0.130 46	0.610 13	0.682 25	0.648 19	0.734 18	0.544 30	0.857 30	0.779 30	0.541 36

SPH3D-GCN	0.610 33	0.858 6	0.772 10	0.489 48	0.532 34	0.792 24	0.404 33	0.643 37	0.570 19	0.507 25	0.935 34	0.414 33	0.046 54	0.510 31	0.702 22	0.602 31	0.705 23	0.549 29	0.859 29	0.773 33	0.534 38
Huan Lei, Naveed Akhtar, and Ajmal Mian: Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds. TPAMI 2020
AttAN	0.609 34	0.760 21	0.667 39	0.649 36	0.521 35	0.793 22	0.457 18	0.648 35	0.528 30	0.434 40	0.947 21	0.401 34	0.153 42	0.454 38	0.721 19	0.648 19	0.717 20	0.536 34	0.904 6	0.765 35	0.485 44
Gege Zhang, Qinghua Ma, Licheng Jiao, Fang Liu and Qigong Sun: AttAN: Attention Adversarial Networks for 3D Point Cloud Semantic Segmentation. IJCAI2020
SegGCN	0.589 37	0.833 8	0.731 21	0.539 46	0.514 36	0.789 27	0.448 20	0.467 45	0.573 17	0.484 31	0.936 33	0.396 35	0.061 53	0.501 34	0.507 36	0.594 34	0.700 25	0.563 23	0.874 19	0.771 34	0.493 42
Huan Lei, Naveed Akhtar, and Ajmal Mian: SegGCN: Efficient 3D Point Cloud Segmentation with Fuzzy Spherical Kernel. CVPR 2020
TextureNet	0.566 39	0.672 34	0.664 40	0.671 32	0.494 37	0.719 40	0.445 22	0.678 30	0.411 46	0.396 41	0.935 34	0.356 41	0.225 24	0.412 42	0.535 32	0.565 39	0.636 41	0.464 43	0.794 39	0.680 46	0.568 32
Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkerhouser, Matthias Niessner, Leonidas Guibas: TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshes. CVPR
Pointnet++ & Feature	0.557 41	0.735 24	0.661 41	0.686 28	0.491 38	0.744 38	0.392 36	0.539 43	0.451 42	0.375 44	0.946 23	0.376 38	0.205 28	0.403 43	0.356 44	0.553 40	0.643 38	0.497 40	0.824 34	0.756 37	0.515 40

DPC	0.592 36	0.720 27	0.700 29	0.602 42	0.480 39	0.762 34	0.380 40	0.713 24	0.585 16	0.437 39	0.940 30	0.369 39	0.288 10	0.434 40	0.509 35	0.590 36	0.639 40	0.567 22	0.772 42	0.755 38	0.592 26
Francis Engelmann, Theodora Kontogianni, Bastian Leibe: Dilated Point Convolutions: On the Receptive Field Size of Point Convolutions on 3D Point Clouds. ICRA 2020
3DMV, FTSDF	0.501 43	0.558 46	0.608 48	0.424 53	0.478 40	0.690 43	0.246 49	0.586 40	0.468 40	0.450 37	0.911 46	0.394 36	0.160 40	0.438 39	0.212 50	0.432 47	0.541 48	0.475 42	0.742 44	0.727 42	0.477 45

CCRFNet	0.589 37	0.766 20	0.659 42	0.683 29	0.470 41	0.740 39	0.387 39	0.620 38	0.490 37	0.476 33	0.922 42	0.355 42	0.245 21	0.511 30	0.511 34	0.571 38	0.643 38	0.493 41	0.872 22	0.762 36	0.600 22

LAP-D	0.594 35	0.720 27	0.692 33	0.637 39	0.456 42	0.773 31	0.391 38	0.730 19	0.587 15	0.445 38	0.940 30	0.381 37	0.288 10	0.434 40	0.453 38	0.591 35	0.649 35	0.581 18	0.777 41	0.749 40	0.610 19

3DMV	0.484 45	0.484 50	0.538 51	0.643 38	0.424 43	0.606 52	0.310 43	0.574 41	0.433 45	0.378 43	0.796 53	0.301 45	0.214 27	0.537 28	0.208 51	0.472 46	0.507 52	0.413 50	0.693 47	0.602 52	0.539 37
Angela Dai, Matthias Niessner: 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation. ECCV'18
PCNN	0.498 44	0.559 45	0.644 45	0.560 45	0.420 44	0.711 42	0.229 51	0.414 46	0.436 43	0.352 45	0.941 28	0.324 44	0.155 41	0.238 50	0.387 43	0.493 42	0.529 49	0.509 37	0.813 37	0.751 39	0.504 41

PanopticFusion-label	0.529 42	0.491 49	0.688 34	0.604 41	0.386 45	0.632 49	0.225 53	0.705 25	0.434 44	0.293 48	0.815 52	0.348 43	0.241 22	0.499 35	0.669 26	0.507 41	0.649 35	0.442 47	0.796 38	0.602 52	0.561 34
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
FCPN	0.447 47	0.679 33	0.604 49	0.578 44	0.380 46	0.682 45	0.291 46	0.106 55	0.483 38	0.258 53	0.920 43	0.258 49	0.025 55	0.231 52	0.325 45	0.480 45	0.560 46	0.463 44	0.725 45	0.666 48	0.231 55
Dario Rethage, Johanna Wald, Jürgen Sturm, Nassir Navab, Federico Tombari: Fully-Convolutional Point Networks for Large-Scale Point Clouds. ECCV 2018
Tangent Convolutions	0.438 50	0.437 53	0.646 44	0.474 50	0.369 47	0.645 48	0.353 41	0.258 52	0.282 54	0.279 49	0.918 45	0.298 46	0.147 45	0.283 47	0.294 46	0.487 43	0.562 45	0.427 49	0.619 50	0.633 49	0.352 50
Maxim Tatarchenko, Jaesik Park, Vladlen Koltun, Qian-Yi Zhou: Tangent convolutions for dense prediction in 3d. CVPR 2018
PNET2	0.442 48	0.548 47	0.548 50	0.597 43	0.363 48	0.628 50	0.300 44	0.292 50	0.374 49	0.307 47	0.881 50	0.268 48	0.186 34	0.238 50	0.204 52	0.407 49	0.506 53	0.449 46	0.667 48	0.620 50	0.462 47

ScanNet+FTSDF	0.383 53	0.297 55	0.491 53	0.432 52	0.358 49	0.612 51	0.274 47	0.116 54	0.411 46	0.265 51	0.904 49	0.229 52	0.079 51	0.250 48	0.185 53	0.320 53	0.510 50	0.385 51	0.548 52	0.597 54	0.394 49

SurfaceConvPF	0.442 48	0.505 48	0.622 46	0.380 54	0.342 50	0.654 47	0.227 52	0.397 48	0.367 50	0.276 50	0.924 40	0.240 51	0.198 31	0.359 44	0.262 47	0.366 50	0.581 44	0.435 48	0.640 49	0.668 47	0.398 48
Hao Pan, Shilin Liu, Yang Liu, Xin Tong: Convolutional Neural Networks on 3D Surfaces Using Parallel Frames.
PointCNN with RGB	0.458 46	0.577 44	0.611 47	0.356 55	0.321 51	0.715 41	0.299 45	0.376 49	0.328 52	0.319 46	0.944 26	0.285 47	0.164 39	0.216 53	0.229 49	0.484 44	0.545 47	0.456 45	0.755 43	0.709 43	0.475 46
Yangyan Li, Rui Bu, Mingchao Sun, Baoquan Chen: PointCNN. NeurIPS 2018
SPLAT Net	0.393 52	0.472 52	0.511 52	0.606 40	0.311 52	0.656 46	0.245 50	0.405 47	0.328 52	0.197 54	0.927 39	0.227 53	0.000 57	0.001 57	0.249 48	0.271 55	0.510 50	0.383 52	0.593 51	0.699 44	0.267 53
Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, Jan Kautz: SPLATNet: Sparse Lattice Networks for Point Cloud Processing. CVPR 2018
ScanNet	0.306 56	0.203 56	0.366 55	0.501 47	0.311 52	0.524 54	0.211 54	0.002 57	0.342 51	0.189 55	0.786 54	0.145 56	0.102 50	0.245 49	0.152 54	0.318 54	0.348 55	0.300 55	0.460 55	0.437 56	0.182 56
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nießner: ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. CVPR'17
subcloud_weak	0.411 51	0.479 51	0.650 43	0.475 49	0.285 54	0.519 55	0.087 56	0.725 21	0.396 48	0.386 42	0.621 56	0.250 50	0.117 47	0.338 46	0.443 39	0.188 56	0.594 43	0.369 53	0.377 56	0.616 51	0.306 51

PointNet++	0.339 54	0.584 43	0.478 54	0.458 51	0.256 55	0.360 56	0.250 48	0.247 53	0.278 55	0.261 52	0.677 55	0.183 54	0.117 47	0.212 54	0.145 55	0.364 51	0.346 56	0.232 56	0.548 52	0.523 55	0.252 54
Charles R. Qi, Li Yi, Hao Su, Leonidas J. Guibas: pointnet++: deep hierarchical feature learning on point sets in a metric space.
SSC-UNet	0.308 55	0.353 54	0.290 56	0.278 56	0.166 56	0.553 53	0.169 55	0.286 51	0.147 56	0.148 56	0.908 47	0.182 55	0.064 52	0.023 56	0.018 57	0.354 52	0.363 54	0.345 54	0.546 54	0.685 45	0.278 52

ERROR	0.054 57	0.000 57	0.041 57	0.172 57	0.030 57	0.062 57	0.001 57	0.035 56	0.004 57	0.051 57	0.143 57	0.019 57	0.003 56	0.041 55	0.050 56	0.003 57	0.054 57	0.018 57	0.005 57	0.264 57	0.082 57

This table lists the benchmark results for the 3D semantic instance scenario.

Method	avg ap 50%	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	window

CRNet	0.688 1	1.000 1	0.802 6	0.783 3	0.644 1	0.804 4	0.182 2	0.874 1	0.511 4	0.501 2	0.592 1	0.536 3	0.587 8	1.000 1	0.637 4	0.730 7	0.683 1	0.994 9	0.523 2

Mask-Group	0.664 3	1.000 1	0.822 3	0.764 5	0.616 2	0.815 2	0.139 5	0.694 6	0.597 2	0.459 5	0.566 3	0.599 1	0.600 5	0.516 17	0.715 2	0.819 2	0.635 3	1.000 1	0.603 1

OccuSeg+instance	0.672 2	1.000 1	0.758 10	0.682 6	0.576 3	0.842 1	0.477 1	0.504 14	0.524 3	0.567 1	0.585 2	0.451 5	0.557 9	1.000 1	0.751 1	0.797 3	0.563 7	1.000 1	0.467 7

PE	0.645 4	1.000 1	0.773 7	0.798 2	0.538 4	0.786 8	0.088 11	0.799 3	0.350 9	0.435 9	0.547 5	0.545 2	0.646 4	0.933 6	0.562 6	0.761 5	0.556 11	0.997 7	0.501 5

3D-MPA	0.611 7	1.000 1	0.833 2	0.765 4	0.526 5	0.756 9	0.136 7	0.588 11	0.470 5	0.438 8	0.432 11	0.358 10	0.650 2	0.857 8	0.429 10	0.765 4	0.557 9	1.000 1	0.430 9
Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, Matthias Nießner: 3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation. CVPR 2020
Occipital-SCS	0.512 12	1.000 1	0.716 12	0.509 12	0.506 6	0.611 13	0.092 10	0.602 10	0.177 15	0.346 14	0.383 14	0.165 15	0.442 12	0.850 11	0.386 14	0.618 14	0.543 12	0.889 15	0.389 11

PointGroup	0.636 6	1.000 1	0.765 8	0.624 7	0.505 7	0.797 5	0.116 8	0.696 5	0.384 7	0.441 7	0.559 4	0.476 4	0.596 6	1.000 1	0.666 3	0.756 6	0.556 10	0.997 7	0.513 3
Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi-Wing Fu, Jiaya Jia: PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation. CVPR 2020 [oral]
GICN	0.638 5	1.000 1	0.895 1	0.800 1	0.480 8	0.676 10	0.144 4	0.737 4	0.354 8	0.447 6	0.400 13	0.365 9	0.700 1	1.000 1	0.569 5	0.836 1	0.599 4	1.000 1	0.473 6

SSEN	0.575 9	1.000 1	0.761 9	0.473 13	0.477 9	0.795 6	0.066 12	0.529 12	0.658 1	0.460 4	0.461 8	0.380 8	0.331 16	0.859 7	0.401 13	0.692 10	0.653 2	1.000 1	0.348 13
Dongsu Zhang, Junha Chun, Sang Kyun Cha, Young Min Kim: Spatial Semantic Embedding Network: Fast 3D Instance Segmentation with Deep Metric Learning. Arxiv
Sparse R-CNN	0.515 11	1.000 1	0.538 18	0.282 17	0.468 10	0.790 7	0.173 3	0.345 16	0.429 6	0.413 12	0.484 6	0.176 14	0.595 7	0.591 15	0.522 7	0.668 12	0.476 14	0.986 10	0.327 14

PCJC	0.578 8	1.000 1	0.810 4	0.583 11	0.449 11	0.813 3	0.042 14	0.603 9	0.341 10	0.490 3	0.465 7	0.410 7	0.650 2	0.835 12	0.264 17	0.694 9	0.561 8	0.889 15	0.504 4

MASC	0.447 16	0.528 20	0.555 17	0.381 14	0.382 12	0.633 12	0.002 21	0.509 13	0.260 12	0.361 13	0.432 10	0.327 11	0.451 10	0.571 16	0.367 15	0.639 13	0.386 15	0.980 11	0.276 16
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
R-PointNet	0.306 19	0.500 21	0.405 21	0.311 15	0.348 13	0.589 14	0.054 13	0.068 22	0.126 17	0.283 17	0.290 16	0.028 21	0.219 18	0.214 21	0.331 16	0.396 21	0.275 19	0.821 20	0.245 17

MTML	0.549 10	1.000 1	0.807 5	0.588 10	0.327 14	0.647 11	0.004 20	0.815 2	0.180 14	0.418 10	0.364 15	0.182 13	0.445 11	1.000 1	0.442 9	0.688 11	0.571 6	1.000 1	0.396 10
Jean Lahoud, Bernard Ghanem, Marc Pollefeys, Martin R. Oswald: 3D Instance Segmentation via Multi-task Metric Learning. ICCV 2019 [oral]
3D-BoNet	0.488 13	1.000 1	0.672 15	0.590 9	0.301 15	0.484 19	0.098 9	0.620 7	0.306 11	0.341 15	0.259 17	0.125 17	0.434 13	0.796 13	0.402 12	0.499 19	0.513 13	0.909 14	0.439 8
Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, Niki Trigoni: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds. NeurIPS 2019 Spotlight
SALoss-ResNet	0.459 15	1.000 1	0.737 11	0.159 22	0.259 16	0.587 15	0.138 6	0.475 15	0.217 13	0.416 11	0.408 12	0.128 16	0.315 17	0.714 14	0.411 11	0.536 18	0.590 5	0.873 18	0.304 15
Zhidong Liang, Ming Yang, Hao Li, Chunxiang Wang: 3D Instance Embedding Learning With a Structure-Aware Loss Function for Point Cloud Segmentation. IEEE Robotics and Automation Letters (IROS2020)
PanopticFusion-inst	0.478 14	0.667 16	0.712 14	0.595 8	0.259 17	0.550 17	0.000 23	0.613 8	0.175 16	0.250 18	0.434 9	0.437 6	0.411 15	0.857 8	0.485 8	0.591 17	0.267 21	0.944 12	0.359 12
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
3D-SIS	0.382 17	1.000 1	0.432 20	0.245 18	0.190 18	0.577 16	0.013 18	0.263 18	0.033 22	0.320 16	0.240 18	0.075 19	0.422 14	0.857 8	0.117 20	0.699 8	0.271 20	0.883 17	0.235 18
Ji Hou, Angela Dai, Matthias Niessner: 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans. CVPR 2019
UNet-backbone	0.319 18	0.667 16	0.715 13	0.233 19	0.189 19	0.479 20	0.008 19	0.218 19	0.067 21	0.201 19	0.173 19	0.107 18	0.123 20	0.438 18	0.150 18	0.615 15	0.355 16	0.916 13	0.093 23

Region	0.248 20	0.667 16	0.437 19	0.188 20	0.153 20	0.491 18	0.000 23	0.208 20	0.094 19	0.153 20	0.099 21	0.057 20	0.217 19	0.119 22	0.039 23	0.466 20	0.302 18	0.640 21	0.140 20

Seg-Cluster	0.215 22	0.370 22	0.337 23	0.285 16	0.105 21	0.325 22	0.025 17	0.282 17	0.085 20	0.105 21	0.107 20	0.007 24	0.079 22	0.317 20	0.114 21	0.309 23	0.304 17	0.587 22	0.123 22

Sgpn_scannet	0.143 23	0.208 24	0.390 22	0.169 21	0.065 22	0.275 23	0.029 15	0.069 21	0.000 23	0.087 23	0.043 22	0.014 23	0.027 24	0.000 23	0.112 22	0.351 22	0.168 23	0.438 23	0.138 21

MaskRCNN 2d->3d Proj	0.058 24	0.333 23	0.002 24	0.000 24	0.053 23	0.002 24	0.002 22	0.021 24	0.000 23	0.045 24	0.024 24	0.238 12	0.065 23	0.000 23	0.014 24	0.107 24	0.020 24	0.110 24	0.006 24

3D-BEVIS	0.248 20	0.667 16	0.566 16	0.076 23	0.035 24	0.394 21	0.027 16	0.035 23	0.098 18	0.099 22	0.030 23	0.025 22	0.098 21	0.375 19	0.126 19	0.604 16	0.181 22	0.854 19	0.171 19
Cathrin Elich, Francis Engelmann, Jonas Schult, Theodora Kontogianni, Bastian Leibe: 3D-BEVIS: Birds-Eye-View Instance Segmentation.

This table lists the benchmark results for the 2D semantic label scenario.

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

Virtual MVFusion (R)	0.745 1	0.861 1	0.839 1	0.881 1	0.672 1	0.512 1	0.422 10	0.898 1	0.723 1	0.714 1	0.954 2	0.454 1	0.509 1	0.773 1	0.895 1	0.756 1	0.820 1	0.653 1	0.935 1	0.891 1	0.728 1
Abhijit Kundu, Xiaoqi Yin, Alireza Fathi, David Ross, Brian Brewington, Thomas Funkhouser, Caroline Pantofaru: Virtual Multi-view Fusion for 3D Semantic Segmentation. ECCV 2020
BPNet_2D	0.670 2	0.822 3	0.795 3	0.836 2	0.659 2	0.481 2	0.451 6	0.769 2	0.656 3	0.567 3	0.931 3	0.395 3	0.390 4	0.700 2	0.534 3	0.689 5	0.770 2	0.574 3	0.865 3	0.831 3	0.675 2

CU-Hybrid-2D Net	0.636 3	0.825 2	0.820 2	0.179 16	0.648 3	0.463 3	0.549 1	0.742 3	0.676 2	0.628 2	0.961 1	0.420 2	0.379 5	0.684 3	0.381 10	0.732 2	0.723 3	0.599 2	0.827 8	0.851 2	0.634 3

DMMF_3d	0.605 4	0.651 6	0.744 7	0.782 3	0.637 4	0.387 4	0.536 2	0.732 4	0.590 5	0.540 4	0.856 13	0.359 6	0.306 10	0.596 6	0.539 2	0.627 12	0.706 4	0.497 6	0.785 12	0.757 11	0.476 13

MCA-Net	0.595 6	0.533 12	0.756 5	0.746 5	0.590 5	0.334 8	0.506 3	0.670 7	0.587 6	0.500 8	0.905 8	0.366 5	0.352 6	0.601 5	0.506 6	0.669 10	0.648 6	0.501 5	0.839 7	0.769 9	0.516 12

DMMF	0.597 5	0.543 11	0.755 6	0.749 4	0.585 6	0.338 6	0.494 4	0.704 6	0.598 4	0.494 10	0.911 6	0.347 8	0.327 9	0.593 7	0.527 4	0.675 7	0.646 8	0.513 4	0.842 6	0.774 8	0.527 11

RFBNet	0.592 7	0.616 7	0.758 4	0.659 6	0.581 7	0.330 9	0.469 5	0.655 10	0.543 9	0.524 5	0.924 4	0.355 7	0.336 8	0.572 8	0.479 8	0.671 8	0.648 6	0.480 7	0.814 10	0.814 4	0.614 6

SSMA	0.577 9	0.695 4	0.716 10	0.439 12	0.563 8	0.314 10	0.444 8	0.719 5	0.551 7	0.503 7	0.887 10	0.346 9	0.348 7	0.603 4	0.353 12	0.709 3	0.600 11	0.457 10	0.901 2	0.786 6	0.599 7
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. International Journal of Computer Vision, 2019
SN_RN152pyrx8_RVC	0.546 10	0.572 9	0.663 13	0.638 8	0.518 9	0.298 12	0.366 15	0.633 12	0.510 11	0.446 12	0.864 11	0.296 11	0.267 12	0.542 10	0.346 13	0.704 4	0.575 14	0.431 12	0.853 5	0.766 10	0.630 4

FuseNet	0.535 11	0.570 10	0.681 12	0.182 15	0.512 10	0.290 13	0.431 9	0.659 9	0.504 12	0.495 9	0.903 9	0.308 10	0.428 3	0.523 12	0.365 11	0.676 6	0.621 10	0.470 9	0.762 13	0.779 7	0.541 9
Caner Hazirbas, Lingni Ma, Csaba Domokos, Daniel Cremers: FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-based CNN Architecture. ACCV 2016
DCRedNet	0.583 8	0.682 5	0.723 8	0.542 10	0.510 11	0.310 11	0.451 6	0.668 8	0.549 8	0.520 6	0.920 5	0.375 4	0.446 2	0.528 11	0.417 9	0.670 9	0.577 13	0.478 8	0.862 4	0.806 5	0.628 5

ILC-PSPNet	0.475 15	0.490 14	0.581 15	0.289 14	0.507 12	0.067 17	0.379 13	0.610 14	0.417 15	0.435 13	0.822 16	0.278 12	0.267 12	0.503 13	0.228 14	0.616 14	0.533 15	0.375 14	0.820 9	0.729 12	0.560 8

3DMV (2d proj)	0.498 13	0.481 15	0.612 14	0.579 9	0.456 13	0.343 5	0.384 12	0.623 13	0.525 10	0.381 14	0.845 14	0.254 13	0.264 14	0.557 9	0.182 15	0.581 15	0.598 12	0.429 13	0.760 14	0.661 16	0.446 15
Angela Dai, Matthias Niessner: 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation. ECCV'18
MSeg1080_RVC	0.485 14	0.505 13	0.709 11	0.092 17	0.427 14	0.241 14	0.411 11	0.654 11	0.385 17	0.457 11	0.861 12	0.053 17	0.279 11	0.503 13	0.481 7	0.645 11	0.626 9	0.365 15	0.748 15	0.725 13	0.529 10
John Lambert, Zhuang Liu, Ozan Sener, James Hays, Vladlen Koltun: MSeg: A Composite Dataset for Multi-domain Semantic Segmentation. CVPR 2020
Enet (reimpl)	0.376 16	0.264 17	0.452 17	0.452 11	0.365 15	0.181 15	0.143 17	0.456 16	0.409 16	0.346 16	0.769 17	0.164 15	0.218 15	0.359 16	0.123 17	0.403 17	0.381 17	0.313 17	0.571 16	0.685 15	0.472 14
Re-implementation of Adam Paszke, Abhishek Chaurasia, Sangpil Kim, Eugenio Culurciello: ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation.
ScanNet (2d proj)	0.330 17	0.293 16	0.521 16	0.657 7	0.361 16	0.161 16	0.250 16	0.004 17	0.440 14	0.183 17	0.836 15	0.125 16	0.060 17	0.319 17	0.132 16	0.417 16	0.412 16	0.344 16	0.541 17	0.427 17	0.109 17
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nießner: ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. CVPR'17
AdapNet++	0.503 12	0.613 8	0.722 9	0.418 13	0.358 17	0.337 7	0.370 14	0.479 15	0.443 13	0.368 15	0.907 7	0.207 14	0.213 16	0.464 15	0.525 5	0.618 13	0.657 5	0.450 11	0.788 11	0.721 14	0.408 16
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. International Journal of Computer Vision, 2019

This table lists the benchmark results for the 2D semantic instance scenario.

Method	avg ap	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	window

UniDet_RVC	0.205 1	0.381 1	0.323 1	0.037 1	0.226 1	0.177 1	0.063 1	0.277 1	0.120 1	0.067 1	0.131 1	0.074 2	0.317 1	0.080 1	0.235 1	0.289 1	0.141 1	0.678 1	0.080 1

MaskRCNN_ScanNet	0.119 2	0.129 2	0.212 2	0.002 2	0.112 2	0.148 2	0.014 2	0.205 2	0.044 2	0.066 2	0.078 2	0.095 1	0.142 2	0.030 2	0.128 2	0.139 2	0.080 2	0.459 2	0.057 2
Re-implementation of Kaiming He, Georgia Gkioxari, Piotr Dollár, Ross Girshick: Mask R-CNN. ICCV'17

This table lists the benchmark results for the scene type classification scenario.

Method	avg recall	apartment	bathroom	bedroom / hotel	bookstore / library	conference room	copy/mail room	hallway	kitchen	laundry room	living room / lounge	misc	office	storage / basement / garage

SE-ResNeXt-SSMA	0.498 2	0.000 3	0.812 2	0.941 1	0.500 1	0.500 2	0.500 2	0.500 1	0.429 3	0.500 2	0.667 2	0.500 1	0.625 2	0.000 1
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. arXiv
multi-task	0.700 1	0.500 1	1.000 1	0.882 2	0.500 1	1.000 1	1.000 1	0.500 1	1.000 1	1.000 1	0.778 1	0.000 2	0.938 1	0.000 1
Shengyu Huang, Mikhail Usvyatsov, Konrad Schindler: Indoor Scene Recognition in 3D. IROS 2020
resnet50_scannet	0.353 3	0.250 2	0.812 2	0.529 3	0.500 1	0.500 2	0.000 3	0.500 1	0.571 2	0.000 3	0.556 3	0.000 2	0.375 3	0.000 1

3D Semantic label benchmark

3D Semantic instance benchmark

2D Semantic label benchmark

2D Semantic instance benchmark

Scene type classification benchmark