Benchmark Results - ScanNet Benchmark

This table lists the benchmark results for the 3D semantic label scenario.

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

Virtual MVFusion	0.746 3	0.771 15	0.819 2	0.848 1	0.702 6	0.865 3	0.397 28	0.899 1	0.699 1	0.664 2	0.948 16	0.588 2	0.330 3	0.746 4	0.851 6	0.764 3	0.796 5	0.704 1	0.935 3	0.866 2	0.728 1
Abhijit Kundu, Xiaoqi Yin, Alireza Fathi, David Ross, Brian Brewington, Thomas Funkhouser, Caroline Pantofaru: Virtual Multi-view Fusion for 3D Semantic Segmentation. ECCV 2020
OccuSeg+Semantic	0.764 1	0.758 18	0.796 6	0.839 3	0.746 2	0.907 1	0.562 1	0.850 4	0.680 2	0.672 1	0.978 1	0.610 1	0.335 2	0.777 1	0.819 10	0.847 1	0.830 1	0.691 3	0.972 1	0.885 1	0.727 2

MinkowskiNet	0.736 4	0.859 4	0.818 3	0.832 4	0.709 5	0.840 4	0.521 4	0.853 3	0.660 4	0.643 4	0.951 8	0.544 4	0.286 12	0.731 5	0.893 1	0.675 12	0.772 8	0.683 4	0.874 17	0.852 5	0.727 2
C. Choy, J. Gwak, S. Savarese: 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks. CVPR 2019
BPNet	0.749 2	0.909 1	0.818 3	0.811 6	0.752 1	0.839 5	0.485 9	0.842 6	0.673 3	0.644 3	0.957 2	0.528 5	0.305 7	0.773 2	0.859 3	0.788 2	0.818 3	0.693 2	0.916 5	0.856 4	0.723 4

PointASNL	0.666 11	0.703 27	0.781 8	0.751 18	0.655 10	0.830 7	0.471 12	0.769 11	0.474 33	0.537 13	0.951 8	0.475 11	0.279 14	0.635 8	0.698 20	0.675 12	0.751 13	0.553 23	0.816 31	0.806 13	0.703 5
Xu Yan, Chaoda Zheng, Zhen Li, Sheng Wang, Shuguang Cui: PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Sampling. CVPR 2020
MatchingNet	0.724 6	0.812 11	0.812 5	0.810 7	0.735 3	0.834 6	0.495 8	0.860 2	0.572 15	0.602 7	0.954 4	0.512 7	0.280 13	0.757 3	0.845 8	0.725 5	0.780 7	0.606 12	0.937 2	0.851 6	0.700 6

SparseConvNet	0.725 5	0.647 31	0.821 1	0.846 2	0.721 4	0.869 2	0.533 2	0.754 12	0.603 10	0.614 5	0.955 3	0.572 3	0.325 4	0.710 6	0.870 2	0.724 6	0.823 2	0.628 7	0.934 4	0.865 3	0.683 7

FusionNet	0.688 9	0.704 26	0.741 16	0.754 16	0.656 9	0.829 8	0.501 6	0.741 13	0.609 8	0.548 12	0.950 12	0.522 6	0.371 1	0.633 10	0.756 12	0.715 7	0.771 9	0.623 8	0.861 25	0.814 10	0.658 8
Feihu Zhang, Jin Fang, Benjamin Wah, Philip Torr: Deep FusionNet for Point Cloud Semantic Segmentation. ECCV 2020
CU-Hybrid Net	0.693 8	0.596 36	0.789 7	0.803 9	0.677 7	0.800 16	0.469 13	0.846 5	0.554 21	0.591 9	0.948 16	0.500 8	0.316 6	0.609 12	0.847 7	0.732 4	0.808 4	0.593 13	0.894 10	0.839 7	0.652 9

JSENet	0.699 7	0.881 2	0.762 11	0.821 5	0.667 8	0.800 16	0.522 3	0.792 8	0.613 6	0.607 6	0.935 30	0.492 10	0.205 24	0.576 19	0.853 5	0.691 9	0.758 11	0.652 6	0.872 20	0.828 8	0.649 10
Zeyu HU, Mingmin Zhen, Xuyang BAI, Hongbo Fu, Chiew-lan Tai: JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds. ECCV 2020
PointConv	0.666 11	0.781 13	0.759 13	0.699 20	0.644 12	0.822 10	0.475 10	0.779 9	0.564 18	0.504 22	0.953 5	0.428 25	0.203 26	0.586 16	0.754 13	0.661 15	0.753 12	0.588 14	0.902 7	0.813 12	0.642 11
Wenxuan Wu, Zhongang Qi, Li Fuxin: PointConv: Deep Convolutional Networks on 3D Point Clouds. CVPR 2019
DCM-Net	0.658 13	0.778 14	0.702 22	0.806 8	0.619 14	0.813 12	0.468 14	0.693 22	0.494 30	0.524 17	0.941 24	0.449 18	0.298 8	0.510 25	0.821 9	0.675 12	0.727 15	0.568 17	0.826 28	0.803 15	0.637 12
Jonas Schult, Francis Engelmann, Theodora Kontogianni, Bastian Leibe: DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshes. CVPR 2020 [Oral]
KP-FCNN	0.684 10	0.847 6	0.758 14	0.784 11	0.647 11	0.814 11	0.473 11	0.772 10	0.605 9	0.594 8	0.935 30	0.450 17	0.181 32	0.587 14	0.805 11	0.690 10	0.785 6	0.614 9	0.882 13	0.819 9	0.632 13
H. Thomas, C. Qi, J. Deschaud, B. Marcotegui, F. Goulette, L. Guibas.: KPConv: Flexible and Deformable Convolution for Point Clouds. ICCV 2019
FusionAwareConv	0.630 23	0.604 35	0.741 16	0.766 14	0.590 19	0.747 31	0.501 6	0.734 14	0.503 28	0.527 15	0.919 38	0.454 16	0.323 5	0.550 21	0.420 35	0.678 11	0.688 24	0.544 26	0.896 9	0.795 19	0.627 14
Jiazhao Zhang, Chenyang Zhu, Lintao Zheng, Kai Xu: Fusion-Aware Point Convolution for Online Semantic 3D Scene Segmentation. CVPR 2020
MCCNN	0.633 19	0.866 3	0.731 18	0.771 12	0.576 23	0.809 13	0.410 24	0.684 23	0.497 29	0.491 25	0.949 13	0.466 12	0.105 43	0.581 17	0.646 23	0.620 21	0.680 26	0.542 27	0.817 30	0.795 19	0.618 15
P. Hermosilla, T. Ritschel, P.P. Vazquez, A. Vinacua, T. Ropinski: Monte Carlo Convolution for Learning on Non-Uniformly Sampled Point Clouds. SIGGRAPH Asia 2018
LAP-D	0.594 29	0.720 23	0.692 28	0.637 33	0.456 36	0.773 26	0.391 32	0.730 15	0.587 12	0.445 32	0.940 26	0.381 31	0.288 10	0.434 34	0.453 32	0.591 29	0.649 30	0.581 15	0.777 35	0.749 34	0.610 16

MVPNet	0.641 14	0.831 8	0.715 20	0.671 27	0.590 19	0.781 24	0.394 29	0.679 25	0.642 5	0.553 11	0.937 28	0.462 13	0.256 15	0.649 7	0.406 36	0.626 20	0.691 23	0.666 5	0.877 14	0.792 22	0.608 17
Maximilian Jaritz, Jiayuan Gu, Hao Su: Multi-view PointNet for 3D Scene Understanding. GMDL Workshop, ICCV 2019
HPEIN	0.618 26	0.729 22	0.668 32	0.647 31	0.597 17	0.766 28	0.414 23	0.680 24	0.520 25	0.525 16	0.946 20	0.432 22	0.215 22	0.493 31	0.599 26	0.638 18	0.617 36	0.570 16	0.897 8	0.806 13	0.605 18
Li Jiang, Hengshuang Zhao, Shu Liu, Xiaoyong Shen, Chi-Wing Fu, Jiaya Jia: Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentation. ICCV 2019
CCRFNet	0.589 31	0.766 16	0.659 36	0.683 24	0.470 35	0.740 33	0.387 33	0.620 33	0.490 31	0.476 28	0.922 36	0.355 36	0.245 18	0.511 24	0.511 28	0.571 32	0.643 32	0.493 35	0.872 20	0.762 30	0.600 19

joint point-based	0.634 18	0.614 34	0.778 9	0.667 29	0.633 13	0.825 9	0.420 22	0.804 7	0.467 35	0.561 10	0.951 8	0.494 9	0.291 9	0.566 20	0.458 31	0.579 31	0.764 10	0.559 21	0.838 27	0.814 10	0.598 20
Hung-Yueh Chiang, Yen-Liang Lin, Yueh-Cheng Liu, Winston H. Hsu: A Unified Point-Based Framework for 3D Segmentation. 3DV 2019
SConv	0.636 17	0.830 9	0.697 26	0.752 17	0.572 25	0.780 25	0.445 19	0.716 18	0.529 23	0.530 14	0.951 8	0.446 19	0.170 33	0.507 27	0.666 22	0.636 19	0.682 25	0.541 28	0.886 12	0.799 16	0.594 21

DPC	0.592 30	0.720 23	0.700 24	0.602 36	0.480 33	0.762 29	0.380 34	0.713 20	0.585 13	0.437 33	0.940 26	0.369 33	0.288 10	0.434 34	0.509 29	0.590 30	0.639 34	0.567 18	0.772 36	0.755 32	0.592 22
Francis Engelmann, Theodora Kontogianni, Bastian Leibe: Dilated Point Convolutions: On the Receptive Field Size of Point Convolutions on 3D Point Clouds. ICRA 2020
APCF-Net	0.631 21	0.742 19	0.687 31	0.672 26	0.557 27	0.792 20	0.408 25	0.665 28	0.545 22	0.508 20	0.952 7	0.428 25	0.186 30	0.634 9	0.702 18	0.620 21	0.706 18	0.555 22	0.873 19	0.798 18	0.581 23

PointMTL	0.632 20	0.731 21	0.688 29	0.675 25	0.591 18	0.784 23	0.444 21	0.565 36	0.610 7	0.492 24	0.949 13	0.456 15	0.254 16	0.587 14	0.706 17	0.599 26	0.665 29	0.612 11	0.868 24	0.791 23	0.579 24

PointMRNet	0.640 15	0.717 25	0.701 23	0.692 22	0.576 23	0.801 14	0.467 15	0.716 18	0.563 19	0.459 30	0.953 5	0.429 24	0.169 34	0.581 17	0.854 4	0.605 23	0.710 17	0.550 24	0.894 10	0.793 21	0.575 25

SIConv	0.625 24	0.830 9	0.694 27	0.757 15	0.563 26	0.772 27	0.448 17	0.647 31	0.520 25	0.509 19	0.949 13	0.431 23	0.191 29	0.496 30	0.614 25	0.647 17	0.672 27	0.535 30	0.876 15	0.783 25	0.571 26

TextureNet	0.566 33	0.672 29	0.664 34	0.671 27	0.494 31	0.719 34	0.445 19	0.678 26	0.411 40	0.396 35	0.935 30	0.356 35	0.225 20	0.412 36	0.535 27	0.565 33	0.636 35	0.464 37	0.794 34	0.680 40	0.568 27
Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkerhouser, Matthias Niessner, Leonidas Guibas: TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshes. CVPR
FPConv	0.639 16	0.785 12	0.760 12	0.713 19	0.603 16	0.798 18	0.392 30	0.534 38	0.603 10	0.524 17	0.948 16	0.457 14	0.250 17	0.538 22	0.723 14	0.598 27	0.696 22	0.614 9	0.872 20	0.799 16	0.567 28
Yiqun Lin, Zizheng Yan, Haibin Huang, Dong Du, Ligang Liu, Shuguang Cui, Xiaoguang Han: FPConv: Learning Local Flattening for Point Convolution. CVPR 2020
PanopticFusion-label	0.529 36	0.491 43	0.688 29	0.604 35	0.386 39	0.632 43	0.225 47	0.705 21	0.434 38	0.293 42	0.815 46	0.348 37	0.241 19	0.499 29	0.669 21	0.507 35	0.649 30	0.442 41	0.796 33	0.602 46	0.561 29
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
PointMRNet-lite	0.625 24	0.643 32	0.711 21	0.697 21	0.581 22	0.801 14	0.408 25	0.670 27	0.558 20	0.497 23	0.944 22	0.436 20	0.152 39	0.617 11	0.708 16	0.603 24	0.743 14	0.532 31	0.870 23	0.784 24	0.545 30

3DMV	0.484 39	0.484 44	0.538 45	0.643 32	0.424 37	0.606 46	0.310 37	0.574 35	0.433 39	0.378 37	0.796 47	0.301 39	0.214 23	0.537 23	0.208 45	0.472 40	0.507 46	0.413 44	0.693 41	0.602 46	0.539 31
Angela Dai, Matthias Niessner: 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation. ECCV'18
SPH3D-GCN	0.610 27	0.858 5	0.772 10	0.489 42	0.532 28	0.792 20	0.404 27	0.643 32	0.570 16	0.507 21	0.935 30	0.414 27	0.046 48	0.510 25	0.702 18	0.602 25	0.705 19	0.549 25	0.859 26	0.773 27	0.534 32
Huan Lei, Naveed Akhtar, and Ajmal Mian: Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds. TPAMI 2020
DVVNet	0.562 34	0.648 30	0.700 24	0.770 13	0.586 21	0.687 38	0.333 36	0.650 29	0.514 27	0.475 29	0.906 42	0.359 34	0.223 21	0.340 39	0.442 34	0.422 42	0.668 28	0.501 33	0.708 40	0.779 26	0.534 32

Pointnet++ & Feature	0.557 35	0.735 20	0.661 35	0.686 23	0.491 32	0.744 32	0.392 30	0.539 37	0.451 36	0.375 38	0.946 20	0.376 32	0.205 24	0.403 37	0.356 38	0.553 34	0.643 32	0.497 34	0.824 29	0.756 31	0.515 34

PCNN	0.498 38	0.559 39	0.644 39	0.560 39	0.420 38	0.711 36	0.229 45	0.414 40	0.436 37	0.352 39	0.941 24	0.324 38	0.155 37	0.238 44	0.387 37	0.493 36	0.529 43	0.509 32	0.813 32	0.751 33	0.504 35

SegGCN	0.589 31	0.833 7	0.731 18	0.539 40	0.514 30	0.789 22	0.448 17	0.467 39	0.573 14	0.484 27	0.936 29	0.396 29	0.061 47	0.501 28	0.507 30	0.594 28	0.700 21	0.563 19	0.874 17	0.771 28	0.493 36
Huan Lei, Naveed Akhtar, and Ajmal Mian: SegGCN: Efficient 3D Point Cloud Segmentation with Fuzzy Spherical Kernel. CVPR 2020
3DSM_DMMF	0.631 21	0.626 33	0.745 15	0.801 10	0.607 15	0.751 30	0.506 5	0.729 16	0.565 17	0.491 25	0.866 45	0.434 21	0.197 28	0.595 13	0.630 24	0.709 8	0.705 19	0.560 20	0.875 16	0.740 35	0.491 37

AttAN	0.609 28	0.760 17	0.667 33	0.649 30	0.521 29	0.793 19	0.457 16	0.648 30	0.528 24	0.434 34	0.947 19	0.401 28	0.153 38	0.454 32	0.721 15	0.648 16	0.717 16	0.536 29	0.904 6	0.765 29	0.485 38
Gege Zhang, Qinghua Ma, Licheng Jiao, Fang Liu and Qigong Sun: AttAN: Attention Adversarial Networks for 3D Point Cloud Semantic Segmentation. IJCAI2020
3DMV, FTSDF	0.501 37	0.558 40	0.608 42	0.424 47	0.478 34	0.690 37	0.246 43	0.586 34	0.468 34	0.450 31	0.911 40	0.394 30	0.160 36	0.438 33	0.212 44	0.432 41	0.541 42	0.475 36	0.742 38	0.727 36	0.477 39

PointCNN with RGB	0.458 40	0.577 38	0.611 41	0.356 49	0.321 45	0.715 35	0.299 39	0.376 43	0.328 46	0.319 40	0.944 22	0.285 41	0.164 35	0.216 47	0.229 43	0.484 38	0.545 41	0.456 39	0.755 37	0.709 37	0.475 40
Yangyan Li, Rui Bu, Mingchao Sun, Baoquan Chen: PointCNN. NeurIPS 2018
PNET2	0.442 42	0.548 41	0.548 44	0.597 37	0.363 42	0.628 44	0.300 38	0.292 44	0.374 43	0.307 41	0.881 44	0.268 42	0.186 30	0.238 44	0.204 46	0.407 43	0.506 47	0.449 40	0.667 42	0.620 44	0.462 41

SurfaceConvPF	0.442 42	0.505 42	0.622 40	0.380 48	0.342 44	0.654 41	0.227 46	0.397 42	0.367 44	0.276 44	0.924 35	0.240 45	0.198 27	0.359 38	0.262 41	0.366 44	0.581 38	0.435 42	0.640 43	0.668 41	0.398 42
Hao Pan, Shilin Liu, Yang Liu, Xin Tong: Convolutional Neural Networks on 3D Surfaces Using Parallel Frames.
ScanNet+FTSDF	0.383 47	0.297 49	0.491 47	0.432 46	0.358 43	0.612 45	0.274 41	0.116 48	0.411 40	0.265 45	0.904 43	0.229 46	0.079 45	0.250 42	0.185 47	0.320 47	0.510 44	0.385 45	0.548 46	0.597 48	0.394 43

Tangent Convolutions	0.438 44	0.437 47	0.646 38	0.474 44	0.369 41	0.645 42	0.353 35	0.258 46	0.282 48	0.279 43	0.918 39	0.298 40	0.147 40	0.283 41	0.294 40	0.487 37	0.562 39	0.427 43	0.619 44	0.633 43	0.352 44
Maxim Tatarchenko, Jaesik Park, Vladlen Koltun, Qian-Yi Zhou: Tangent convolutions for dense prediction in 3d. CVPR 2018
subcloud_weak	0.411 45	0.479 45	0.650 37	0.475 43	0.285 48	0.519 49	0.087 50	0.725 17	0.396 42	0.386 36	0.621 50	0.250 44	0.117 41	0.338 40	0.443 33	0.188 50	0.594 37	0.369 47	0.377 50	0.616 45	0.306 45

SSC-UNet	0.308 49	0.353 48	0.290 50	0.278 50	0.166 50	0.553 47	0.169 49	0.286 45	0.147 50	0.148 50	0.908 41	0.182 49	0.064 46	0.023 50	0.018 51	0.354 46	0.363 48	0.345 48	0.546 48	0.685 39	0.278 46

SPLAT Net	0.393 46	0.472 46	0.511 46	0.606 34	0.311 46	0.656 40	0.245 44	0.405 41	0.328 46	0.197 48	0.927 34	0.227 47	0.000 51	0.001 51	0.249 42	0.271 49	0.510 44	0.383 46	0.593 45	0.699 38	0.267 47
Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, Jan Kautz: SPLATNet: Sparse Lattice Networks for Point Cloud Processing. CVPR 2018
PointNet++	0.339 48	0.584 37	0.478 48	0.458 45	0.256 49	0.360 50	0.250 42	0.247 47	0.278 49	0.261 46	0.677 49	0.183 48	0.117 41	0.212 48	0.145 49	0.364 45	0.346 50	0.232 50	0.548 46	0.523 49	0.252 48
Charles R. Qi, Li Yi, Hao Su, Leonidas J. Guibas: pointnet++: deep hierarchical feature learning on point sets in a metric space.
FCPN	0.447 41	0.679 28	0.604 43	0.578 38	0.380 40	0.682 39	0.291 40	0.106 49	0.483 32	0.258 47	0.920 37	0.258 43	0.025 49	0.231 46	0.325 39	0.480 39	0.560 40	0.463 38	0.725 39	0.666 42	0.231 49
Dario Rethage, Johanna Wald, Jürgen Sturm, Nassir Navab, Federico Tombari: Fully-Convolutional Point Networks for Large-Scale Point Clouds. ECCV 2018
ScanNet	0.306 50	0.203 50	0.366 49	0.501 41	0.311 46	0.524 48	0.211 48	0.002 51	0.342 45	0.189 49	0.786 48	0.145 50	0.102 44	0.245 43	0.152 48	0.318 48	0.348 49	0.300 49	0.460 49	0.437 50	0.182 50
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nießner: ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. CVPR'17
ERROR	0.054 51	0.000 51	0.041 51	0.172 51	0.030 51	0.062 51	0.001 51	0.035 50	0.004 51	0.051 51	0.143 51	0.019 51	0.003 50	0.041 49	0.050 50	0.003 51	0.054 51	0.018 51	0.005 51	0.264 51	0.082 51

This table lists the benchmark results for the 3D semantic instance scenario.

Method	avg ap 50%	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	window

PointGroup	0.636 3	1.000 1	0.765 5	0.624 5	0.505 5	0.797 4	0.116 7	0.696 3	0.384 6	0.441 6	0.559 2	0.476 1	0.596 4	1.000 1	0.666 2	0.756 5	0.556 9	0.997 6	0.513 1
Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi-Wing Fu, Jiaya Jia: PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation. CVPR 2020 [oral]
PCJC	0.578 5	1.000 1	0.810 3	0.583 9	0.449 9	0.813 2	0.042 12	0.603 6	0.341 8	0.490 3	0.465 4	0.410 4	0.650 2	0.835 10	0.264 15	0.694 7	0.561 7	0.889 13	0.504 2

GICN	0.638 2	1.000 1	0.895 1	0.800 1	0.480 6	0.676 8	0.144 4	0.737 2	0.354 7	0.447 5	0.400 11	0.365 6	0.700 1	1.000 1	0.569 3	0.836 1	0.599 3	1.000 1	0.473 3

OccuSeg+instance	0.672 1	1.000 1	0.758 7	0.682 3	0.576 2	0.842 1	0.477 1	0.504 12	0.524 2	0.567 1	0.585 1	0.451 2	0.557 6	1.000 1	0.751 1	0.797 2	0.563 6	1.000 1	0.467 4

3D-BoNet	0.488 11	1.000 1	0.672 13	0.590 7	0.301 13	0.484 17	0.098 8	0.620 4	0.306 9	0.341 13	0.259 15	0.125 14	0.434 11	0.796 11	0.402 10	0.499 17	0.513 11	0.909 12	0.439 5
Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, Niki Trigoni: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds. NeurIPS 2019 Spotlight
3D-MPA	0.611 4	1.000 1	0.833 2	0.765 2	0.526 3	0.756 7	0.136 6	0.588 8	0.470 4	0.438 7	0.432 9	0.358 7	0.650 2	0.857 6	0.429 8	0.765 4	0.557 8	1.000 1	0.430 6
Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, Matthias Nießner: 3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation. CVPR 2020
MTML	0.549 8	1.000 1	0.807 4	0.588 8	0.327 12	0.647 9	0.004 18	0.815 1	0.180 12	0.418 8	0.364 13	0.182 10	0.445 9	1.000 1	0.442 7	0.688 9	0.571 5	1.000 1	0.396 7
Jean Lahoud, Bernard Ghanem, Marc Pollefeys, Martin R. Oswald: 3D Instance Segmentation via Multi-task Metric Learning. ICCV 2019 [oral]
Occipital-SCS	0.512 10	1.000 1	0.716 10	0.509 10	0.506 4	0.611 11	0.092 9	0.602 7	0.177 13	0.346 12	0.383 12	0.165 12	0.442 10	0.850 9	0.386 12	0.618 12	0.543 10	0.889 13	0.389 8

PanopticFusion-inst	0.478 12	0.667 13	0.712 12	0.595 6	0.259 15	0.550 15	0.000 21	0.613 5	0.175 14	0.250 16	0.434 7	0.437 3	0.411 13	0.857 6	0.485 6	0.591 15	0.267 19	0.944 9	0.359 9
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
SSEN	0.575 6	1.000 1	0.761 6	0.473 11	0.477 7	0.795 5	0.066 10	0.529 9	0.658 1	0.460 4	0.461 5	0.380 5	0.331 14	0.859 5	0.401 11	0.692 8	0.653 1	1.000 1	0.348 10
Dongsu Zhang, Junha Chun, Sang Kyun Cha, Young Min Kim: Spatial Semantic Embedding Network: Fast 3D Instance Segmentation with Deep Metric Learning. Arxiv
Sparse R-CNN	0.515 9	1.000 1	0.538 16	0.282 15	0.468 8	0.790 6	0.173 3	0.345 14	0.429 5	0.413 10	0.484 3	0.176 11	0.595 5	0.591 14	0.522 4	0.668 10	0.476 12	0.986 7	0.327 11

PE	0.560 7	0.667 13	0.722 9	0.673 4	0.577 1	0.806 3	0.202 2	0.520 10	0.524 3	0.491 2	0.450 6	0.061 17	0.553 7	0.688 13	0.507 5	0.777 3	0.604 2	0.941 10	0.322 12

SALoss-ResNet	0.459 13	1.000 1	0.737 8	0.159 20	0.259 14	0.587 13	0.138 5	0.475 13	0.217 11	0.416 9	0.408 10	0.128 13	0.315 15	0.714 12	0.411 9	0.536 16	0.590 4	0.873 16	0.304 13
Zhidong Liang, Ming Yang, Hao Li, Chunxiang Wang: 3D Instance Embedding Learning With a Structure-Aware Loss Function for Point Cloud Segmentation. IEEE Robotics and Automation Letters (IROS2020)
MASC	0.447 14	0.528 18	0.555 15	0.381 12	0.382 10	0.633 10	0.002 19	0.509 11	0.260 10	0.361 11	0.432 8	0.327 8	0.451 8	0.571 15	0.367 13	0.639 11	0.386 13	0.980 8	0.276 14
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
R-PointNet	0.306 17	0.500 19	0.405 19	0.311 13	0.348 11	0.589 12	0.054 11	0.068 20	0.126 15	0.283 15	0.290 14	0.028 19	0.219 16	0.214 19	0.331 14	0.396 19	0.275 17	0.821 18	0.245 15

3D-SIS	0.382 15	1.000 1	0.432 18	0.245 16	0.190 16	0.577 14	0.013 16	0.263 16	0.033 20	0.320 14	0.240 16	0.075 16	0.422 12	0.857 6	0.117 18	0.699 6	0.271 18	0.883 15	0.235 16
Ji Hou, Angela Dai, Matthias Niessner: 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans. CVPR 2019
3D-BEVIS	0.248 18	0.667 13	0.566 14	0.076 21	0.035 22	0.394 19	0.027 14	0.035 21	0.098 16	0.099 20	0.030 21	0.025 20	0.098 19	0.375 17	0.126 17	0.604 14	0.181 20	0.854 17	0.171 17
Cathrin Elich, Francis Engelmann, Jonas Schult, Theodora Kontogianni, Bastian Leibe: 3D-BEVIS: Birds-Eye-View Instance Segmentation.
Region	0.248 18	0.667 13	0.437 17	0.188 18	0.153 18	0.491 16	0.000 21	0.208 18	0.094 17	0.153 18	0.099 19	0.057 18	0.217 17	0.119 20	0.039 21	0.466 18	0.302 16	0.640 19	0.140 18

Sgpn_scannet	0.143 21	0.208 22	0.390 20	0.169 19	0.065 20	0.275 21	0.029 13	0.069 19	0.000 21	0.087 21	0.043 20	0.014 21	0.027 22	0.000 21	0.112 20	0.351 20	0.168 21	0.438 21	0.138 19

Seg-Cluster	0.215 20	0.370 20	0.337 21	0.285 14	0.105 19	0.325 20	0.025 15	0.282 15	0.085 18	0.105 19	0.107 18	0.007 22	0.079 20	0.317 18	0.114 19	0.309 21	0.304 15	0.587 20	0.123 20

UNet-backbone	0.319 16	0.667 13	0.715 11	0.233 17	0.189 17	0.479 18	0.008 17	0.218 17	0.067 19	0.201 17	0.173 17	0.107 15	0.123 18	0.438 16	0.150 16	0.615 13	0.355 14	0.916 11	0.093 21

MaskRCNN 2d->3d Proj	0.058 22	0.333 21	0.002 22	0.000 22	0.053 21	0.002 22	0.002 20	0.021 22	0.000 21	0.045 22	0.024 22	0.238 9	0.065 21	0.000 21	0.014 22	0.107 22	0.020 22	0.110 22	0.006 22

This table lists the benchmark results for the 2D semantic label scenario.

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

Virtual MVFusion (R)	0.745 1	0.861 1	0.839 1	0.881 1	0.672 1	0.512 1	0.422 10	0.898 1	0.723 1	0.714 1	0.954 2	0.454 1	0.509 1	0.773 1	0.895 1	0.756 1	0.820 1	0.653 1	0.935 1	0.891 1	0.728 1
Abhijit Kundu, Xiaoqi Yin, Alireza Fathi, David Ross, Brian Brewington, Thomas Funkhouser, Caroline Pantofaru: Virtual Multi-view Fusion for 3D Semantic Segmentation. ECCV 2020
BPNet_2D	0.670 2	0.822 3	0.795 3	0.836 2	0.659 2	0.481 2	0.451 6	0.769 2	0.656 3	0.567 3	0.931 3	0.395 3	0.390 4	0.700 2	0.534 3	0.689 5	0.770 2	0.574 3	0.865 3	0.831 3	0.675 2

CU-Hybrid-2D Net	0.636 3	0.825 2	0.820 2	0.179 16	0.648 3	0.463 3	0.549 1	0.742 3	0.676 2	0.628 2	0.961 1	0.420 2	0.379 5	0.684 3	0.381 10	0.732 2	0.723 3	0.599 2	0.827 8	0.851 2	0.634 3

SN_RN152pyrx8_RVC	0.546 10	0.572 9	0.663 13	0.638 8	0.518 9	0.298 12	0.366 15	0.633 12	0.510 11	0.446 12	0.864 11	0.296 11	0.267 12	0.542 10	0.346 13	0.704 4	0.575 14	0.431 12	0.853 5	0.766 10	0.630 4
Marin Oršić, Ivan Krešo, Petra Bevandić, Siniša Šegvić: In Defense of Pre-trained ImageNet Architectures for Real-time Semantic Segmentation of Road-driving Images. CVPR 2019
DCRedNet	0.583 8	0.682 5	0.723 8	0.542 10	0.510 11	0.310 11	0.451 6	0.668 8	0.549 8	0.520 6	0.920 5	0.375 4	0.446 2	0.528 11	0.417 9	0.670 9	0.577 13	0.478 8	0.862 4	0.806 5	0.628 5

RFBNet	0.592 7	0.616 7	0.758 4	0.659 6	0.581 7	0.330 9	0.469 5	0.655 10	0.543 9	0.524 5	0.924 4	0.355 7	0.336 8	0.572 8	0.479 8	0.671 8	0.648 6	0.480 7	0.814 10	0.814 4	0.614 6

SSMA	0.577 9	0.695 4	0.716 10	0.439 12	0.563 8	0.314 10	0.444 8	0.719 5	0.551 7	0.503 7	0.887 10	0.346 9	0.348 7	0.603 4	0.353 12	0.709 3	0.600 11	0.457 10	0.901 2	0.786 6	0.599 7
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. International Journal of Computer Vision, 2019
ILC-PSPNet	0.475 15	0.490 14	0.581 15	0.289 14	0.507 12	0.067 17	0.379 13	0.610 14	0.417 15	0.435 13	0.822 16	0.278 12	0.267 12	0.503 13	0.228 14	0.616 14	0.533 15	0.375 14	0.820 9	0.729 12	0.560 8

FuseNet	0.535 11	0.570 10	0.681 12	0.182 15	0.512 10	0.290 13	0.431 9	0.659 9	0.504 12	0.495 9	0.903 9	0.308 10	0.428 3	0.523 12	0.365 11	0.676 6	0.621 10	0.470 9	0.762 13	0.779 7	0.541 9
Caner Hazirbas, Lingni Ma, Csaba Domokos, Daniel Cremers: FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-based CNN Architecture. ACCV 2016
MSeg1080_RVC	0.485 14	0.505 13	0.709 11	0.092 17	0.427 14	0.241 14	0.411 11	0.654 11	0.385 17	0.457 11	0.861 12	0.053 17	0.279 11	0.503 13	0.481 7	0.645 11	0.626 9	0.365 15	0.748 15	0.725 13	0.529 10
John Lambert, Zhuang Liu, Ozan Sener, James Hays, Vladlen Koltun: MSeg: A Composite Dataset for Multi-domain Semantic Segmentation. CVPR 2020
DMMF	0.597 5	0.543 11	0.755 6	0.749 4	0.585 6	0.338 6	0.494 4	0.704 6	0.598 4	0.494 10	0.911 6	0.347 8	0.327 9	0.593 7	0.527 4	0.675 7	0.646 8	0.513 4	0.842 6	0.774 8	0.527 11

MCA-Net	0.595 6	0.533 12	0.756 5	0.746 5	0.590 5	0.334 8	0.506 3	0.670 7	0.587 6	0.500 8	0.905 8	0.366 5	0.352 6	0.601 5	0.506 6	0.669 10	0.648 6	0.501 5	0.839 7	0.769 9	0.516 12

DMMF_3d	0.605 4	0.651 6	0.744 7	0.782 3	0.637 4	0.387 4	0.536 2	0.732 4	0.590 5	0.540 4	0.856 13	0.359 6	0.306 10	0.596 6	0.539 2	0.627 12	0.706 4	0.497 6	0.785 12	0.757 11	0.476 13

Enet (reimpl)	0.376 16	0.264 17	0.452 17	0.452 11	0.365 15	0.181 15	0.143 17	0.456 16	0.409 16	0.346 16	0.769 17	0.164 15	0.218 15	0.359 16	0.123 17	0.403 17	0.381 17	0.313 17	0.571 16	0.685 15	0.472 14
Re-implementation of Adam Paszke, Abhishek Chaurasia, Sangpil Kim, Eugenio Culurciello: ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation.
3DMV (2d proj)	0.498 13	0.481 15	0.612 14	0.579 9	0.456 13	0.343 5	0.384 12	0.623 13	0.525 10	0.381 14	0.845 14	0.254 13	0.264 14	0.557 9	0.182 15	0.581 15	0.598 12	0.429 13	0.760 14	0.661 16	0.446 15
Angela Dai, Matthias Niessner: 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation. ECCV'18
AdapNet++	0.503 12	0.613 8	0.722 9	0.418 13	0.358 17	0.337 7	0.370 14	0.479 15	0.443 13	0.368 15	0.907 7	0.207 14	0.213 16	0.464 15	0.525 5	0.618 13	0.657 5	0.450 11	0.788 11	0.721 14	0.408 16
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. International Journal of Computer Vision, 2019
ScanNet (2d proj)	0.330 17	0.293 16	0.521 16	0.657 7	0.361 16	0.161 16	0.250 16	0.004 17	0.440 14	0.183 17	0.836 15	0.125 16	0.060 17	0.319 17	0.132 16	0.417 16	0.412 16	0.344 16	0.541 17	0.427 17	0.109 17
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nießner: ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. CVPR'17

This table lists the benchmark results for the 2D semantic instance scenario.

Method	avg ap	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	window

UniDet_RVC	0.205 1	0.381 1	0.323 1	0.037 1	0.226 1	0.177 1	0.063 1	0.277 1	0.120 1	0.067 1	0.131 1	0.074 2	0.317 1	0.080 1	0.235 1	0.289 1	0.141 1	0.678 1	0.080 1

MaskRCNN_ScanNet	0.119 2	0.129 2	0.212 2	0.002 2	0.112 2	0.148 2	0.014 2	0.205 2	0.044 2	0.066 2	0.078 2	0.095 1	0.142 2	0.030 2	0.128 2	0.139 2	0.080 2	0.459 2	0.057 2
Re-implementation of Kaiming He, Georgia Gkioxari, Piotr Dollár, Ross Girshick: Mask R-CNN. ICCV'17

This table lists the benchmark results for the scene type classification scenario.

Method	avg recall	apartment	bathroom	bedroom / hotel	bookstore / library	conference room	copy/mail room	hallway	kitchen	laundry room	living room / lounge	misc	office	storage / basement / garage

multi-task	0.700 1	0.500 1	1.000 1	0.882 2	0.500 1	1.000 1	1.000 1	0.500 1	1.000 1	1.000 1	0.778 1	0.000 2	0.938 1	0.000 1
Shengyu Huang, Mikhail Usvyatsov, Konrad Schindler: Indoor Scene Recognition in 3D. IROS 2020
SE-ResNeXt-SSMA	0.498 2	0.000 3	0.812 2	0.941 1	0.500 1	0.500 2	0.500 2	0.500 1	0.429 3	0.500 2	0.667 2	0.500 1	0.625 2	0.000 1
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. arXiv
resnet50_scannet	0.353 3	0.250 2	0.812 2	0.529 3	0.500 1	0.500 2	0.000 3	0.500 1	0.571 2	0.000 3	0.556 3	0.000 2	0.375 3	0.000 1

3D Semantic label benchmark

3D Semantic instance benchmark

2D Semantic label benchmark

2D Semantic instance benchmark

Scene type classification benchmark