Benchmark Results - ScanNet Benchmark

This table lists the benchmark results for the 3D semantic label scenario.

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

FusionNet	0.688 10	0.704 32	0.741 19	0.754 18	0.656 10	0.829 8	0.501 7	0.741 18	0.609 11	0.548 14	0.950 12	0.522 6	0.371 1	0.633 12	0.756 16	0.715 7	0.771 10	0.623 9	0.861 29	0.814 11	0.658 9
Feihu Zhang, Jin Fang, Benjamin Wah, Philip Torr: Deep FusionNet for Point Cloud Semantic Segmentation. ECCV 2020
OccuSeg+Semantic	0.764 1	0.758 23	0.796 6	0.839 3	0.746 2	0.907 1	0.562 1	0.850 4	0.680 2	0.672 1	0.978 1	0.610 1	0.335 2	0.777 1	0.819 11	0.847 1	0.830 1	0.691 3	0.972 1	0.885 1	0.727 2

Virtual MVFusion	0.746 3	0.771 20	0.819 2	0.848 1	0.702 6	0.865 3	0.397 36	0.899 1	0.699 1	0.664 2	0.948 17	0.588 2	0.330 3	0.746 4	0.851 6	0.764 3	0.796 5	0.704 1	0.935 3	0.866 2	0.728 1
Abhijit Kundu, Xiaoqi Yin, Alireza Fathi, David Ross, Brian Brewington, Thomas Funkhouser, Caroline Pantofaru: Virtual Multi-view Fusion for 3D Semantic Segmentation. ECCV 2020
SparseConvNet	0.725 5	0.647 39	0.821 1	0.846 2	0.721 4	0.869 2	0.533 2	0.754 15	0.603 13	0.614 5	0.955 3	0.572 3	0.325 4	0.710 6	0.870 2	0.724 6	0.823 2	0.628 8	0.934 4	0.865 3	0.683 7

FusionAwareConv	0.630 31	0.604 43	0.741 19	0.766 16	0.590 23	0.747 39	0.501 7	0.734 20	0.503 35	0.527 19	0.919 46	0.454 21	0.323 5	0.550 27	0.420 43	0.678 13	0.688 31	0.544 33	0.896 9	0.795 21	0.627 15
Jiazhao Zhang, Chenyang Zhu, Lintao Zheng, Kai Xu: Fusion-Aware Point Convolution for Online Semantic 3D Scene Segmentation. CVPR 2020
CU-Hybrid Net	0.693 9	0.596 44	0.789 7	0.803 10	0.677 7	0.800 21	0.469 15	0.846 5	0.554 27	0.591 10	0.948 17	0.500 8	0.316 6	0.609 15	0.847 8	0.732 4	0.808 4	0.593 17	0.894 10	0.839 7	0.652 10

BPNet	0.749 2	0.909 1	0.818 3	0.811 7	0.752 1	0.839 5	0.485 11	0.842 6	0.673 3	0.644 3	0.957 2	0.528 5	0.305 7	0.773 2	0.859 3	0.788 2	0.818 3	0.693 2	0.916 5	0.856 4	0.723 4

DCM-Net	0.658 15	0.778 17	0.702 28	0.806 9	0.619 16	0.813 16	0.468 16	0.693 28	0.494 37	0.524 21	0.941 30	0.449 24	0.298 8	0.510 32	0.821 10	0.675 14	0.727 19	0.568 23	0.826 35	0.803 16	0.637 13
Jonas Schult, Francis Engelmann, Theodora Kontogianni, Bastian Leibe: DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshes. CVPR 2020 [Oral]
joint point-based	0.634 26	0.614 42	0.778 9	0.667 36	0.633 15	0.825 11	0.420 30	0.804 8	0.467 43	0.561 12	0.951 8	0.494 9	0.291 9	0.566 26	0.458 39	0.579 39	0.764 11	0.559 27	0.838 33	0.814 11	0.598 24
Hung-Yueh Chiang, Yen-Liang Lin, Yueh-Cheng Liu, Winston H. Hsu: A Unified Point-Based Framework for 3D Segmentation. 3DV 2019
LAP-D	0.594 37	0.720 29	0.692 35	0.637 41	0.456 44	0.773 33	0.391 40	0.730 21	0.587 16	0.445 39	0.940 32	0.381 39	0.288 10	0.434 42	0.453 40	0.591 37	0.649 37	0.581 19	0.777 43	0.749 42	0.610 20

DPC	0.592 38	0.720 29	0.700 31	0.602 44	0.480 41	0.762 36	0.380 42	0.713 26	0.585 17	0.437 40	0.940 32	0.369 41	0.288 10	0.434 42	0.509 37	0.590 38	0.639 42	0.567 24	0.772 44	0.755 40	0.592 27
Francis Engelmann, Theodora Kontogianni, Bastian Leibe: Dilated Point Convolutions: On the Receptive Field Size of Point Convolutions on 3D Point Clouds. ICRA 2020
MinkowskiNet	0.736 4	0.859 5	0.818 3	0.832 4	0.709 5	0.840 4	0.521 4	0.853 3	0.660 4	0.643 4	0.951 8	0.544 4	0.286 12	0.731 5	0.893 1	0.675 14	0.772 9	0.683 4	0.874 20	0.852 5	0.727 2
C. Choy, J. Gwak, S. Savarese: 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks. CVPR 2019
MatchingNet	0.724 6	0.812 14	0.812 5	0.810 8	0.735 3	0.834 6	0.495 9	0.860 2	0.572 19	0.602 8	0.954 4	0.512 7	0.280 13	0.757 3	0.845 9	0.725 5	0.780 7	0.606 15	0.937 2	0.851 6	0.700 6

PointASNL	0.666 13	0.703 33	0.781 8	0.751 20	0.655 11	0.830 7	0.471 14	0.769 12	0.474 41	0.537 16	0.951 8	0.475 13	0.279 14	0.635 10	0.698 27	0.675 14	0.751 14	0.553 30	0.816 38	0.806 14	0.703 5
Xu Yan, Chaoda Zheng, Zhen Li, Sheng Wang, Shuguang Cui: PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Sampling. CVPR 2020
Supervoxel-CNN	0.635 25	0.656 37	0.711 25	0.719 22	0.613 17	0.757 37	0.444 27	0.765 13	0.534 29	0.566 11	0.928 40	0.478 12	0.272 15	0.636 9	0.531 35	0.664 17	0.645 39	0.508 40	0.864 28	0.792 25	0.611 18

RandLA-Net	0.645 17	0.778 17	0.731 21	0.699 25	0.577 27	0.829 8	0.446 23	0.736 19	0.477 40	0.523 23	0.945 24	0.454 21	0.269 16	0.484 39	0.749 18	0.618 29	0.738 17	0.599 16	0.827 34	0.792 25	0.621 16

MVPNet	0.641 18	0.831 9	0.715 24	0.671 34	0.590 23	0.781 31	0.394 37	0.679 31	0.642 6	0.553 13	0.937 34	0.462 18	0.256 17	0.649 7	0.406 44	0.626 26	0.691 30	0.666 5	0.877 16	0.792 25	0.608 21
Maximilian Jaritz, Jiayuan Gu, Hao Su: Multi-view PointNet for 3D Scene Understanding. GMDL Workshop, ICCV 2019
PointMTL	0.632 28	0.731 27	0.688 37	0.675 32	0.591 22	0.784 30	0.444 27	0.565 44	0.610 10	0.492 29	0.949 14	0.456 20	0.254 18	0.587 18	0.706 24	0.599 34	0.665 36	0.612 14	0.868 27	0.791 28	0.579 29

VACNN++	0.638 22	0.820 12	0.701 29	0.687 29	0.594 21	0.791 28	0.430 29	0.587 41	0.569 21	0.529 18	0.950 12	0.467 16	0.253 19	0.524 30	0.722 21	0.618 29	0.694 29	0.570 21	0.793 42	0.802 17	0.659 8

FPConv	0.639 21	0.785 15	0.760 13	0.713 24	0.603 19	0.798 23	0.392 38	0.534 46	0.603 13	0.524 21	0.948 17	0.457 19	0.250 20	0.538 28	0.723 20	0.598 35	0.696 28	0.614 11	0.872 23	0.799 18	0.567 34
Yiqun Lin, Zizheng Yan, Haibin Huang, Dong Du, Ligang Liu, Shuguang Cui, Xiaoguang Han: FPConv: Learning Local Flattening for Point Convolution. CVPR 2020
RFCR	0.702 7	0.889 2	0.745 16	0.813 6	0.672 8	0.818 14	0.493 10	0.815 7	0.623 8	0.610 6	0.947 20	0.470 15	0.249 21	0.594 17	0.848 7	0.705 9	0.779 8	0.646 7	0.892 12	0.823 9	0.611 18

CCRFNet	0.589 39	0.766 21	0.659 44	0.683 31	0.470 43	0.740 41	0.387 41	0.620 40	0.490 38	0.476 35	0.922 44	0.355 44	0.245 22	0.511 31	0.511 36	0.571 40	0.643 40	0.493 43	0.872 23	0.762 38	0.600 23

PanopticFusion-label	0.529 44	0.491 51	0.688 37	0.604 43	0.386 47	0.632 51	0.225 55	0.705 27	0.434 46	0.293 50	0.815 54	0.348 45	0.241 23	0.499 36	0.669 28	0.507 43	0.649 37	0.442 49	0.796 40	0.602 54	0.561 36
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
HPGCNN	0.656 16	0.698 34	0.743 18	0.650 37	0.564 32	0.820 13	0.505 6	0.758 14	0.631 7	0.479 34	0.945 24	0.480 11	0.226 24	0.572 24	0.774 15	0.690 11	0.735 18	0.614 11	0.853 31	0.776 34	0.597 25
Jisheng Dang, Qingyong Hu, Yulan Guo, Jun Yang: HPGCNN.
TextureNet	0.566 41	0.672 36	0.664 42	0.671 34	0.494 39	0.719 42	0.445 25	0.678 32	0.411 48	0.396 43	0.935 36	0.356 43	0.225 25	0.412 44	0.535 34	0.565 41	0.636 43	0.464 45	0.794 41	0.680 48	0.568 33
Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkerhouser, Matthias Niessner, Leonidas Guibas: TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshes. CVPR
DVVNet	0.562 42	0.648 38	0.700 31	0.770 14	0.586 25	0.687 46	0.333 44	0.650 36	0.514 34	0.475 36	0.906 50	0.359 42	0.223 26	0.340 47	0.442 42	0.422 50	0.668 35	0.501 41	0.708 48	0.779 32	0.534 40

HPEIN	0.618 34	0.729 28	0.668 40	0.647 39	0.597 20	0.766 35	0.414 31	0.680 30	0.520 32	0.525 20	0.946 22	0.432 28	0.215 27	0.493 38	0.599 33	0.638 23	0.617 44	0.570 21	0.897 8	0.806 14	0.605 22
Li Jiang, Hengshuang Zhao, Shu Liu, Xiaoyong Shen, Chi-Wing Fu, Jiaya Jia: Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentation. ICCV 2019
3DMV	0.484 47	0.484 52	0.538 53	0.643 40	0.424 45	0.606 54	0.310 45	0.574 43	0.433 47	0.378 45	0.796 55	0.301 47	0.214 28	0.537 29	0.208 53	0.472 48	0.507 54	0.413 52	0.693 49	0.602 54	0.539 39
Angela Dai, Matthias Niessner: 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation. ECCV'18
Pointnet++ & Feature	0.557 43	0.735 25	0.661 43	0.686 30	0.491 40	0.744 40	0.392 38	0.539 45	0.451 44	0.375 46	0.946 22	0.376 40	0.205 29	0.403 45	0.356 46	0.553 42	0.643 40	0.497 42	0.824 36	0.756 39	0.515 42

JSENet	0.699 8	0.881 3	0.762 12	0.821 5	0.667 9	0.800 21	0.522 3	0.792 9	0.613 9	0.607 7	0.935 36	0.492 10	0.205 29	0.576 23	0.853 5	0.691 10	0.758 12	0.652 6	0.872 23	0.828 8	0.649 11
Zeyu HU, Mingmin Zhen, Xuyang BAI, Hongbo Fu, Chiew-lan Tai: JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds. ECCV 2020
PointConv	0.666 13	0.781 16	0.759 14	0.699 25	0.644 14	0.822 12	0.475 12	0.779 10	0.564 23	0.504 27	0.953 5	0.428 31	0.203 31	0.586 20	0.754 17	0.661 18	0.753 13	0.588 18	0.902 7	0.813 13	0.642 12
Wenxuan Wu, Zhongang Qi, Li Fuxin: PointConv: Deep Convolutional Networks on 3D Point Clouds. CVPR 2019
SurfaceConvPF	0.442 50	0.505 50	0.622 48	0.380 56	0.342 52	0.654 49	0.227 54	0.397 50	0.367 52	0.276 52	0.924 42	0.240 53	0.198 32	0.359 46	0.262 49	0.366 52	0.581 46	0.435 50	0.640 51	0.668 49	0.398 50
Hao Pan, Shilin Liu, Yang Liu, Xin Tong: Convolutional Neural Networks on 3D Surfaces Using Parallel Frames.
3DSM_DMMF	0.631 29	0.626 41	0.745 16	0.801 11	0.607 18	0.751 38	0.506 5	0.729 22	0.565 22	0.491 30	0.866 53	0.434 27	0.197 33	0.595 16	0.630 31	0.709 8	0.705 24	0.560 26	0.875 18	0.740 43	0.491 45

SIConv	0.625 32	0.830 10	0.694 34	0.757 17	0.563 33	0.772 34	0.448 21	0.647 38	0.520 32	0.509 24	0.949 14	0.431 29	0.191 34	0.496 37	0.614 32	0.647 22	0.672 34	0.535 37	0.876 17	0.783 31	0.571 31

APCF-Net	0.631 29	0.742 24	0.687 39	0.672 33	0.557 34	0.792 26	0.408 33	0.665 35	0.545 28	0.508 25	0.952 7	0.428 31	0.186 35	0.634 11	0.702 25	0.620 27	0.706 23	0.555 29	0.873 22	0.798 20	0.581 28
Haojia, Lin: Adaptive Pyramid Context Fusion for Point Cloud Perception. GRSL
PNET2	0.442 50	0.548 49	0.548 52	0.597 45	0.363 50	0.628 52	0.300 46	0.292 52	0.374 51	0.307 49	0.881 52	0.268 50	0.186 35	0.238 52	0.204 54	0.407 51	0.506 55	0.449 48	0.667 50	0.620 52	0.462 49

KP-FCNN	0.684 11	0.847 7	0.758 15	0.784 12	0.647 13	0.814 15	0.473 13	0.772 11	0.605 12	0.594 9	0.935 36	0.450 23	0.181 37	0.587 18	0.805 13	0.690 11	0.785 6	0.614 11	0.882 15	0.819 10	0.632 14
H. Thomas, C. Qi, J. Deschaud, B. Marcotegui, F. Goulette, L. Guibas.: KPConv: Flexible and Deformable Convolution for Point Clouds. ICCV 2019
SConv	0.636 24	0.830 10	0.697 33	0.752 19	0.572 31	0.780 32	0.445 25	0.716 24	0.529 30	0.530 17	0.951 8	0.446 25	0.170 38	0.507 34	0.666 29	0.636 24	0.682 32	0.541 35	0.886 14	0.799 18	0.594 26

PointMRNet	0.640 20	0.717 31	0.701 29	0.692 28	0.576 28	0.801 19	0.467 17	0.716 24	0.563 24	0.459 37	0.953 5	0.429 30	0.169 39	0.581 21	0.854 4	0.605 31	0.710 21	0.550 31	0.894 10	0.793 24	0.575 30

PointCNN with RGB	0.458 48	0.577 46	0.611 49	0.356 57	0.321 53	0.715 43	0.299 47	0.376 51	0.328 54	0.319 48	0.944 26	0.285 49	0.164 40	0.216 55	0.229 51	0.484 46	0.545 49	0.456 47	0.755 45	0.709 45	0.475 48
Yangyan Li, Rui Bu, Mingchao Sun, Baoquan Chen: PointCNN. NeurIPS 2018
PointConv-SFPN	0.641 18	0.776 19	0.703 27	0.721 21	0.557 34	0.826 10	0.451 19	0.672 33	0.563 24	0.483 33	0.943 28	0.425 33	0.162 41	0.644 8	0.726 19	0.659 19	0.709 22	0.572 20	0.875 18	0.786 29	0.559 37

3DMV, FTSDF	0.501 45	0.558 48	0.608 50	0.424 55	0.478 42	0.690 45	0.246 51	0.586 42	0.468 42	0.450 38	0.911 48	0.394 38	0.160 42	0.438 41	0.212 52	0.432 49	0.541 50	0.475 44	0.742 46	0.727 44	0.477 47

PCNN	0.498 46	0.559 47	0.644 47	0.560 47	0.420 46	0.711 44	0.229 53	0.414 48	0.436 45	0.352 47	0.941 30	0.324 46	0.155 43	0.238 52	0.387 45	0.493 44	0.529 51	0.509 39	0.813 39	0.751 41	0.504 43

AttAN	0.609 36	0.760 22	0.667 41	0.649 38	0.521 37	0.793 25	0.457 18	0.648 37	0.528 31	0.434 42	0.947 20	0.401 36	0.153 44	0.454 40	0.721 22	0.648 21	0.717 20	0.536 36	0.904 6	0.765 37	0.485 46
Gege Zhang, Qinghua Ma, Licheng Jiao, Fang Liu and Qigong Sun: AttAN: Attention Adversarial Networks for 3D Point Cloud Semantic Segmentation. IJCAI2020
PointMRNet-lite	0.625 32	0.643 40	0.711 25	0.697 27	0.581 26	0.801 19	0.408 33	0.670 34	0.558 26	0.497 28	0.944 26	0.436 26	0.152 45	0.617 14	0.708 23	0.603 32	0.743 16	0.532 38	0.870 26	0.784 30	0.545 38

PointSPNet	0.637 23	0.734 26	0.692 35	0.714 23	0.576 28	0.797 24	0.446 23	0.743 17	0.598 15	0.437 40	0.942 29	0.403 35	0.150 46	0.626 13	0.800 14	0.649 20	0.697 27	0.557 28	0.846 32	0.777 33	0.563 35

SALANet	0.670 12	0.816 13	0.770 11	0.768 15	0.652 12	0.807 18	0.451 19	0.747 16	0.659 5	0.545 15	0.924 42	0.473 14	0.149 47	0.571 25	0.811 12	0.635 25	0.746 15	0.623 9	0.892 12	0.794 23	0.570 32

Tangent Convolutions	0.438 52	0.437 55	0.646 46	0.474 52	0.369 49	0.645 50	0.353 43	0.258 54	0.282 56	0.279 51	0.918 47	0.298 48	0.147 48	0.283 49	0.294 48	0.487 45	0.562 47	0.427 51	0.619 52	0.633 51	0.352 52
Maxim Tatarchenko, Jaesik Park, Vladlen Koltun, Qian-Yi Zhou: Tangent convolutions for dense prediction in 3d. CVPR 2018
PointNet++	0.339 56	0.584 45	0.478 56	0.458 53	0.256 57	0.360 58	0.250 50	0.247 55	0.278 57	0.261 54	0.677 57	0.183 56	0.117 49	0.212 56	0.145 57	0.364 53	0.346 58	0.232 58	0.548 54	0.523 57	0.252 56
Charles R. Qi, Li Yi, Hao Su, Leonidas J. Guibas: pointnet++: deep hierarchical feature learning on point sets in a metric space.
subcloud_weak	0.411 53	0.479 53	0.650 45	0.475 51	0.285 56	0.519 57	0.087 58	0.725 23	0.396 50	0.386 44	0.621 58	0.250 52	0.117 49	0.338 48	0.443 41	0.188 58	0.594 45	0.369 55	0.377 58	0.616 53	0.306 53

MCCNN	0.633 27	0.866 4	0.731 21	0.771 13	0.576 28	0.809 17	0.410 32	0.684 29	0.497 36	0.491 30	0.949 14	0.466 17	0.105 51	0.581 21	0.646 30	0.620 27	0.680 33	0.542 34	0.817 37	0.795 21	0.618 17
P. Hermosilla, T. Ritschel, P.P. Vazquez, A. Vinacua, T. Ropinski: Monte Carlo Convolution for Learning on Non-Uniformly Sampled Point Clouds. SIGGRAPH Asia 2018
ScanNet	0.306 58	0.203 58	0.366 57	0.501 49	0.311 54	0.524 56	0.211 56	0.002 59	0.342 53	0.189 57	0.786 56	0.145 58	0.102 52	0.245 51	0.152 56	0.318 56	0.348 57	0.300 57	0.460 57	0.437 58	0.182 58
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nießner: ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. CVPR'17
ScanNet+FTSDF	0.383 55	0.297 57	0.491 55	0.432 54	0.358 51	0.612 53	0.274 49	0.116 56	0.411 48	0.265 53	0.904 51	0.229 54	0.079 53	0.250 50	0.185 55	0.320 55	0.510 52	0.385 53	0.548 54	0.597 56	0.394 51

SSC-UNet	0.308 57	0.353 56	0.290 58	0.278 58	0.166 58	0.553 55	0.169 57	0.286 53	0.147 58	0.148 58	0.908 49	0.182 57	0.064 54	0.023 58	0.018 59	0.354 54	0.363 56	0.345 56	0.546 56	0.685 47	0.278 54

SegGCN	0.589 39	0.833 8	0.731 21	0.539 48	0.514 38	0.789 29	0.448 21	0.467 47	0.573 18	0.484 32	0.936 35	0.396 37	0.061 55	0.501 35	0.507 38	0.594 36	0.700 26	0.563 25	0.874 20	0.771 36	0.493 44
Huan Lei, Naveed Akhtar, and Ajmal Mian: SegGCN: Efficient 3D Point Cloud Segmentation with Fuzzy Spherical Kernel. CVPR 2020
SPH3D-GCN	0.610 35	0.858 6	0.772 10	0.489 50	0.532 36	0.792 26	0.404 35	0.643 39	0.570 20	0.507 26	0.935 36	0.414 34	0.046 56	0.510 32	0.702 25	0.602 33	0.705 24	0.549 32	0.859 30	0.773 35	0.534 40
Huan Lei, Naveed Akhtar, and Ajmal Mian: Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds. TPAMI 2020
FCPN	0.447 49	0.679 35	0.604 51	0.578 46	0.380 48	0.682 47	0.291 48	0.106 57	0.483 39	0.258 55	0.920 45	0.258 51	0.025 57	0.231 54	0.325 47	0.480 47	0.560 48	0.463 46	0.725 47	0.666 50	0.231 57
Dario Rethage, Johanna Wald, Jürgen Sturm, Nassir Navab, Federico Tombari: Fully-Convolutional Point Networks for Large-Scale Point Clouds. ECCV 2018
ERROR	0.054 59	0.000 59	0.041 59	0.172 59	0.030 59	0.062 59	0.001 59	0.035 58	0.004 59	0.051 59	0.143 59	0.019 59	0.003 58	0.041 57	0.050 58	0.003 59	0.054 59	0.018 59	0.005 59	0.264 59	0.082 59

SPLAT Net	0.393 54	0.472 54	0.511 54	0.606 42	0.311 54	0.656 48	0.245 52	0.405 49	0.328 54	0.197 56	0.927 41	0.227 55	0.000 59	0.001 59	0.249 50	0.271 57	0.510 52	0.383 54	0.593 53	0.699 46	0.267 55
Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, Jan Kautz: SPLATNet: Sparse Lattice Networks for Point Cloud Processing. CVPR 2018

This table lists the benchmark results for the 3D semantic instance scenario.

Method	avg ap 50%	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	window

Mask-Group	0.664 3	1.000 1	0.822 3	0.764 5	0.616 2	0.815 2	0.139 5	0.694 6	0.597 2	0.459 5	0.566 3	0.599 1	0.600 5	0.516 17	0.715 2	0.819 2	0.635 3	1.000 1	0.603 1

PE	0.645 4	1.000 1	0.773 7	0.798 2	0.538 4	0.786 8	0.088 11	0.799 3	0.350 9	0.435 9	0.547 5	0.545 2	0.646 4	0.933 6	0.562 6	0.761 5	0.556 11	0.997 7	0.501 5

CRNet	0.688 1	1.000 1	0.802 6	0.783 3	0.644 1	0.804 4	0.182 2	0.874 1	0.511 4	0.501 2	0.592 1	0.536 3	0.587 8	1.000 1	0.637 4	0.730 7	0.683 1	0.994 9	0.523 2

PointGroup	0.636 6	1.000 1	0.765 8	0.624 7	0.505 7	0.797 5	0.116 8	0.696 5	0.384 7	0.441 7	0.559 4	0.476 4	0.596 6	1.000 1	0.666 3	0.756 6	0.556 10	0.997 7	0.513 3
Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi-Wing Fu, Jiaya Jia: PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation. CVPR 2020 [oral]
OccuSeg+instance	0.672 2	1.000 1	0.758 10	0.682 6	0.576 3	0.842 1	0.477 1	0.504 14	0.524 3	0.567 1	0.585 2	0.451 5	0.557 9	1.000 1	0.751 1	0.797 3	0.563 7	1.000 1	0.467 7

PanopticFusion-inst	0.478 14	0.667 16	0.712 14	0.595 8	0.259 17	0.550 17	0.000 23	0.613 8	0.175 16	0.250 18	0.434 9	0.437 6	0.411 15	0.857 8	0.485 8	0.591 17	0.267 21	0.944 12	0.359 12
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
PCJC	0.578 8	1.000 1	0.810 4	0.583 11	0.449 11	0.813 3	0.042 14	0.603 9	0.341 10	0.490 3	0.465 7	0.410 7	0.650 2	0.835 12	0.264 17	0.694 9	0.561 8	0.889 15	0.504 4

SSEN	0.575 9	1.000 1	0.761 9	0.473 13	0.477 9	0.795 6	0.066 12	0.529 12	0.658 1	0.460 4	0.461 8	0.380 8	0.331 16	0.859 7	0.401 13	0.692 10	0.653 2	1.000 1	0.348 13
Dongsu Zhang, Junha Chun, Sang Kyun Cha, Young Min Kim: Spatial Semantic Embedding Network: Fast 3D Instance Segmentation with Deep Metric Learning. Arxiv
GICN	0.638 5	1.000 1	0.895 1	0.800 1	0.480 8	0.676 10	0.144 4	0.737 4	0.354 8	0.447 6	0.400 13	0.365 9	0.700 1	1.000 1	0.569 5	0.836 1	0.599 4	1.000 1	0.473 6

3D-MPA	0.611 7	1.000 1	0.833 2	0.765 4	0.526 5	0.756 9	0.136 7	0.588 11	0.470 5	0.438 8	0.432 11	0.358 10	0.650 2	0.857 8	0.429 10	0.765 4	0.557 9	1.000 1	0.430 9
Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, Matthias Nießner: 3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation. CVPR 2020
MASC	0.447 16	0.528 20	0.555 17	0.381 14	0.382 12	0.633 12	0.002 21	0.509 13	0.260 12	0.361 13	0.432 10	0.327 11	0.451 10	0.571 16	0.367 15	0.639 13	0.386 15	0.980 11	0.276 16
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
MaskRCNN 2d->3d Proj	0.058 24	0.333 23	0.002 24	0.000 24	0.053 23	0.002 24	0.002 22	0.021 24	0.000 23	0.045 24	0.024 24	0.238 12	0.065 23	0.000 23	0.014 24	0.107 24	0.020 24	0.110 24	0.006 24

MTML	0.549 10	1.000 1	0.807 5	0.588 10	0.327 14	0.647 11	0.004 20	0.815 2	0.180 14	0.418 10	0.364 15	0.182 13	0.445 11	1.000 1	0.442 9	0.688 11	0.571 6	1.000 1	0.396 10
Jean Lahoud, Bernard Ghanem, Marc Pollefeys, Martin R. Oswald: 3D Instance Segmentation via Multi-task Metric Learning. ICCV 2019 [oral]
Sparse R-CNN	0.515 11	1.000 1	0.538 18	0.282 17	0.468 10	0.790 7	0.173 3	0.345 16	0.429 6	0.413 12	0.484 6	0.176 14	0.595 7	0.591 15	0.522 7	0.668 12	0.476 14	0.986 10	0.327 14

Occipital-SCS	0.512 12	1.000 1	0.716 12	0.509 12	0.506 6	0.611 13	0.092 10	0.602 10	0.177 15	0.346 14	0.383 14	0.165 15	0.442 12	0.850 11	0.386 14	0.618 14	0.543 12	0.889 15	0.389 11

SALoss-ResNet	0.459 15	1.000 1	0.737 11	0.159 22	0.259 16	0.587 15	0.138 6	0.475 15	0.217 13	0.416 11	0.408 12	0.128 16	0.315 17	0.714 14	0.411 11	0.536 18	0.590 5	0.873 18	0.304 15
Zhidong Liang, Ming Yang, Hao Li, Chunxiang Wang: 3D Instance Embedding Learning With a Structure-Aware Loss Function for Point Cloud Segmentation. IEEE Robotics and Automation Letters (IROS2020)
3D-BoNet	0.488 13	1.000 1	0.672 15	0.590 9	0.301 15	0.484 19	0.098 9	0.620 7	0.306 11	0.341 15	0.259 17	0.125 17	0.434 13	0.796 13	0.402 12	0.499 19	0.513 13	0.909 14	0.439 8
Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, Niki Trigoni: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds. NeurIPS 2019 Spotlight
UNet-backbone	0.319 18	0.667 16	0.715 13	0.233 19	0.189 19	0.479 20	0.008 19	0.218 19	0.067 21	0.201 19	0.173 19	0.107 18	0.123 20	0.438 18	0.150 18	0.615 15	0.355 16	0.916 13	0.093 23

3D-SIS	0.382 17	1.000 1	0.432 20	0.245 18	0.190 18	0.577 16	0.013 18	0.263 18	0.033 22	0.320 16	0.240 18	0.075 19	0.422 14	0.857 8	0.117 20	0.699 8	0.271 20	0.883 17	0.235 18
Ji Hou, Angela Dai, Matthias Niessner: 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans. CVPR 2019
Region	0.248 20	0.667 16	0.437 19	0.188 20	0.153 20	0.491 18	0.000 23	0.208 20	0.094 19	0.153 20	0.099 21	0.057 20	0.217 19	0.119 22	0.039 23	0.466 20	0.302 18	0.640 21	0.140 20

R-PointNet	0.306 19	0.500 21	0.405 21	0.311 15	0.348 13	0.589 14	0.054 13	0.068 22	0.126 17	0.283 17	0.290 16	0.028 21	0.219 18	0.214 21	0.331 16	0.396 21	0.275 19	0.821 20	0.245 17

3D-BEVIS	0.248 20	0.667 16	0.566 16	0.076 23	0.035 24	0.394 21	0.027 16	0.035 23	0.098 18	0.099 22	0.030 23	0.025 22	0.098 21	0.375 19	0.126 19	0.604 16	0.181 22	0.854 19	0.171 19
Cathrin Elich, Francis Engelmann, Jonas Schult, Theodora Kontogianni, Bastian Leibe: 3D-BEVIS: Birds-Eye-View Instance Segmentation.
Sgpn_scannet	0.143 23	0.208 24	0.390 22	0.169 21	0.065 22	0.275 23	0.029 15	0.069 21	0.000 23	0.087 23	0.043 22	0.014 23	0.027 24	0.000 23	0.112 22	0.351 22	0.168 23	0.438 23	0.138 21

Seg-Cluster	0.215 22	0.370 22	0.337 23	0.285 16	0.105 21	0.325 22	0.025 17	0.282 17	0.085 20	0.105 21	0.107 20	0.007 24	0.079 22	0.317 20	0.114 21	0.309 23	0.304 17	0.587 22	0.123 22

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

Virtual MVFusion (R)	0.745 1	0.861 1	0.839 1	0.881 1	0.672 1	0.512 1	0.422 10	0.898 1	0.723 1	0.714 1	0.954 2	0.454 1	0.509 1	0.773 1	0.895 1	0.756 1	0.820 1	0.653 1	0.935 1	0.891 1	0.728 1
Abhijit Kundu, Xiaoqi Yin, Alireza Fathi, David Ross, Brian Brewington, Thomas Funkhouser, Caroline Pantofaru: Virtual Multi-view Fusion for 3D Semantic Segmentation. ECCV 2020
DCRedNet	0.583 8	0.682 5	0.723 8	0.542 10	0.510 11	0.310 11	0.451 6	0.668 8	0.549 8	0.520 6	0.920 5	0.375 4	0.446 2	0.528 11	0.417 9	0.670 9	0.577 13	0.478 8	0.862 4	0.806 5	0.628 5

FuseNet	0.535 11	0.570 10	0.681 12	0.182 15	0.512 10	0.290 13	0.431 9	0.659 9	0.504 12	0.495 9	0.903 9	0.308 10	0.428 3	0.523 12	0.365 11	0.676 6	0.621 10	0.470 9	0.762 13	0.779 7	0.541 9
Caner Hazirbas, Lingni Ma, Csaba Domokos, Daniel Cremers: FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-based CNN Architecture. ACCV 2016
BPNet_2D	0.670 2	0.822 3	0.795 3	0.836 2	0.659 2	0.481 2	0.451 6	0.769 2	0.656 3	0.567 3	0.931 3	0.395 3	0.390 4	0.700 2	0.534 3	0.689 5	0.770 2	0.574 3	0.865 3	0.831 3	0.675 2

CU-Hybrid-2D Net	0.636 3	0.825 2	0.820 2	0.179 16	0.648 3	0.463 3	0.549 1	0.742 3	0.676 2	0.628 2	0.961 1	0.420 2	0.379 5	0.684 3	0.381 10	0.732 2	0.723 3	0.599 2	0.827 8	0.851 2	0.634 3

MCA-Net	0.595 6	0.533 12	0.756 5	0.746 5	0.590 5	0.334 8	0.506 3	0.670 7	0.587 6	0.500 8	0.905 8	0.366 5	0.352 6	0.601 5	0.506 6	0.669 10	0.648 6	0.501 5	0.839 7	0.769 9	0.516 12

SSMA	0.577 9	0.695 4	0.716 10	0.439 12	0.563 8	0.314 10	0.444 8	0.719 5	0.551 7	0.503 7	0.887 10	0.346 9	0.348 7	0.603 4	0.353 12	0.709 3	0.600 11	0.457 10	0.901 2	0.786 6	0.599 7
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. International Journal of Computer Vision, 2019
RFBNet	0.592 7	0.616 7	0.758 4	0.659 6	0.581 7	0.330 9	0.469 5	0.655 10	0.543 9	0.524 5	0.924 4	0.355 7	0.336 8	0.572 8	0.479 8	0.671 8	0.648 6	0.480 7	0.814 10	0.814 4	0.614 6

DMMF	0.597 5	0.543 11	0.755 6	0.749 4	0.585 6	0.338 6	0.494 4	0.704 6	0.598 4	0.494 10	0.911 6	0.347 8	0.327 9	0.593 7	0.527 4	0.675 7	0.646 8	0.513 4	0.842 6	0.774 8	0.527 11

DMMF_3d	0.605 4	0.651 6	0.744 7	0.782 3	0.637 4	0.387 4	0.536 2	0.732 4	0.590 5	0.540 4	0.856 13	0.359 6	0.306 10	0.596 6	0.539 2	0.627 12	0.706 4	0.497 6	0.785 12	0.757 11	0.476 13

MSeg1080_RVC	0.485 14	0.505 13	0.709 11	0.092 17	0.427 14	0.241 14	0.411 11	0.654 11	0.385 17	0.457 11	0.861 12	0.053 17	0.279 11	0.503 13	0.481 7	0.645 11	0.626 9	0.365 15	0.748 15	0.725 13	0.529 10
John Lambert, Zhuang Liu, Ozan Sener, James Hays, Vladlen Koltun: MSeg: A Composite Dataset for Multi-domain Semantic Segmentation. CVPR 2020
ILC-PSPNet	0.475 15	0.490 14	0.581 15	0.289 14	0.507 12	0.067 17	0.379 13	0.610 14	0.417 15	0.435 13	0.822 16	0.278 12	0.267 12	0.503 13	0.228 14	0.616 14	0.533 15	0.375 14	0.820 9	0.729 12	0.560 8

SN_RN152pyrx8_RVC	0.546 10	0.572 9	0.663 13	0.638 8	0.518 9	0.298 12	0.366 15	0.633 12	0.510 11	0.446 12	0.864 11	0.296 11	0.267 12	0.542 10	0.346 13	0.704 4	0.575 14	0.431 12	0.853 5	0.766 10	0.630 4

3DMV (2d proj)	0.498 13	0.481 15	0.612 14	0.579 9	0.456 13	0.343 5	0.384 12	0.623 13	0.525 10	0.381 14	0.845 14	0.254 13	0.264 14	0.557 9	0.182 15	0.581 15	0.598 12	0.429 13	0.760 14	0.661 16	0.446 15
Angela Dai, Matthias Niessner: 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation. ECCV'18
Enet (reimpl)	0.376 16	0.264 17	0.452 17	0.452 11	0.365 15	0.181 15	0.143 17	0.456 16	0.409 16	0.346 16	0.769 17	0.164 15	0.218 15	0.359 16	0.123 17	0.403 17	0.381 17	0.313 17	0.571 16	0.685 15	0.472 14
Re-implementation of Adam Paszke, Abhishek Chaurasia, Sangpil Kim, Eugenio Culurciello: ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation.
AdapNet++	0.503 12	0.613 8	0.722 9	0.418 13	0.358 17	0.337 7	0.370 14	0.479 15	0.443 13	0.368 15	0.907 7	0.207 14	0.213 16	0.464 15	0.525 5	0.618 13	0.657 5	0.450 11	0.788 11	0.721 14	0.408 16
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. International Journal of Computer Vision, 2019
ScanNet (2d proj)	0.330 17	0.293 16	0.521 16	0.657 7	0.361 16	0.161 16	0.250 16	0.004 17	0.440 14	0.183 17	0.836 15	0.125 16	0.060 17	0.319 17	0.132 16	0.417 16	0.412 16	0.344 16	0.541 17	0.427 17	0.109 17
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nießner: ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. CVPR'17

Method	avg ap	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	window

MaskRCNN_ScanNet	0.119 2	0.129 2	0.212 2	0.002 2	0.112 2	0.148 2	0.014 2	0.205 2	0.044 2	0.066 2	0.078 2	0.095 1	0.142 2	0.030 2	0.128 2	0.139 2	0.080 2	0.459 2	0.057 2
Re-implementation of Kaiming He, Georgia Gkioxari, Piotr Dollár, Ross Girshick: Mask R-CNN. ICCV'17
UniDet_RVC	0.205 1	0.381 1	0.323 1	0.037 1	0.226 1	0.177 1	0.063 1	0.277 1	0.120 1	0.067 1	0.131 1	0.074 2	0.317 1	0.080 1	0.235 1	0.289 1	0.141 1	0.678 1	0.080 1

Method	avg recall	apartment	bathroom	bedroom / hotel	bookstore / library	conference room	copy/mail room	hallway	kitchen	laundry room	living room / lounge	misc	office	storage / basement / garage

multi-task	0.700 1	0.500 1	1.000 1	0.882 2	0.500 1	1.000 1	1.000 1	0.500 1	1.000 1	1.000 1	0.778 1	0.000 2	0.938 1	0.000 1
Shengyu Huang, Mikhail Usvyatsov, Konrad Schindler: Indoor Scene Recognition in 3D. IROS 2020
SE-ResNeXt-SSMA	0.498 2	0.000 3	0.812 2	0.941 1	0.500 1	0.500 2	0.500 2	0.500 1	0.429 3	0.500 2	0.667 2	0.500 1	0.625 2	0.000 1
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. arXiv
resnet50_scannet	0.353 3	0.250 2	0.812 2	0.529 3	0.500 1	0.500 2	0.000 3	0.500 1	0.571 2	0.000 3	0.556 3	0.000 2	0.375 3	0.000 1

3D Semantic label benchmark

3D Semantic instance benchmark

2D Semantic label benchmark

2D Semantic instance benchmark

Scene type classification benchmark