3D Semantic Label Results - ScanNet Benchmark

The 3D semantic labeling task involves predicting a semantic labeling of a 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the PASCAL VOC intersection-over-union metric (IoU). IoU = TP/(TP+FP+FN), where TP, FP, and FN are the numbers of true positive, false positive, and false negative pixels, respectively. Predicted labels are evaluated per-vertex over the respective 3D scan mesh; for 3D approaches that operate on other representations like grids or points, the predicted labels should be mapped onto the mesh vertices (e.g., one such example for grid to mesh vertices is provided in the evaluation helpers).

This table lists the benchmark results for the 3D semantic label scenario.

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

Mix3D	0.781 1	0.964 1	0.855 1	0.843 5	0.781 1	0.858 4	0.575 2	0.831 9	0.685 3	0.714 1	0.979 1	0.594 2	0.310 10	0.801 1	0.892 3	0.841 2	0.819 3	0.723 2	0.940 3	0.887 1	0.725 5
Alexey Nekrasov, Jonas Schult, Or Litany, Bastian Leibe, Francis Engelmann: Mix3D: Out-of-Context Data Augmentation for 3D Scenes. 3DV 2021 (Oral)
OccuSeg+Semantic	0.764 2	0.758 34	0.796 9	0.839 6	0.746 4	0.907 1	0.562 3	0.850 5	0.680 4	0.672 2	0.978 2	0.610 1	0.335 4	0.777 2	0.819 18	0.847 1	0.830 1	0.691 5	0.972 1	0.885 2	0.727 3

O-CNN	0.762 3	0.924 2	0.823 3	0.844 4	0.770 2	0.852 5	0.577 1	0.847 6	0.711 1	0.640 7	0.958 5	0.592 3	0.217 38	0.762 5	0.888 4	0.758 5	0.813 5	0.726 1	0.932 8	0.868 3	0.744 1
Peng-Shuai Wang, Yang Liu, Yu-Xiao Guo, Chun-Yu Sun, Xin Tong: O-CNN: Octree-based Convolutional Neural Networks for 3D Shape Analysis. SIGGRAPH 2017
BPNet	0.749 4	0.909 3	0.818 6	0.811 10	0.752 3	0.839 8	0.485 16	0.842 8	0.673 5	0.644 5	0.957 6	0.528 10	0.305 12	0.773 3	0.859 6	0.788 3	0.818 4	0.693 4	0.916 9	0.856 7	0.723 6
Wenbo Hu, Hengshuang Zhao, Li Jiang, Jiaya Jia, Tien-Tsin Wong: Bidirectional Projection Network for Cross Dimension Scene Understanding. CVPR 2021 (Oral)
VMNet	0.746 5	0.870 8	0.838 2	0.858 1	0.729 6	0.850 6	0.501 10	0.874 2	0.587 24	0.658 4	0.956 7	0.564 6	0.299 13	0.765 4	0.900 1	0.716 11	0.812 6	0.631 14	0.939 4	0.858 6	0.709 7
Zeyu HU, Xuyang Bai, Jiaxiang Shang, Runze Zhang, Jiayu Dong, Xin Wang, Guangyuan Sun, Hongbo Fu, Chiew-Lan Tai: VMNet: Voxel-Mesh Network for Geodesic-Aware 3D Semantic Segmentation. ICCV 2021 (Oral)
Virtual MVFusion	0.746 5	0.771 29	0.819 5	0.848 2	0.702 10	0.865 3	0.397 49	0.899 1	0.699 2	0.664 3	0.948 23	0.588 4	0.330 5	0.746 8	0.851 11	0.764 4	0.796 9	0.704 3	0.935 6	0.866 4	0.728 2
Abhijit Kundu, Xiaoqi Yin, Alireza Fathi, David Ross, Brian Brewington, Thomas Funkhouser, Caroline Pantofaru: Virtual Multi-view Fusion for 3D Semantic Segmentation. ECCV 2020
MinkowskiNet	0.736 7	0.859 10	0.818 6	0.832 7	0.709 9	0.840 7	0.521 7	0.853 4	0.660 6	0.643 6	0.951 14	0.544 7	0.286 18	0.731 9	0.893 2	0.675 22	0.772 16	0.683 6	0.874 35	0.852 9	0.727 3
C. Choy, J. Gwak, S. Savarese: 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks. CVPR 2019
SparseConvNet	0.725 8	0.647 54	0.821 4	0.846 3	0.721 7	0.869 2	0.533 5	0.754 25	0.603 21	0.614 9	0.955 8	0.572 5	0.325 7	0.710 10	0.870 5	0.724 10	0.823 2	0.628 15	0.934 7	0.865 5	0.683 10

MatchingNet	0.724 9	0.812 20	0.812 8	0.810 11	0.735 5	0.834 10	0.495 14	0.860 3	0.572 30	0.602 15	0.954 9	0.512 12	0.280 19	0.757 6	0.845 14	0.725 9	0.780 13	0.606 22	0.937 5	0.851 10	0.700 9

RFCR	0.702 10	0.889 4	0.745 27	0.813 9	0.672 14	0.818 21	0.493 15	0.815 12	0.623 12	0.610 10	0.947 26	0.470 23	0.249 30	0.594 23	0.848 12	0.705 15	0.779 14	0.646 9	0.892 22	0.823 18	0.611 28
Jingyu Gong, Jiachen Xu, Xin Tan, Haichuan Song, Yanyun Qu, Yuan Xie, Lizhuang Ma: Omni-Supervised Point Cloud Segmentation via Gradual Receptive Field Component Reasoning. CVPR2021
One Thing One Click	0.701 11	0.825 18	0.796 9	0.723 31	0.716 8	0.832 11	0.433 38	0.816 10	0.634 10	0.609 11	0.969 4	0.418 47	0.344 2	0.559 35	0.833 15	0.715 12	0.808 7	0.560 38	0.902 14	0.847 11	0.680 11

JSENet	0.699 12	0.881 7	0.762 18	0.821 8	0.667 15	0.800 35	0.522 6	0.792 17	0.613 14	0.607 12	0.935 50	0.492 16	0.205 42	0.576 29	0.853 8	0.691 17	0.758 23	0.652 8	0.872 38	0.828 15	0.649 17
Zeyu HU, Mingmin Zhen, Xuyang BAI, Hongbo Fu, Chiew-lan Tai: JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds. ECCV 2020
CU-Hybrid Net	0.693 13	0.596 60	0.789 12	0.803 13	0.677 13	0.800 35	0.469 20	0.846 7	0.554 38	0.591 18	0.948 23	0.500 14	0.316 9	0.609 20	0.847 13	0.732 6	0.808 7	0.593 28	0.894 20	0.839 12	0.652 16

One-Thing-One-Click	0.693 13	0.743 37	0.794 11	0.655 51	0.684 12	0.822 18	0.497 13	0.719 36	0.622 13	0.617 8	0.977 3	0.447 36	0.339 3	0.750 7	0.664 40	0.703 16	0.790 11	0.596 25	0.946 2	0.855 8	0.647 18
Zhengzhe Liu, Xiaojuan Qi, Chi-Wing Fu: One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation. CVPR 2021
Feature-Geometry Net	0.690 15	0.884 5	0.754 23	0.795 16	0.647 20	0.818 21	0.422 39	0.802 15	0.612 15	0.604 13	0.945 31	0.462 26	0.189 48	0.563 33	0.853 8	0.726 7	0.765 18	0.632 12	0.904 11	0.821 20	0.606 32
Kangcheng Liu, Ben M. Chen: https://arxiv.org/abs/2012.09439. arXiv Preprint
Feature_GeometricNet	0.690 15	0.884 5	0.754 23	0.795 16	0.647 20	0.818 21	0.422 39	0.802 15	0.612 15	0.604 13	0.945 31	0.462 26	0.189 48	0.563 33	0.853 8	0.726 7	0.765 18	0.632 12	0.904 11	0.821 20	0.606 32
Kangcheng Liu, Ben M. Chen: https://arxiv.org/abs/2012.09439. arXiv Preprint
FusionNet	0.688 17	0.704 47	0.741 30	0.754 28	0.656 16	0.829 13	0.501 10	0.741 30	0.609 18	0.548 25	0.950 18	0.522 11	0.371 1	0.633 18	0.756 24	0.715 12	0.771 17	0.623 16	0.861 43	0.814 23	0.658 14
Feihu Zhang, Jin Fang, Benjamin Wah, Philip Torr: Deep FusionNet for Point Cloud Semantic Segmentation. ECCV 2020
VACNN++	0.684 18	0.728 43	0.757 22	0.776 21	0.690 11	0.804 33	0.464 24	0.816 10	0.577 28	0.587 19	0.945 31	0.508 13	0.276 21	0.671 11	0.710 32	0.663 27	0.750 26	0.589 30	0.881 29	0.832 14	0.653 15

KP-FCNN	0.684 18	0.847 13	0.758 21	0.784 19	0.647 20	0.814 25	0.473 18	0.772 20	0.605 19	0.594 17	0.935 50	0.450 34	0.181 52	0.587 24	0.805 20	0.690 18	0.785 12	0.614 18	0.882 27	0.819 22	0.632 23
H. Thomas, C. Qi, J. Deschaud, B. Marcotegui, F. Goulette, L. Guibas.: KPConv: Flexible and Deformable Convolution for Point Clouds. ICCV 2019
Superpoint Network	0.683 20	0.851 12	0.728 36	0.800 15	0.653 18	0.806 31	0.468 21	0.804 13	0.572 30	0.602 15	0.946 28	0.453 33	0.239 33	0.519 43	0.822 16	0.689 20	0.762 21	0.595 27	0.895 19	0.827 16	0.630 24

PointContrast_LA_SEM	0.683 20	0.757 35	0.784 13	0.786 18	0.639 24	0.824 17	0.408 44	0.775 19	0.604 20	0.541 27	0.934 54	0.532 8	0.269 24	0.552 36	0.777 22	0.645 36	0.793 10	0.640 10	0.913 10	0.824 17	0.671 12

VI-PointConv	0.676 22	0.770 31	0.754 23	0.783 20	0.621 27	0.814 25	0.552 4	0.758 23	0.571 32	0.557 23	0.954 9	0.529 9	0.268 26	0.530 41	0.682 37	0.675 22	0.719 32	0.603 23	0.888 24	0.833 13	0.665 13
Xingyi Li, Wenxuan Wu, Xiaoli Z. Fern, Li Fuxin: The Devils in the Point Clouds: Studying the Robustness of Point Cloud Convolutions.
ROSMRF3D	0.673 23	0.789 22	0.748 26	0.763 26	0.635 25	0.814 25	0.407 46	0.747 27	0.581 27	0.573 20	0.950 18	0.484 17	0.271 23	0.607 21	0.754 25	0.649 32	0.774 15	0.596 25	0.883 26	0.823 18	0.606 32

SALANet	0.670 24	0.816 19	0.770 17	0.768 24	0.652 19	0.807 30	0.451 26	0.747 27	0.659 7	0.545 26	0.924 57	0.473 22	0.149 63	0.571 31	0.811 19	0.635 39	0.746 27	0.623 16	0.892 22	0.794 35	0.570 45

PointASNL	0.666 25	0.703 48	0.781 14	0.751 30	0.655 17	0.830 12	0.471 19	0.769 21	0.474 53	0.537 28	0.951 14	0.475 21	0.279 20	0.635 16	0.698 36	0.675 22	0.751 25	0.553 43	0.816 53	0.806 26	0.703 8
Xu Yan, Chaoda Zheng, Zhen Li, Sheng Wang, Shuguang Cui: PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Sampling. CVPR 2020
PointConv	0.666 25	0.781 24	0.759 20	0.699 37	0.644 23	0.822 18	0.475 17	0.779 18	0.564 35	0.504 40	0.953 11	0.428 42	0.203 44	0.586 26	0.754 25	0.661 28	0.753 24	0.588 31	0.902 14	0.813 25	0.642 19
Wenxuan Wu, Zhongang Qi, Li Fuxin: PointConv: Deep Convolutional Networks on 3D Point Clouds. CVPR 2019
DCM-Net	0.658 27	0.778 25	0.702 41	0.806 12	0.619 28	0.813 28	0.468 21	0.693 43	0.494 49	0.524 33	0.941 41	0.449 35	0.298 14	0.510 45	0.821 17	0.675 22	0.727 31	0.568 35	0.826 50	0.803 28	0.637 21
Jonas Schult, Francis Engelmann, Theodora Kontogianni, Bastian Leibe: DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshes. CVPR 2020 [Oral]
HPGCNN	0.656 28	0.698 49	0.743 29	0.650 52	0.564 43	0.820 20	0.505 9	0.758 23	0.631 11	0.479 46	0.945 31	0.480 19	0.226 34	0.572 30	0.774 23	0.690 18	0.735 30	0.614 18	0.853 46	0.776 47	0.597 38
Jisheng Dang, Qingyong Hu, Yulan Guo, Jun Yang: HPGCNN.
SAFNet-seg	0.654 29	0.752 36	0.734 32	0.664 49	0.583 37	0.815 24	0.399 48	0.754 25	0.639 9	0.535 29	0.942 39	0.470 23	0.309 11	0.665 12	0.539 47	0.650 31	0.708 37	0.635 11	0.857 45	0.793 36	0.642 19
Linqing Zhao, Jiwen Lu, Jie Zhou: Similarity-Aware Fusion Network for 3D Semantic Segmentation. IROS 2021
RandLA-Net	0.645 30	0.778 25	0.731 33	0.699 37	0.577 38	0.829 13	0.446 30	0.736 31	0.477 52	0.523 35	0.945 31	0.454 31	0.269 24	0.484 53	0.749 28	0.618 43	0.738 28	0.599 24	0.827 49	0.792 38	0.621 26

MVPNet	0.641 31	0.831 15	0.715 37	0.671 46	0.590 34	0.781 45	0.394 50	0.679 46	0.642 8	0.553 24	0.937 47	0.462 26	0.256 27	0.649 13	0.406 60	0.626 40	0.691 44	0.666 7	0.877 31	0.792 38	0.608 31
Maximilian Jaritz, Jiayuan Gu, Hao Su: Multi-view PointNet for 3D Scene Understanding. GMDL Workshop, ICCV 2019
PointConv-SFPN	0.641 31	0.776 27	0.703 40	0.721 32	0.557 46	0.826 15	0.451 26	0.672 48	0.563 36	0.483 45	0.943 38	0.425 45	0.162 58	0.644 14	0.726 29	0.659 29	0.709 36	0.572 33	0.875 33	0.786 42	0.559 50

PointMRNet	0.640 33	0.717 46	0.701 42	0.692 39	0.576 39	0.801 34	0.467 23	0.716 37	0.563 36	0.459 50	0.953 11	0.429 41	0.169 55	0.581 27	0.854 7	0.605 44	0.710 35	0.550 44	0.894 20	0.793 36	0.575 43

FPConv	0.639 34	0.785 23	0.760 19	0.713 36	0.603 31	0.798 38	0.392 51	0.534 62	0.603 21	0.524 33	0.948 23	0.457 29	0.250 29	0.538 39	0.723 30	0.598 47	0.696 43	0.614 18	0.872 38	0.799 29	0.567 47
Yiqun Lin, Zizheng Yan, Haibin Huang, Dong Du, Ligang Liu, Shuguang Cui, Xiaoguang Han: FPConv: Learning Local Flattening for Point Convolution. CVPR 2020
PointSPNet	0.637 35	0.734 40	0.692 47	0.714 35	0.576 39	0.797 39	0.446 30	0.743 29	0.598 23	0.437 54	0.942 39	0.403 49	0.150 62	0.626 19	0.800 21	0.649 32	0.697 42	0.557 41	0.846 47	0.777 46	0.563 48

SConv	0.636 36	0.830 16	0.697 45	0.752 29	0.572 42	0.780 46	0.445 32	0.716 37	0.529 42	0.530 30	0.951 14	0.446 37	0.170 54	0.507 47	0.666 39	0.636 38	0.682 46	0.541 49	0.886 25	0.799 29	0.594 39

Supervoxel-CNN	0.635 37	0.656 52	0.711 38	0.719 33	0.613 29	0.757 52	0.444 35	0.765 22	0.534 41	0.566 21	0.928 55	0.478 20	0.272 22	0.636 15	0.531 49	0.664 26	0.645 54	0.508 53	0.864 42	0.792 38	0.611 28

joint point-based	0.634 38	0.614 57	0.778 15	0.667 48	0.633 26	0.825 16	0.420 41	0.804 13	0.467 55	0.561 22	0.951 14	0.494 15	0.291 15	0.566 32	0.458 54	0.579 52	0.764 20	0.559 40	0.838 48	0.814 23	0.598 37
Hung-Yueh Chiang, Yen-Liang Lin, Yueh-Cheng Liu, Winston H. Hsu: A Unified Point-Based Framework for 3D Segmentation. 3DV 2019
MCCNN	0.633 39	0.866 9	0.731 33	0.771 22	0.576 39	0.809 29	0.410 43	0.684 44	0.497 48	0.491 42	0.949 20	0.466 25	0.105 68	0.581 27	0.646 41	0.620 41	0.680 47	0.542 48	0.817 52	0.795 33	0.618 27
P. Hermosilla, T. Ritschel, P.P. Vazquez, A. Vinacua, T. Ropinski: Monte Carlo Convolution for Learning on Non-Uniformly Sampled Point Clouds. SIGGRAPH Asia 2018
PointMTL	0.632 40	0.731 41	0.688 50	0.675 43	0.591 33	0.784 44	0.444 35	0.565 60	0.610 17	0.492 41	0.949 20	0.456 30	0.254 28	0.587 24	0.706 33	0.599 46	0.665 51	0.612 21	0.868 41	0.791 41	0.579 42

PointNet2-SFPN	0.631 41	0.771 29	0.692 47	0.672 44	0.524 50	0.837 9	0.440 37	0.706 41	0.538 40	0.446 52	0.944 36	0.421 46	0.219 37	0.552 36	0.751 27	0.591 49	0.737 29	0.543 47	0.901 16	0.768 50	0.557 51

3DSM_DMMF	0.631 41	0.626 56	0.745 27	0.801 14	0.607 30	0.751 53	0.506 8	0.729 34	0.565 34	0.491 42	0.866 68	0.434 38	0.197 46	0.595 22	0.630 42	0.709 14	0.705 39	0.560 38	0.875 33	0.740 57	0.491 60

APCF-Net	0.631 41	0.742 38	0.687 52	0.672 44	0.557 46	0.792 41	0.408 44	0.665 49	0.545 39	0.508 38	0.952 13	0.428 42	0.186 50	0.634 17	0.702 34	0.620 41	0.706 38	0.555 42	0.873 37	0.798 31	0.581 41
Haojia, Lin: Adaptive Pyramid Context Fusion for Point Cloud Perception. GRSL
FusionAwareConv	0.630 44	0.604 59	0.741 30	0.766 25	0.590 34	0.747 54	0.501 10	0.734 32	0.503 47	0.527 31	0.919 61	0.454 31	0.323 8	0.550 38	0.420 59	0.678 21	0.688 45	0.544 46	0.896 18	0.795 33	0.627 25
Jiazhao Zhang, Chenyang Zhu, Lintao Zheng, Kai Xu: Fusion-Aware Point Convolution for Online Semantic 3D Scene Segmentation. CVPR 2020
DenSeR	0.628 45	0.800 21	0.625 64	0.719 33	0.545 48	0.806 31	0.445 32	0.597 56	0.448 59	0.519 36	0.938 46	0.481 18	0.328 6	0.489 52	0.499 53	0.657 30	0.759 22	0.592 29	0.881 29	0.797 32	0.634 22

SIConv	0.625 46	0.830 16	0.694 46	0.757 27	0.563 44	0.772 48	0.448 28	0.647 52	0.520 44	0.509 37	0.949 20	0.431 40	0.191 47	0.496 50	0.614 43	0.647 35	0.672 49	0.535 51	0.876 32	0.783 43	0.571 44

HPEIN	0.618 47	0.729 42	0.668 53	0.647 54	0.597 32	0.766 49	0.414 42	0.680 45	0.520 44	0.525 32	0.946 28	0.432 39	0.215 39	0.493 51	0.599 44	0.638 37	0.617 59	0.570 34	0.897 17	0.806 26	0.605 35
Li Jiang, Hengshuang Zhao, Shu Liu, Xiaoyong Shen, Chi-Wing Fu, Jiaya Jia: Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentation. ICCV 2019
SPH3D-GCN	0.610 48	0.858 11	0.772 16	0.489 67	0.532 49	0.792 41	0.404 47	0.643 53	0.570 33	0.507 39	0.935 50	0.414 48	0.046 73	0.510 45	0.702 34	0.602 45	0.705 39	0.549 45	0.859 44	0.773 48	0.534 53
Huan Lei, Naveed Akhtar, and Ajmal Mian: Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds. TPAMI 2020
AttAN	0.609 49	0.760 33	0.667 54	0.649 53	0.521 51	0.793 40	0.457 25	0.648 51	0.528 43	0.434 56	0.947 26	0.401 50	0.153 61	0.454 54	0.721 31	0.648 34	0.717 33	0.536 50	0.904 11	0.765 51	0.485 61
Gege Zhang, Qinghua Ma, Licheng Jiao, Fang Liu and Qigong Sun: AttAN: Attention Adversarial Networks for 3D Point Cloud Semantic Segmentation. IJCAI2020
LAP-D	0.594 50	0.720 44	0.692 47	0.637 56	0.456 58	0.773 47	0.391 53	0.730 33	0.587 24	0.445 53	0.940 43	0.381 53	0.288 16	0.434 57	0.453 55	0.591 49	0.649 52	0.581 32	0.777 57	0.749 56	0.610 30

DPC	0.592 51	0.720 44	0.700 43	0.602 59	0.480 55	0.762 51	0.380 56	0.713 39	0.585 26	0.437 54	0.940 43	0.369 55	0.288 16	0.434 57	0.509 51	0.590 51	0.639 57	0.567 36	0.772 58	0.755 54	0.592 40
Francis Engelmann, Theodora Kontogianni, Bastian Leibe: Dilated Point Convolutions: On the Receptive Field Size of Point Convolutions on 3D Point Clouds. ICRA 2020
CCRFNet	0.589 52	0.766 32	0.659 57	0.683 41	0.470 57	0.740 56	0.387 55	0.620 55	0.490 50	0.476 47	0.922 59	0.355 59	0.245 31	0.511 44	0.511 50	0.571 53	0.643 55	0.493 56	0.872 38	0.762 52	0.600 36

SegGCN	0.589 52	0.833 14	0.731 33	0.539 64	0.514 52	0.789 43	0.448 28	0.467 64	0.573 29	0.484 44	0.936 48	0.396 51	0.061 72	0.501 48	0.507 52	0.594 48	0.700 41	0.563 37	0.874 35	0.771 49	0.493 59
Huan Lei, Naveed Akhtar, and Ajmal Mian: SegGCN: Efficient 3D Point Cloud Segmentation with Fuzzy Spherical Kernel. CVPR 2020
ROSMRF	0.580 54	0.772 28	0.707 39	0.681 42	0.563 44	0.764 50	0.362 57	0.515 63	0.465 56	0.465 49	0.936 48	0.427 44	0.207 41	0.438 55	0.577 45	0.536 57	0.675 48	0.486 57	0.723 62	0.779 44	0.524 55

TextureNet	0.566 55	0.672 51	0.664 55	0.671 46	0.494 53	0.719 57	0.445 32	0.678 47	0.411 64	0.396 59	0.935 50	0.356 58	0.225 35	0.412 59	0.535 48	0.565 54	0.636 58	0.464 60	0.794 56	0.680 64	0.568 46
Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkerhouser, Matthias Niessner, Leonidas Guibas: TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshes. CVPR
DVVNet	0.562 56	0.648 53	0.700 43	0.770 23	0.586 36	0.687 61	0.333 60	0.650 50	0.514 46	0.475 48	0.906 65	0.359 57	0.223 36	0.340 63	0.442 58	0.422 66	0.668 50	0.501 54	0.708 63	0.779 44	0.534 53

Pointnet++ & Feature	0.557 57	0.735 39	0.661 56	0.686 40	0.491 54	0.744 55	0.392 51	0.539 61	0.451 58	0.375 62	0.946 28	0.376 54	0.205 42	0.403 60	0.356 63	0.553 56	0.643 55	0.497 55	0.824 51	0.756 53	0.515 56

PointMRNet-lite	0.553 58	0.633 55	0.648 59	0.659 50	0.430 60	0.800 35	0.390 54	0.592 57	0.454 57	0.371 63	0.939 45	0.368 56	0.136 65	0.368 61	0.448 56	0.560 55	0.715 34	0.486 57	0.882 27	0.720 59	0.462 64

PanopticFusion-label	0.529 59	0.491 68	0.688 50	0.604 58	0.386 63	0.632 66	0.225 71	0.705 42	0.434 62	0.293 67	0.815 69	0.348 60	0.241 32	0.499 49	0.669 38	0.507 59	0.649 52	0.442 65	0.796 55	0.602 70	0.561 49
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
Online SegFusion	0.515 60	0.607 58	0.644 62	0.579 61	0.434 59	0.630 67	0.353 58	0.628 54	0.440 60	0.410 57	0.762 72	0.307 62	0.167 56	0.520 42	0.403 61	0.516 58	0.565 62	0.447 64	0.678 65	0.701 61	0.514 57
Davide Menini, Suryansh Kumar, Martin R. Oswald, Erik Sandstroem, Cristian Sminchisescu, Luc van Gool: A Real-Time Learning Framework for Joint 3D Reconstruction and Semantic Segmentation. Robotics and Automation Letters Submission
3DMV, FTSDF	0.501 61	0.558 64	0.608 67	0.424 72	0.478 56	0.690 60	0.246 67	0.586 58	0.468 54	0.450 51	0.911 63	0.394 52	0.160 59	0.438 55	0.212 69	0.432 65	0.541 67	0.475 59	0.742 60	0.727 58	0.477 62

PCNN	0.498 62	0.559 63	0.644 62	0.560 63	0.420 62	0.711 59	0.229 69	0.414 65	0.436 61	0.352 64	0.941 41	0.324 61	0.155 60	0.238 69	0.387 62	0.493 60	0.529 68	0.509 52	0.813 54	0.751 55	0.504 58

3DMV	0.484 63	0.484 69	0.538 70	0.643 55	0.424 61	0.606 70	0.310 61	0.574 59	0.433 63	0.378 61	0.796 70	0.301 63	0.214 40	0.537 40	0.208 70	0.472 64	0.507 71	0.413 68	0.693 64	0.602 70	0.539 52
Angela Dai, Matthias Niessner: 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation. ECCV'18
PointCNN with RGB	0.458 64	0.577 62	0.611 66	0.356 74	0.321 70	0.715 58	0.299 63	0.376 68	0.328 71	0.319 65	0.944 36	0.285 65	0.164 57	0.216 72	0.229 68	0.484 62	0.545 66	0.456 62	0.755 59	0.709 60	0.475 63
Yangyan Li, Rui Bu, Mingchao Sun, Baoquan Chen: PointCNN. NeurIPS 2018
FCPN	0.447 65	0.679 50	0.604 68	0.578 62	0.380 64	0.682 62	0.291 64	0.106 74	0.483 51	0.258 72	0.920 60	0.258 68	0.025 74	0.231 71	0.325 64	0.480 63	0.560 64	0.463 61	0.725 61	0.666 66	0.231 74
Dario Rethage, Johanna Wald, Jürgen Sturm, Nassir Navab, Federico Tombari: Fully-Convolutional Point Networks for Large-Scale Point Clouds. ECCV 2018
PNET2	0.442 66	0.548 65	0.548 69	0.597 60	0.363 66	0.628 68	0.300 62	0.292 69	0.374 67	0.307 66	0.881 67	0.268 67	0.186 50	0.238 69	0.204 71	0.407 67	0.506 72	0.449 63	0.667 66	0.620 68	0.462 64

SurfaceConvPF	0.442 66	0.505 67	0.622 65	0.380 73	0.342 68	0.654 64	0.227 70	0.397 67	0.367 68	0.276 69	0.924 57	0.240 70	0.198 45	0.359 62	0.262 66	0.366 68	0.581 61	0.435 66	0.640 67	0.668 65	0.398 66
Hao Pan, Shilin Liu, Yang Liu, Xin Tong: Convolutional Neural Networks on 3D Surfaces Using Parallel Frames.
Tangent Convolutions	0.438 68	0.437 72	0.646 61	0.474 69	0.369 65	0.645 65	0.353 58	0.258 71	0.282 73	0.279 68	0.918 62	0.298 64	0.147 64	0.283 66	0.294 65	0.487 61	0.562 63	0.427 67	0.619 68	0.633 67	0.352 69
Maxim Tatarchenko, Jaesik Park, Vladlen Koltun, Qian-Yi Zhou: Tangent convolutions for dense prediction in 3d. CVPR 2018
3DWSSS	0.425 69	0.525 66	0.647 60	0.522 65	0.324 69	0.488 74	0.077 75	0.712 40	0.353 69	0.401 58	0.636 74	0.281 66	0.176 53	0.340 63	0.565 46	0.175 75	0.551 65	0.398 69	0.370 75	0.602 70	0.361 68

subcloud_weak	0.411 70	0.479 70	0.650 58	0.475 68	0.285 73	0.519 73	0.087 74	0.725 35	0.396 66	0.386 60	0.621 75	0.250 69	0.117 66	0.338 65	0.443 57	0.188 74	0.594 60	0.369 72	0.377 74	0.616 69	0.306 70

SPLAT Net	0.393 71	0.472 71	0.511 71	0.606 57	0.311 71	0.656 63	0.245 68	0.405 66	0.328 71	0.197 73	0.927 56	0.227 72	0.000 76	0.001 76	0.249 67	0.271 73	0.510 69	0.383 71	0.593 69	0.699 62	0.267 72
Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, Jan Kautz: SPLATNet: Sparse Lattice Networks for Point Cloud Processing. CVPR 2018
ScanNet+FTSDF	0.383 72	0.297 74	0.491 72	0.432 71	0.358 67	0.612 69	0.274 65	0.116 73	0.411 64	0.265 70	0.904 66	0.229 71	0.079 70	0.250 67	0.185 72	0.320 71	0.510 69	0.385 70	0.548 70	0.597 73	0.394 67

PointNet++	0.339 73	0.584 61	0.478 73	0.458 70	0.256 74	0.360 75	0.250 66	0.247 72	0.278 74	0.261 71	0.677 73	0.183 73	0.117 66	0.212 73	0.145 74	0.364 69	0.346 75	0.232 75	0.548 70	0.523 74	0.252 73
Charles R. Qi, Li Yi, Hao Su, Leonidas J. Guibas: pointnet++: deep hierarchical feature learning on point sets in a metric space.
SSC-UNet	0.308 74	0.353 73	0.290 75	0.278 75	0.166 75	0.553 71	0.169 73	0.286 70	0.147 75	0.148 75	0.908 64	0.182 74	0.064 71	0.023 75	0.018 76	0.354 70	0.363 73	0.345 73	0.546 72	0.685 63	0.278 71

ScanNet	0.306 75	0.203 75	0.366 74	0.501 66	0.311 71	0.524 72	0.211 72	0.002 76	0.342 70	0.189 74	0.786 71	0.145 75	0.102 69	0.245 68	0.152 73	0.318 72	0.348 74	0.300 74	0.460 73	0.437 75	0.182 75
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nießner: ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. CVPR'17
ERROR	0.054 76	0.000 76	0.041 76	0.172 76	0.030 76	0.062 76	0.001 76	0.035 75	0.004 76	0.051 76	0.143 76	0.019 76	0.003 75	0.041 74	0.050 75	0.003 76	0.054 76	0.018 76	0.005 76	0.264 76	0.082 76

3D Semantic label benchmark