The 3D semantic labeling task involves predicting a semantic labeling of a 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the PASCAL VOC intersection-over-union metric (IoU). IoU = TP/(TP+FP+FN), where TP, FP, and FN are the numbers of true positive, false positive, and false negative pixels, respectively. Predicted labels are evaluated per-vertex over the respective 3D scan mesh; for 3D approaches that operate on other representations like grids or points, the predicted labels should be mapped onto the mesh vertices (e.g., one such example for grid to mesh vertices is provided in the evaluation helpers).



This table lists the benchmark results for the 3D semantic label with limited annotations scenario.




Method Infoavg ioubathtubbedbookshelfcabinetchaircountercurtaindeskdoorfloorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwallwindow
sort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysorted by
ActiveST0.748 10.984 10.804 30.759 50.720 20.849 50.516 20.791 30.670 10.654 20.974 10.495 50.382 10.811 10.828 50.787 10.780 60.640 20.952 10.861 30.701 1
Gengxin Liu, Oliver van Kaick, Hui Huang, Ruizhen Hu: Active Self-Training for Weakly Supervised 3D Scene Semantic Segmentation.
Q2E0.743 20.984 10.803 40.770 10.725 10.881 10.572 10.806 20.663 20.665 10.972 20.506 30.305 20.652 60.829 40.761 20.809 20.660 10.951 20.862 20.682 2
DE-3DLearner LA0.709 30.877 40.772 80.744 90.694 30.836 70.453 60.787 40.623 40.598 40.953 40.490 70.216 110.682 50.879 10.727 30.802 30.604 50.922 30.845 40.676 3
Ping-Chung Yu, Cheng Sun, Min Sun: Data Efficient 3D Learner via Knowledge Transferred from 2D Model. ECCV 2022
One-Thing-One-Click0.694 40.760 90.815 20.706 130.684 50.840 60.492 40.701 90.557 70.596 50.972 20.497 40.281 40.709 20.757 80.689 60.789 40.600 70.907 70.864 10.671 4
Zhengzhe Liu, Xiaojuan Qi, Chi-Wing Fu: One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation. CVPR 2021
Scratch_LA_SEM0.643 120.699 130.793 50.718 120.636 100.816 110.411 110.707 80.490 120.574 80.948 60.448 100.173 130.559 100.689 120.604 110.722 120.556 110.853 120.820 80.651 5
LE0.688 70.856 70.779 60.754 70.687 40.834 80.438 80.732 70.536 90.577 60.948 60.508 20.248 70.699 30.831 30.636 80.752 110.586 90.895 90.821 70.643 6
VIBUSpermissive0.691 60.860 50.731 120.738 100.672 70.860 20.470 50.766 50.625 30.547 110.949 50.491 60.255 50.693 40.715 100.712 40.778 70.597 80.911 50.816 90.635 7
Beiwen Tian,Liyi Luo,Hao Zhao,Guyue Zhou: VIBUS: Data-efficient 3D Scene Parsing with VIewpoint Bottleneck and Uncertainty-Spectrum Modeling. ISPRS Journal of Photogrammetry and Remote Sensing
GaIA0.685 80.759 100.834 10.759 50.650 80.859 30.427 100.694 100.524 100.575 70.948 60.537 10.304 30.534 120.853 20.678 70.820 10.581 100.914 40.828 50.626 8
Min Seok Lee*, Seok Woo Yang*, and Sung Won Han: GaIA: Graphical Information gain based Attention Network for Weakly Supervised 3D Point Cloud Semantic Segmentation. WACV 2023
Viewpoint_BN_LA_AIR0.669 90.847 80.732 110.724 110.613 120.827 90.443 70.742 60.562 60.551 100.947 100.441 120.218 100.650 70.753 90.621 90.765 100.601 60.905 80.814 120.618 9
Liyi Luo, Beiwen Tian, Hao Zhao, Guyue Zhou: Pointly-supervised 3D Scene Parsing with Viewpoint Bottleneck.
PointContrast_LA_SEM0.653 110.717 120.775 70.754 70.626 110.804 130.391 120.689 110.485 130.572 90.945 120.448 100.232 90.603 80.813 60.591 120.775 90.537 120.885 100.816 90.608 10
CSC_LA_SEM0.665 100.857 60.756 90.763 40.647 90.852 40.432 90.684 120.543 80.514 120.948 60.469 80.179 120.599 90.702 110.620 100.789 40.614 40.911 50.815 110.607 11
WS3D_LA_Sempermissive0.694 40.895 30.743 100.767 20.675 60.826 100.496 30.817 10.612 50.613 30.947 100.460 90.254 60.558 110.811 70.710 50.776 80.616 30.874 110.822 60.603 12
Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022
SQN_LA0.598 130.741 110.681 130.766 30.482 130.805 120.389 130.658 130.499 110.437 130.936 130.386 130.243 80.422 130.663 130.552 130.700 130.519 130.809 130.750 130.515 13