The 3D semantic labeling task involves predicting a semantic labeling of a 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the PASCAL VOC intersection-over-union metric (IoU). IoU = TP/(TP+FP+FN), where TP, FP, and FN are the numbers of true positive, false positive, and false negative pixels, respectively. Predicted labels are evaluated per-vertex over the respective 3D scan mesh; for 3D approaches that operate on other representations like grids or points, the predicted labels should be mapped onto the mesh vertices (e.g., one such example for grid to mesh vertices is provided in the evaluation helpers).



This table lists the benchmark results for the 3D semantic label with limited annotations scenario.




Method Infoavg ioubathtubbedbookshelfcabinetchaircountercurtaindeskdoorfloorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwallwindow
sort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysorted bysort bysort bysort bysort bysort bysort bysort bysort bysort by
DE-3DLearner LA0.695 30.897 30.784 20.728 50.697 30.846 40.441 70.770 30.615 20.585 30.951 40.504 10.232 40.672 30.760 40.655 40.772 50.599 30.877 70.834 40.678 2
Ping-Chung Yu, Cheng Sun, Min Sun: Data Efficient 3D Learner via Knowledge Transferred from 2D Model. ECCV 2022
Q2E0.739 10.984 10.797 10.761 20.716 10.884 10.588 10.843 10.589 30.656 10.971 20.487 20.271 20.772 10.807 20.726 20.795 20.630 10.945 20.856 10.693 1
GaIA0.643 70.704 110.776 30.670 100.597 90.842 50.382 90.688 90.413 120.556 40.950 50.471 30.334 10.478 100.728 60.640 50.787 40.557 70.937 30.812 60.531 11
Min Seok Lee*, Seok Woo Yang*, and Sung Won Han: GaIA: Graphical Information gain based Attention Network for Weakly Supervised 3D Point Cloud Semantic Segmentation. WACV 2023
VIBUSpermissive0.651 60.868 40.728 110.675 90.624 70.861 30.247 130.734 60.561 50.520 80.948 60.464 40.216 60.670 40.742 50.589 90.746 70.579 40.877 70.800 70.568 6
Beiwen Tian,Liyi Luo,Hao Zhao,Guyue Zhou: VIBUS: Data-efficient 3D Scene Parsing with VIewpoint Bottleneck and Uncertainty-Spectrum Modeling. ISPRS Journal of Photogrammetry and Remote Sensing
One-Thing-One-Click0.642 80.725 100.735 70.717 60.635 60.829 60.457 50.639 110.421 110.552 60.967 30.460 50.240 30.558 70.788 30.621 60.720 80.477 110.915 50.842 30.539 8
Zhengzhe Liu, Xiaojuan Qi, Chi-Wing Fu: One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation. CVPR 2021
ActiveST0.725 20.980 20.764 40.753 30.699 20.863 20.521 20.773 20.671 10.625 20.974 10.456 60.182 90.721 20.874 10.746 10.808 10.628 20.960 10.846 20.664 3
Gengxin Liu, Oliver van Kaick, Hui Huang, Ruizhen Hu: Active Self-Training for Weakly Supervised 3D Scene Semantic Segmentation.
LE0.652 50.816 70.760 50.747 40.648 50.807 80.455 60.765 40.517 70.523 70.941 110.452 70.190 80.586 50.691 70.525 110.762 60.552 80.930 40.795 90.580 4
WS3D_LA_Sempermissive0.670 40.842 60.732 80.825 10.657 40.794 100.506 30.762 50.584 40.553 50.947 70.451 80.219 50.585 60.652 80.670 30.791 30.570 50.857 110.816 50.579 5
Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022
Viewpoint_BN_LA_AIR0.623 90.812 80.743 60.654 110.579 110.800 90.462 40.713 70.533 60.516 90.944 80.434 90.215 70.437 110.521 120.601 70.720 80.563 60.884 60.800 70.534 10
Liyi Luo, Beiwen Tian, Hao Zhao, Guyue Zhou: Pointly-supervised 3D Scene Parsing with Viewpoint Bottleneck.
CSC_LA_SEM0.612 110.747 90.731 90.679 80.603 80.815 70.400 80.648 100.453 90.481 110.944 80.421 100.173 100.504 80.623 100.588 100.690 120.545 90.877 70.778 110.541 7
PointContrast_LA_SEM0.614 100.844 50.731 90.681 70.590 100.791 110.348 110.689 80.503 80.502 100.942 100.361 110.154 120.484 90.624 90.591 80.708 100.524 100.874 100.793 100.536 9
SQN_LA0.542 120.568 130.674 130.618 130.462 120.772 120.351 100.567 120.443 100.378 130.931 130.335 120.173 100.392 120.623 100.455 130.688 130.466 120.769 130.720 130.450 12
Scratch_LA_SEM0.524 130.640 120.690 120.636 120.442 130.756 130.326 120.544 130.365 130.396 120.940 120.284 130.085 130.333 130.479 130.502 120.696 110.453 130.785 120.746 120.372 13