The 3D semantic labeling task involves predicting a semantic labeling of a 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the PASCAL VOC intersection-over-union metric (IoU). IoU = TP/(TP+FP+FN), where TP, FP, and FN are the numbers of true positive, false positive, and false negative pixels, respectively. Predicted labels are evaluated per-vertex over the respective 3D scan mesh; for 3D approaches that operate on other representations like grids or points, the predicted labels should be mapped onto the mesh vertices (e.g., one such example for grid to mesh vertices is provided in the evaluation helpers).



This table lists the benchmark results for the ScanNet200 3D semantic label scenario.




Method Infoavg iouhead ioucommon ioutail ioualarm clockarmchairbackpackbagballbarbasketbathroom cabinetbathroom counterbathroom stallbathroom stall doorbathroom vanitybathtubbedbenchbicyclebinblackboardblanketblindsboardbookbookshelfbottlebowlboxbroombucketbulletin boardcabinetcalendarcandlecartcase of water bottlescd caseceilingceiling lightchairclockclosetcloset doorcloset rodcloset wallclothesclothes dryercoat rackcoffee kettlecoffee makercoffee tablecolumncomputer towercontainercopiercouchcountercratecupcurtaincushiondecorationdeskdining tabledish rackdishwasherdividerdoordoorframedresserdumbbelldustpanend tablefanfile cabinetfire alarmfire extinguisherfireplacefloorfolded chairfurnitureguitarguitar casehair dryerhandicap barhatheadphonesironing boardjacketkeyboardkeyboard pianokitchen cabinetkitchen counterladderlamplaptoplaundry basketlaundry detergentlaundry hamperledgelightlight switchluggagemachinemailboxmatmattressmicrowavemini fridgemirrormonitormousemusic standnightstandobjectoffice chairottomanovenpaperpaper bagpaper cutterpaper towel dispenserpaper towel rollpersonpianopicturepillarpillowpipeplantplateplungerposterpotted plantpower outletpower stripprinterprojectorprojector screenpurserackradiatorrailrange hoodrecycling binrefrigeratorscaleseatshelfshoeshowershower curtainshower curtain rodshower doorshower floorshower headshower wallsignsinksoap dishsoap dispensersofa chairspeakerstair railstairsstandstoolstorage binstorage containerstorage organizerstovestructurestuffed animalsuitcasetabletelephonetissue boxtoastertoaster oventoilettoilet papertoilet paper dispensertoilet paper holdertoilet seat cover dispensertoweltrash bintrash cantraytubetvtv standvacuum cleanerventwallwardrobewashing machinewater bottlewater coolerwater pitcherwhiteboardwindowwindowsill
sort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysorted bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort by
PonderV2 ScanNet2000.346 20.552 40.270 30.175 30.497 40.070 70.239 40.000 10.000 20.000 10.232 100.412 50.584 10.842 20.804 30.212 40.540 40.000 30.433 100.106 60.000 50.590 30.290 60.548 20.243 40.000 40.356 60.000 10.000 30.062 60.398 70.441 40.000 10.104 50.000 20.888 20.076 70.682 30.030 10.094 40.491 50.351 60.869 60.000 10.063 10.403 50.700 20.000 70.660 80.881 20.761 10.050 50.186 40.852 60.000 10.007 40.570 40.100 20.565 20.326 30.641 50.431 30.290 70.621 30.259 20.408 40.622 50.125 10.082 60.950 20.179 30.000 10.263 20.424 20.193 40.558 30.880 10.545 60.375 40.727 20.445 50.499 50.000 30.000 10.475 40.002 40.034 40.083 30.000 30.924 10.290 20.636 30.115 70.400 30.874 30.186 40.000 10.611 40.128 20.113 20.000 40.000 10.000 40.584 50.636 40.103 70.385 50.843 30.283 20.603 30.080 50.825 30.000 30.377 60.000 10.000 30.000 20.457 60.000 10.000 40.000 10.574 70.608 50.481 20.792 30.394 20.000 10.357 60.503 60.261 50.817 60.504 80.304 30.472 30.115 40.000 10.750 30.677 30.202 10.000 70.509 30.729 10.000 10.519 70.000 90.000 50.000 10.620 70.000 20.000 10.660 30.560 40.486 20.384 50.346 40.952 20.247 70.667 20.436 50.269 30.691 30.000 10.010 30.787 50.889 20.880 40.000 10.810 40.336 30.860 60.000 10.606 30.009 50.248 50.681 40.392 5
Haoyi Zhu, Honghui Yang, Xiaoyang Wu, Di Huang, Sha Zhang, Xianglong He, Tong He, Hengshuang Zhao, Chunhua Shen, Yu Qiao, Wanli Ouyang: PonderV2: Pave the Way for 3D Foundataion Model with A Universal Pre-training Paradigm.
PTv3 ScanNet2000.393 10.592 10.330 10.216 10.520 10.109 20.108 100.000 10.337 10.000 10.310 90.394 60.494 70.753 60.848 10.256 10.717 20.000 30.842 10.192 20.065 20.449 50.346 10.546 30.190 60.000 40.384 30.000 10.000 30.218 10.505 10.791 10.000 10.136 10.000 20.903 10.073 80.687 20.000 40.168 10.551 20.387 40.941 10.000 10.000 20.397 60.654 30.000 70.714 30.759 80.752 30.118 30.264 10.926 10.000 10.048 10.575 10.000 60.597 10.366 10.755 10.469 10.474 10.798 10.140 60.617 10.692 20.000 30.592 10.971 10.188 20.000 10.133 30.593 10.349 10.650 10.717 40.699 10.455 10.790 10.523 30.636 10.301 10.000 10.622 20.000 60.017 90.259 10.000 30.921 20.337 10.733 10.210 10.514 10.860 60.407 10.000 10.688 10.109 60.000 90.000 40.000 10.151 10.671 40.782 10.115 60.641 10.903 10.349 10.616 10.088 40.832 10.000 30.480 10.000 10.428 10.000 20.497 50.000 10.000 40.000 10.662 20.690 10.612 10.828 10.575 10.000 10.404 40.644 10.325 30.887 10.728 10.009 90.134 50.026 100.000 10.761 10.731 10.172 30.077 20.528 20.727 20.000 10.603 40.220 20.022 20.000 10.740 10.000 20.000 10.661 10.586 10.566 10.436 30.531 10.978 10.457 10.708 10.583 20.141 70.748 10.000 10.026 10.822 10.871 30.879 50.000 10.851 10.405 20.914 10.000 10.682 10.000 80.281 10.738 10.463 3
OA-CNN-L_ScanNet2000.333 40.558 20.269 40.124 60.448 80.080 50.272 30.000 10.000 20.000 10.342 50.515 20.524 30.713 100.789 40.158 60.384 50.000 30.806 30.125 30.000 50.496 40.332 30.498 90.227 50.024 20.474 10.000 10.003 20.071 50.487 20.000 50.000 10.110 30.000 20.876 30.013 100.703 10.000 40.076 60.473 60.355 50.906 30.000 10.000 20.476 40.706 10.000 70.672 70.835 60.748 40.015 90.223 30.860 40.000 10.000 60.572 30.000 60.509 50.313 40.662 20.398 70.396 20.411 80.276 10.527 20.711 10.000 30.076 70.946 30.166 40.000 10.022 40.160 30.183 60.493 60.699 50.637 30.403 30.330 70.406 60.526 30.024 20.000 10.392 60.000 60.016 100.000 50.196 20.915 40.112 50.557 50.197 20.352 60.877 20.000 50.000 10.592 80.103 80.000 90.067 10.000 10.089 20.735 30.625 50.130 50.568 30.836 40.271 30.534 50.043 80.799 40.001 20.445 20.000 10.000 30.024 10.661 20.000 10.262 10.000 10.591 40.517 90.373 50.788 50.021 40.000 10.455 10.517 50.320 40.823 50.200 100.001 100.150 40.100 50.000 10.736 40.668 40.103 80.052 40.662 10.720 30.000 10.602 50.112 40.002 40.000 10.637 50.000 20.000 10.621 50.569 20.398 40.412 40.234 50.949 30.363 20.492 90.495 40.251 40.665 50.000 10.001 70.805 30.833 40.794 60.000 10.821 20.314 40.843 70.000 10.560 40.245 20.262 30.713 20.370 7
PPT-SpUNet-F.T.0.332 50.556 30.270 20.123 70.519 20.091 30.349 20.000 10.000 20.000 10.339 60.383 70.498 60.833 30.807 20.241 20.584 30.000 30.755 40.124 40.000 50.608 20.330 40.530 60.314 10.000 40.374 40.000 10.000 30.197 20.459 40.000 50.000 10.117 20.000 20.876 30.095 10.682 30.000 40.086 50.518 30.433 10.930 20.000 10.000 20.563 30.542 70.077 40.715 20.858 40.756 20.008 100.171 60.874 30.000 10.039 20.550 50.000 60.545 40.256 50.657 40.453 20.351 40.449 70.213 30.392 50.611 60.000 30.037 80.946 30.138 70.000 10.000 60.063 50.308 20.537 40.796 20.673 20.323 70.392 50.400 70.509 40.000 30.000 10.649 10.000 60.023 60.000 50.000 30.914 50.002 90.506 90.163 60.359 50.872 40.000 50.000 10.623 30.112 40.001 80.000 40.000 10.021 30.753 10.565 90.150 10.579 20.806 60.267 40.616 10.042 90.783 60.000 30.374 70.000 10.000 30.000 20.620 40.000 10.000 40.000 10.572 80.634 20.350 60.792 30.000 50.000 10.376 50.535 30.378 20.855 20.672 20.074 60.000 60.185 30.000 10.727 50.660 50.076 100.000 70.432 60.646 50.000 10.594 60.006 80.000 50.000 10.658 30.000 20.000 10.661 10.549 50.300 70.291 70.045 70.942 60.304 30.600 50.572 30.135 90.695 20.000 10.008 50.793 40.942 10.899 20.000 10.816 30.181 60.897 20.000 10.679 20.223 30.264 20.691 30.345 8
Xiaoyang Wu, Zhuotao Tian, Xin Wen, Bohao Peng, Xihui Liu, Kaicheng Yu, Hengshuang Zhao: Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Training.
OctFormer ScanNet200permissive0.326 60.539 60.265 50.131 50.499 30.110 10.522 10.000 10.000 20.000 10.318 80.427 40.455 80.743 80.765 60.175 50.842 10.000 30.828 20.204 10.033 30.429 60.335 20.601 10.312 20.000 40.357 50.000 10.000 30.047 70.423 50.000 50.000 10.105 40.000 20.873 50.079 60.670 60.000 40.117 20.471 70.432 20.829 70.000 10.000 20.584 20.417 100.089 30.684 60.837 50.705 90.021 80.178 50.892 20.000 10.028 30.505 70.000 60.457 60.200 80.662 20.412 50.244 80.496 50.000 100.451 30.626 40.000 30.102 50.943 50.138 70.000 10.000 60.149 40.291 30.534 50.722 30.632 40.331 60.253 90.453 40.487 60.000 30.000 10.479 30.000 60.022 70.000 50.000 30.900 60.128 40.684 20.164 50.413 20.854 70.000 50.000 10.512 100.074 100.003 70.000 40.000 10.000 40.469 80.613 60.132 40.529 40.871 20.227 90.582 40.026 100.787 50.000 30.339 80.000 10.000 30.000 20.626 30.000 10.029 30.000 10.587 50.612 40.411 40.724 70.000 50.000 10.407 30.552 20.513 10.849 30.655 30.408 10.000 60.296 10.000 10.686 80.645 70.145 50.022 50.414 70.633 60.000 10.637 10.224 10.000 50.000 10.650 40.000 20.000 10.622 40.535 60.343 50.483 20.230 60.943 50.289 40.618 40.596 10.140 80.679 40.000 10.022 20.783 60.620 80.906 10.000 10.806 50.137 80.865 30.000 10.378 60.000 80.168 100.680 50.227 9
Peng-Shuai Wang: OctFormer: Octree-based Transformers for 3D Point Clouds. SIGGRAPH 2023
CeCo0.340 30.551 50.247 60.181 20.475 60.057 100.142 80.000 10.000 20.000 10.387 30.463 30.499 50.924 10.774 50.213 30.257 60.000 30.546 90.100 70.006 40.615 10.177 100.534 40.246 30.000 40.400 20.000 10.338 10.006 90.484 30.609 20.000 10.083 60.000 20.873 50.089 40.661 70.000 40.048 100.560 10.408 30.892 40.000 10.000 20.586 10.616 50.000 70.692 50.900 10.721 50.162 10.228 20.860 40.000 10.000 60.575 10.083 30.550 30.347 20.624 60.410 60.360 30.740 20.109 70.321 80.660 30.000 30.121 30.939 60.143 50.000 10.400 10.003 70.190 50.564 20.652 60.615 50.421 20.304 80.579 10.547 20.000 30.000 10.296 70.000 60.030 50.096 20.000 30.916 30.037 60.551 60.171 40.376 40.865 50.286 20.000 10.633 20.102 90.027 50.011 30.000 10.000 40.474 70.742 20.133 30.311 60.824 50.242 60.503 70.068 60.828 20.000 30.429 30.000 10.063 20.000 20.781 10.000 10.000 40.000 10.665 10.633 30.450 30.818 20.000 50.000 10.429 20.532 40.226 60.825 40.510 70.377 20.709 10.079 70.000 10.753 20.683 20.102 90.063 30.401 90.620 80.000 10.619 20.000 90.000 50.000 10.595 80.000 20.000 10.345 70.564 30.411 30.603 10.384 30.945 40.266 50.643 30.367 70.304 10.663 60.000 10.010 30.726 80.767 50.898 30.000 10.784 60.435 10.861 50.000 10.447 50.000 80.257 40.656 60.377 6
Zhisheng Zhong, Jiequan Cui, Yibo Yang, Xiaoyang Wu, Xiaojuan Qi, Xiangyu Zhang, Jiaya Jia: Understanding Imbalanced Semantic Segmentation Through Neural Collapse. CVPR 2023
AWCS0.305 70.508 70.225 70.142 40.463 70.063 80.195 60.000 10.000 20.000 10.467 20.551 10.504 40.773 40.764 70.142 70.029 100.000 30.626 70.100 70.000 50.360 70.179 80.507 80.137 80.006 30.300 70.000 10.000 30.172 40.364 80.512 30.000 10.056 70.000 20.865 70.093 30.634 100.000 40.071 80.396 80.296 90.876 50.000 10.000 20.373 70.436 90.063 60.749 10.877 30.721 50.131 20.124 70.804 80.000 10.000 60.515 60.010 50.452 70.252 60.578 70.417 40.179 100.484 60.171 40.337 70.606 70.000 30.115 40.937 70.142 60.000 10.008 50.000 90.157 90.484 70.402 100.501 80.339 50.553 30.529 20.478 70.000 30.000 10.404 50.001 50.022 70.077 40.000 30.894 80.219 30.628 40.093 80.305 70.886 10.233 30.000 10.603 50.112 40.023 60.000 40.000 10.000 40.741 20.664 30.097 80.253 70.782 70.264 50.523 60.154 10.707 90.000 30.411 40.000 10.000 30.000 20.332 90.000 10.000 40.000 10.602 30.595 60.185 90.656 90.159 30.000 10.355 70.424 80.154 80.729 80.516 60.220 50.620 20.084 60.000 10.707 70.651 60.173 20.014 60.381 100.582 90.000 10.619 20.049 70.000 50.000 10.702 20.000 20.000 10.302 90.489 80.317 60.334 60.392 20.922 70.254 60.533 80.394 60.129 100.613 80.000 10.000 80.820 20.649 70.749 70.000 10.782 70.282 50.863 40.000 10.288 90.006 60.220 70.633 70.542 1
LGroundpermissive0.272 80.485 80.184 80.106 80.476 50.077 60.218 50.000 10.000 20.000 10.547 10.295 80.540 20.746 70.745 80.058 90.112 90.005 10.658 60.077 100.000 50.322 80.178 90.512 70.190 60.199 10.277 80.000 10.000 30.173 30.399 60.000 50.000 10.039 90.000 20.858 80.085 50.676 50.002 20.103 30.498 40.323 70.703 80.000 10.000 20.296 80.549 60.216 10.702 40.768 70.718 70.028 60.092 90.786 90.000 10.000 60.453 90.022 40.251 100.252 60.572 80.348 80.321 50.514 40.063 80.279 90.552 80.000 30.019 90.932 80.132 90.000 10.000 60.000 90.156 100.457 80.623 70.518 70.265 90.358 60.381 80.395 80.000 30.000 10.127 100.012 30.051 10.000 50.000 30.886 90.014 70.437 100.179 30.244 80.826 80.000 50.000 10.599 60.136 10.085 30.000 40.000 10.000 40.565 60.612 70.143 20.207 80.566 80.232 80.446 80.127 20.708 80.000 30.384 50.000 10.000 30.000 20.402 70.000 10.059 20.000 10.525 100.566 70.229 80.659 80.000 50.000 10.265 80.446 70.147 90.720 100.597 50.066 70.000 60.187 20.000 10.726 60.467 100.134 70.000 70.413 80.629 70.000 10.363 90.055 60.022 20.000 10.626 60.000 20.000 10.323 80.479 100.154 90.117 80.028 90.901 80.243 80.415 100.295 100.143 60.610 90.000 10.000 80.777 70.397 100.324 90.000 10.778 80.179 70.702 90.000 10.274 100.404 10.233 60.622 80.398 4
David Rozenberszki, Or Litany, Angela Dai: Language-Grounded Indoor 3D Semantic Segmentation in the Wild. arXiv
CSC-Pretrainpermissive0.249 100.455 100.171 90.079 100.418 90.059 90.186 70.000 10.000 20.000 10.335 70.250 90.316 90.766 50.697 100.142 70.170 70.003 20.553 80.112 50.097 10.201 100.186 70.476 100.081 90.000 40.216 100.000 10.000 30.001 100.314 100.000 50.000 10.055 80.000 20.832 100.094 20.659 80.002 20.076 60.310 100.293 100.664 100.000 10.000 20.175 100.634 40.130 20.552 100.686 100.700 100.076 40.110 80.770 100.000 10.000 60.430 100.000 60.319 80.166 90.542 100.327 90.205 90.332 90.052 90.375 60.444 100.000 30.012 100.930 100.203 10.000 10.000 60.046 60.175 70.413 90.592 80.471 90.299 80.152 100.340 90.247 100.000 30.000 10.225 80.058 20.037 20.000 50.207 10.862 100.014 70.548 70.033 90.233 90.816 90.000 50.000 10.542 90.123 30.121 10.019 20.000 10.000 40.463 90.454 100.045 100.128 100.557 90.235 70.441 90.063 70.484 100.000 30.308 100.000 10.000 30.000 20.318 100.000 10.000 40.000 10.545 90.543 80.164 100.734 60.000 50.000 10.215 100.371 90.198 70.743 70.205 90.062 80.000 60.079 70.000 10.683 90.547 90.142 60.000 70.441 50.579 100.000 10.464 80.098 50.041 10.000 10.590 90.000 20.000 10.373 60.494 70.174 80.105 90.001 100.895 90.222 90.537 70.307 90.180 50.625 70.000 10.000 80.591 100.609 90.398 80.000 10.766 100.014 100.638 100.000 10.377 70.004 70.206 90.609 100.465 2
Ji Hou, Benjamin Graham, Matthias Nie├čner, Saining Xie: Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts. CVPR 2021
Minkowski 34Dpermissive0.253 90.463 90.154 100.102 90.381 100.084 40.134 90.000 10.000 20.000 10.386 40.141 100.279 100.737 90.703 90.014 100.164 80.000 30.663 50.092 90.000 50.224 90.291 50.531 50.056 100.000 40.242 90.000 10.000 30.013 80.331 90.000 50.000 10.035 100.001 10.858 80.059 90.650 90.000 40.056 90.353 90.299 80.670 90.000 10.000 20.284 90.484 80.071 50.594 90.720 90.710 80.027 70.068 100.813 70.000 10.005 50.492 80.164 10.274 90.111 100.571 90.307 100.293 60.307 100.150 50.163 100.531 90.002 20.545 20.932 80.093 100.000 10.000 60.002 80.159 80.368 100.581 90.440 100.228 100.406 40.282 100.294 90.000 30.000 10.189 90.060 10.036 30.000 50.000 30.897 70.000 100.525 80.025 100.205 100.771 100.000 50.000 10.593 70.108 70.044 40.000 40.000 10.000 40.282 100.589 80.094 90.169 90.466 100.227 90.419 100.125 30.757 70.002 10.334 90.000 10.000 30.000 20.357 80.000 10.000 40.000 10.582 60.513 100.337 70.612 100.000 50.000 10.250 90.352 100.136 100.724 90.655 30.280 40.000 60.046 90.000 10.606 100.559 80.159 40.102 10.445 40.655 40.000 10.310 100.117 30.000 50.000 10.581 100.026 10.000 10.265 100.483 90.084 100.097 100.044 80.865 100.142 100.588 60.351 80.272 20.596 100.000 10.003 60.622 90.720 60.096 100.000 10.771 90.016 90.772 80.000 10.302 80.194 40.214 80.621 90.197 10
C. Choy, J. Gwak, S. Savarese: 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks. CVPR 2019