The 3D semantic labeling task involves predicting a semantic labeling of a 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the PASCAL VOC intersection-over-union metric (IoU). IoU = TP/(TP+FP+FN), where TP, FP, and FN are the numbers of true positive, false positive, and false negative pixels, respectively. Predicted labels are evaluated per-vertex over the respective 3D scan mesh; for 3D approaches that operate on other representations like grids or points, the predicted labels should be mapped onto the mesh vertices (e.g., one such example for grid to mesh vertices is provided in the evaluation helpers).



This table lists the benchmark results for the ScanNet200 3D semantic label scenario.




Method Infoavg iouhead ioucommon ioutail ioualarm clockarmchairbackpackbagballbarbasketbathroom cabinetbathroom counterbathroom stallbathroom stall doorbathroom vanitybathtubbedbenchbicyclebinblackboardblanketblindsboardbookbookshelfbottlebowlboxbroombucketbulletin boardcabinetcalendarcandlecartcase of water bottlescd caseceilingceiling lightchairclockclosetcloset doorcloset rodcloset wallclothesclothes dryercoat rackcoffee kettlecoffee makercoffee tablecolumncomputer towercontainercopiercouchcountercratecupcurtaincushiondecorationdeskdining tabledish rackdishwasherdividerdoordoorframedresserdumbbelldustpanend tablefanfile cabinetfire alarmfire extinguisherfireplacefloorfolded chairfurnitureguitarguitar casehair dryerhandicap barhatheadphonesironing boardjacketkeyboardkeyboard pianokitchen cabinetkitchen counterladderlamplaptoplaundry basketlaundry detergentlaundry hamperledgelightlight switchluggagemachinemailboxmatmattressmicrowavemini fridgemirrormonitormousemusic standnightstandobjectoffice chairottomanovenpaperpaper bagpaper cutterpaper towel dispenserpaper towel rollpersonpianopicturepillarpillowpipeplantplateplungerposterpotted plantpower outletpower stripprinterprojectorprojector screenpurserackradiatorrailrange hoodrecycling binrefrigeratorscaleseatshelfshoeshowershower curtainshower curtain rodshower doorshower floorshower headshower wallsignsinksoap dishsoap dispensersofa chairspeakerstair railstairsstandstoolstorage binstorage containerstorage organizerstovestructurestuffed animalsuitcasetabletelephonetissue boxtoastertoaster oventoilettoilet papertoilet paper dispensertoilet paper holdertoilet seat cover dispensertoweltrash bintrash cantraytubetvtv standvacuum cleanerventwallwardrobewashing machinewater bottlewater coolerwater pitcherwhiteboardwindowwindowsill
sort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysorted bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort by
OA-CNN-L_ScanNet2000.333 20.558 10.269 10.124 30.448 40.080 20.272 10.000 10.000 10.000 10.342 50.515 20.524 20.713 60.789 10.158 20.384 10.000 30.806 10.125 10.000 30.496 20.332 10.498 50.227 20.024 20.474 10.000 10.003 20.071 30.487 10.000 30.000 10.110 10.000 20.876 10.013 60.703 10.000 30.076 20.473 30.355 20.906 10.000 10.000 10.476 20.706 10.000 50.672 40.835 30.748 10.015 60.223 20.860 10.000 10.000 20.572 20.000 50.509 20.313 20.662 10.398 30.396 10.411 40.276 10.527 10.711 10.000 20.076 40.946 10.166 20.000 10.022 20.160 10.183 20.493 20.699 10.637 10.403 20.330 40.406 30.526 20.024 10.000 10.392 20.000 50.016 60.000 30.196 20.915 20.112 20.557 20.197 10.352 20.877 20.000 30.000 10.592 50.103 50.000 60.067 10.000 10.089 10.735 20.625 30.130 30.568 10.836 10.271 10.534 10.043 60.799 20.001 20.445 10.000 10.000 20.024 10.661 20.000 10.262 10.000 10.591 30.517 50.373 20.788 20.021 20.000 10.455 10.517 20.320 10.823 20.200 60.001 60.150 30.100 20.000 10.736 20.668 20.103 50.052 30.662 10.720 10.000 10.602 30.112 20.002 30.000 10.637 20.000 20.000 10.621 10.569 10.398 20.412 20.234 30.949 10.363 10.492 50.495 10.251 30.665 10.000 10.001 30.805 20.833 10.794 20.000 10.821 10.314 20.843 30.000 10.560 10.245 20.262 10.713 10.370 5
CeCo0.340 10.551 20.247 20.181 10.475 20.057 60.142 50.000 10.000 10.000 10.387 30.463 30.499 40.924 10.774 20.213 10.257 20.000 30.546 60.100 30.006 20.615 10.177 60.534 10.246 10.000 40.400 20.000 10.338 10.006 50.484 20.609 10.000 10.083 20.000 20.873 20.089 30.661 30.000 30.048 60.560 10.408 10.892 20.000 10.000 10.586 10.616 30.000 50.692 30.900 10.721 20.162 10.228 10.860 10.000 10.000 20.575 10.083 20.550 10.347 10.624 20.410 20.360 20.740 10.109 40.321 40.660 20.000 20.121 20.939 20.143 30.000 10.400 10.003 30.190 10.564 10.652 20.615 20.421 10.304 50.579 10.547 10.000 20.000 10.296 30.000 50.030 40.096 10.000 30.916 10.037 30.551 30.171 30.376 10.865 30.286 10.000 10.633 10.102 60.027 40.011 30.000 10.000 20.474 40.742 10.133 20.311 20.824 20.242 30.503 30.068 40.828 10.000 30.429 20.000 10.063 10.000 20.781 10.000 10.000 30.000 10.665 10.633 10.450 10.818 10.000 30.000 10.429 20.532 10.226 20.825 10.510 40.377 10.709 10.079 40.000 10.753 10.683 10.102 60.063 20.401 50.620 40.000 10.619 10.000 60.000 40.000 10.595 40.000 20.000 10.345 30.564 20.411 10.603 10.384 20.945 20.266 20.643 10.367 30.304 10.663 20.000 10.010 10.726 40.767 20.898 10.000 10.784 20.435 10.861 20.000 10.447 20.000 60.257 20.656 20.377 4
: Understanding Imbalanced Semantic Segmentation Through Neural Collapse.
AWCS0.305 30.508 30.225 30.142 20.463 30.063 40.195 30.000 10.000 10.000 10.467 20.551 10.504 30.773 20.764 30.142 30.029 60.000 30.626 40.100 30.000 30.360 30.179 40.507 40.137 40.006 30.300 30.000 10.000 30.172 20.364 40.512 20.000 10.056 30.000 20.865 30.093 20.634 60.000 30.071 40.396 40.296 50.876 30.000 10.000 10.373 30.436 60.063 40.749 10.877 20.721 20.131 20.124 30.804 40.000 10.000 20.515 30.010 40.452 30.252 30.578 30.417 10.179 60.484 30.171 20.337 30.606 30.000 20.115 30.937 30.142 40.000 10.008 30.000 50.157 50.484 30.402 60.501 40.339 30.553 10.529 20.478 30.000 20.000 10.404 10.001 40.022 50.077 20.000 30.894 40.219 10.628 10.093 40.305 30.886 10.233 20.000 10.603 20.112 30.023 50.000 40.000 10.000 20.741 10.664 20.097 40.253 30.782 30.264 20.523 20.154 10.707 50.000 30.411 30.000 10.000 20.000 20.332 50.000 10.000 30.000 10.602 20.595 20.185 50.656 50.159 10.000 10.355 30.424 40.154 40.729 40.516 30.220 30.620 20.084 30.000 10.707 40.651 30.173 10.014 40.381 60.582 50.000 10.619 10.049 50.000 40.000 10.702 10.000 20.000 10.302 50.489 40.317 30.334 30.392 10.922 30.254 30.533 40.394 20.129 60.613 40.000 10.000 40.820 10.649 40.749 30.000 10.782 30.282 30.863 10.000 10.288 50.006 40.220 40.633 30.542 1
LGroundpermissive0.272 40.485 40.184 40.106 40.476 10.077 30.218 20.000 10.000 10.000 10.547 10.295 40.540 10.746 40.745 40.058 50.112 50.005 10.658 30.077 60.000 30.322 40.178 50.512 30.190 30.199 10.277 40.000 10.000 30.173 10.399 30.000 30.000 10.039 50.000 20.858 40.085 40.676 20.002 10.103 10.498 20.323 30.703 40.000 10.000 10.296 40.549 40.216 10.702 20.768 40.718 40.028 40.092 50.786 50.000 10.000 20.453 50.022 30.251 60.252 30.572 40.348 40.321 30.514 20.063 50.279 50.552 40.000 20.019 50.932 40.132 50.000 10.000 40.000 50.156 60.457 40.623 30.518 30.265 50.358 30.381 40.395 40.000 20.000 10.127 60.012 30.051 10.000 30.000 30.886 50.014 40.437 60.179 20.244 40.826 40.000 30.000 10.599 30.136 10.085 20.000 40.000 10.000 20.565 30.612 40.143 10.207 40.566 40.232 50.446 40.127 20.708 40.000 30.384 40.000 10.000 20.000 20.402 30.000 10.059 20.000 10.525 60.566 30.229 40.659 40.000 30.000 10.265 40.446 30.147 50.720 60.597 20.066 40.000 40.187 10.000 10.726 30.467 60.134 40.000 50.413 40.629 30.000 10.363 50.055 40.022 20.000 10.626 30.000 20.000 10.323 40.479 60.154 50.117 40.028 50.901 40.243 40.415 60.295 60.143 50.610 50.000 10.000 40.777 30.397 60.324 50.000 10.778 40.179 40.702 50.000 10.274 60.404 10.233 30.622 40.398 3
David Rozenberszki, Or Litany, Angela Dai: Language-Grounded Indoor 3D Semantic Segmentation in the Wild. arXiv
CSC-Pretrainpermissive0.249 60.455 60.171 50.079 60.418 50.059 50.186 40.000 10.000 10.000 10.335 60.250 50.316 50.766 30.697 60.142 30.170 30.003 20.553 50.112 20.097 10.201 60.186 30.476 60.081 50.000 40.216 60.000 10.000 30.001 60.314 60.000 30.000 10.055 40.000 20.832 60.094 10.659 40.002 10.076 20.310 60.293 60.664 60.000 10.000 10.175 60.634 20.130 20.552 60.686 60.700 60.076 30.110 40.770 60.000 10.000 20.430 60.000 50.319 40.166 50.542 60.327 50.205 50.332 50.052 60.375 20.444 60.000 20.012 60.930 60.203 10.000 10.000 40.046 20.175 30.413 50.592 40.471 50.299 40.152 60.340 50.247 60.000 20.000 10.225 40.058 20.037 20.000 30.207 10.862 60.014 40.548 40.033 50.233 50.816 50.000 30.000 10.542 60.123 20.121 10.019 20.000 10.000 20.463 50.454 60.045 60.128 60.557 50.235 40.441 50.063 50.484 60.000 30.308 60.000 10.000 20.000 20.318 60.000 10.000 30.000 10.545 50.543 40.164 60.734 30.000 30.000 10.215 60.371 50.198 30.743 30.205 50.062 50.000 40.079 40.000 10.683 50.547 50.142 30.000 50.441 30.579 60.000 10.464 40.098 30.041 10.000 10.590 50.000 20.000 10.373 20.494 30.174 40.105 50.001 60.895 50.222 50.537 30.307 50.180 40.625 30.000 10.000 40.591 60.609 50.398 40.000 10.766 60.014 60.638 60.000 10.377 30.004 50.206 60.609 60.465 2
Ji Hou, Benjamin Graham, Matthias Nie├čner, Saining Xie: Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts. CVPR 2021
Minkowski 34Dpermissive0.253 50.463 50.154 60.102 50.381 60.084 10.134 60.000 10.000 10.000 10.386 40.141 60.279 60.737 50.703 50.014 60.164 40.000 30.663 20.092 50.000 30.224 50.291 20.531 20.056 60.000 40.242 50.000 10.000 30.013 40.331 50.000 30.000 10.035 60.001 10.858 40.059 50.650 50.000 30.056 50.353 50.299 40.670 50.000 10.000 10.284 50.484 50.071 30.594 50.720 50.710 50.027 50.068 60.813 30.000 10.005 10.492 40.164 10.274 50.111 60.571 50.307 60.293 40.307 60.150 30.163 60.531 50.002 10.545 10.932 40.093 60.000 10.000 40.002 40.159 40.368 60.581 50.440 60.228 60.406 20.282 60.294 50.000 20.000 10.189 50.060 10.036 30.000 30.000 30.897 30.000 60.525 50.025 60.205 60.771 60.000 30.000 10.593 40.108 40.044 30.000 40.000 10.000 20.282 60.589 50.094 50.169 50.466 60.227 60.419 60.125 30.757 30.002 10.334 50.000 10.000 20.000 20.357 40.000 10.000 30.000 10.582 40.513 60.337 30.612 60.000 30.000 10.250 50.352 60.136 60.724 50.655 10.280 20.000 40.046 60.000 10.606 60.559 40.159 20.102 10.445 20.655 20.000 10.310 60.117 10.000 40.000 10.581 60.026 10.000 10.265 60.483 50.084 60.097 60.044 40.865 60.142 60.588 20.351 40.272 20.596 60.000 10.003 20.622 50.720 30.096 60.000 10.771 50.016 50.772 40.000 10.302 40.194 30.214 50.621 50.197 6
C. Choy, J. Gwak, S. Savarese: 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks. CVPR 2019