The 3D semantic labeling task involves predicting a semantic labeling of a 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the PASCAL VOC intersection-over-union metric (IoU). IoU = TP/(TP+FP+FN), where TP, FP, and FN are the numbers of true positive, false positive, and false negative pixels, respectively. Predicted labels are evaluated per-vertex over the respective 3D scan mesh; for 3D approaches that operate on other representations like grids or points, the predicted labels should be mapped onto the mesh vertices (e.g., one such example for grid to mesh vertices is provided in the evaluation helpers).



This table lists the benchmark results for the ScanNet200 3D semantic label scenario.




Method Infoavg iouhead ioucommon ioutail iouwallchairfloortabledoorcouchcabinetshelfdeskoffice chairbedpillowsinkpicturewindowtoiletbookshelfmonitorcurtainbookarmchaircoffee tableboxrefrigeratorlampkitchen cabinettowelclothestvnightstandcounterdresserstoolcushionplantceilingbathtubend tabledining tablekeyboardbagbackpacktoilet paperprintertv standwhiteboardblanketshower curtaintrash canclosetstairsmicrowavestoveshoecomputer towerbottlebinottomanbenchboardwashing machinemirrorcopierbasketsofa chairfile cabinetfanlaptopshowerpaperpersonpaper towel dispenserovenblindsrackplateblackboardpianosuitcaserailradiatorrecycling bincontainerwardrobesoap dispensertelephonebucketclockstandlightlaundry basketpipeclothes dryerguitartoilet paper holderseatspeakercolumnbicycleladderbathroom stallshower wallcupjacketstorage bincoffee makerdishwasherpaper towel rollmachinematwindowsillbartoasterbulletin boardironing boardfireplacesoap dishkitchen counterdoorframetoilet paper dispensermini fridgefire extinguisherballhatshower curtain rodwater coolerpaper cuttertrayshower doorpillarledgetoaster ovenmousetoilet seat cover dispenserfurniturecartstorage containerscaletissue boxlight switchcratepower outletdecorationsignprojectorcloset doorvacuum cleanercandleplungerstuffed animalheadphonesdish rackbroomguitar caserange hooddustpanhair dryerwater bottlehandicap barpurseventshower floorwater pitchermailboxbowlpaper bagalarm clockmusic standprojector screendividerlaundry detergentbathroom counterobjectbathroom vanitycloset walllaundry hamperbathroom stall doorceiling lighttrash bindumbbellstair railtubebathroom cabinetcd casecloset rodcoffee kettlestructureshower headkeyboard pianocase of water bottlescoat rackstorage organizerfolded chairfire alarmpower stripcalendarposterpotted plantluggagemattress
sort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysorted bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort by
OA-CNN-L_ScanNet2000.333 20.558 10.269 10.124 30.821 10.703 10.946 10.569 10.662 10.748 10.487 10.455 10.572 20.000 60.789 10.534 10.736 20.271 10.713 10.949 10.498 50.877 20.860 10.332 10.706 10.474 10.788 20.406 30.637 10.495 10.355 20.805 20.592 50.015 60.396 10.602 30.000 10.799 20.876 10.713 60.276 10.000 50.493 20.080 20.448 40.363 10.661 20.833 10.262 10.125 10.823 20.665 10.076 20.720 10.557 20.637 20.517 20.672 40.227 20.000 30.158 20.496 20.843 30.352 20.835 30.000 10.103 50.711 10.527 10.526 20.320 10.000 10.568 10.625 30.067 10.000 30.000 10.001 20.806 10.836 10.621 10.591 30.373 20.314 20.668 20.398 20.003 20.000 30.000 10.016 60.024 10.043 60.906 10.000 10.052 30.000 50.384 10.330 40.342 50.100 20.223 20.183 20.112 20.476 20.313 20.130 30.196 20.112 20.370 50.000 10.234 30.071 30.160 10.403 20.398 30.492 50.197 10.076 40.272 10.000 10.200 60.560 10.735 20.000 10.000 50.000 30.110 10.002 30.021 20.412 20.000 30.000 20.000 20.000 10.000 10.794 20.000 10.445 10.000 10.022 20.509 20.000 10.517 50.000 10.000 10.001 60.245 20.915 20.024 20.089 10.000 10.262 10.000 10.103 50.524 20.392 20.515 20.013 60.251 30.411 40.662 10.001 30.000 10.473 30.000 10.000 20.150 30.699 10.000 20.000 10.000 10.166 20.000 20.024 10.000 30.000 1
CeCo0.340 10.551 20.247 20.181 10.784 20.661 30.939 20.564 20.624 20.721 20.484 20.429 20.575 10.027 40.774 20.503 30.753 10.242 30.656 20.945 20.534 10.865 30.860 10.177 60.616 30.400 20.818 10.579 10.615 20.367 30.408 10.726 40.633 10.162 10.360 20.619 10.000 10.828 10.873 20.924 10.109 40.083 20.564 10.057 60.475 20.266 20.781 10.767 20.257 20.100 30.825 10.663 20.048 60.620 40.551 30.595 40.532 10.692 30.246 10.000 30.213 10.615 10.861 20.376 10.900 10.000 10.102 60.660 20.321 40.547 10.226 20.000 10.311 20.742 10.011 30.006 20.000 10.000 30.546 60.824 20.345 30.665 10.450 10.435 10.683 10.411 10.338 10.000 30.000 10.030 40.000 20.068 40.892 20.000 10.063 20.000 50.257 20.304 50.387 30.079 40.228 10.190 10.000 60.586 10.347 10.133 20.000 30.037 30.377 40.000 10.384 20.006 50.003 30.421 10.410 20.643 10.171 30.121 20.142 50.000 10.510 40.447 20.474 40.000 10.000 50.286 10.083 20.000 40.000 30.603 10.096 10.063 10.000 20.000 10.000 10.898 10.000 10.429 20.000 10.400 10.550 10.000 10.633 10.000 10.000 10.377 10.000 60.916 10.000 40.000 20.000 10.000 30.000 10.102 60.499 40.296 30.463 30.089 30.304 10.740 10.401 50.010 10.000 10.560 10.000 10.000 20.709 10.652 20.000 20.000 10.000 10.143 30.000 20.000 20.609 10.000 1
: Understanding Imbalanced Semantic Segmentation Through Neural Collapse.
AWCS0.305 30.508 30.225 30.142 20.782 30.634 60.937 30.489 40.578 30.721 20.364 40.355 30.515 30.023 50.764 30.523 20.707 40.264 20.633 30.922 30.507 40.886 10.804 40.179 40.436 60.300 30.656 50.529 20.501 40.394 20.296 50.820 10.603 20.131 20.179 60.619 10.000 10.707 50.865 30.773 20.171 20.010 40.484 30.063 40.463 30.254 30.332 50.649 40.220 40.100 30.729 40.613 40.071 40.582 50.628 10.702 10.424 40.749 10.137 40.000 30.142 30.360 30.863 10.305 30.877 20.000 10.173 10.606 30.337 30.478 30.154 40.000 10.253 30.664 20.000 40.000 30.000 10.000 30.626 40.782 30.302 50.602 20.185 50.282 30.651 30.317 30.000 30.000 30.000 10.022 50.000 20.154 10.876 30.000 10.014 40.063 40.029 60.553 10.467 20.084 30.124 30.157 50.049 50.373 30.252 30.097 40.000 30.219 10.542 10.000 10.392 10.172 20.000 50.339 30.417 10.533 40.093 40.115 30.195 30.000 10.516 30.288 50.741 10.000 10.001 40.233 20.056 30.000 40.159 10.334 30.077 20.000 20.000 20.000 10.000 10.749 30.000 10.411 30.000 10.008 30.452 30.000 10.595 20.000 10.000 10.220 30.006 40.894 40.006 30.000 20.000 10.000 30.000 10.112 30.504 30.404 10.551 10.093 20.129 60.484 30.381 60.000 40.000 10.396 40.000 10.000 20.620 20.402 60.000 20.000 10.000 10.142 40.000 20.000 20.512 20.000 1
LGroundpermissive0.272 40.485 40.184 40.106 40.778 40.676 20.932 40.479 60.572 40.718 40.399 30.265 40.453 50.085 20.745 40.446 40.726 30.232 50.622 40.901 40.512 30.826 40.786 50.178 50.549 40.277 40.659 40.381 40.518 30.295 60.323 30.777 30.599 30.028 40.321 30.363 50.000 10.708 40.858 40.746 40.063 50.022 30.457 40.077 30.476 10.243 40.402 30.397 60.233 30.077 60.720 60.610 50.103 10.629 30.437 60.626 30.446 30.702 20.190 30.005 10.058 50.322 40.702 50.244 40.768 40.000 10.134 40.552 40.279 50.395 40.147 50.000 10.207 40.612 40.000 40.000 30.000 10.000 30.658 30.566 40.323 40.525 60.229 40.179 40.467 60.154 50.000 30.002 10.000 10.051 10.000 20.127 20.703 40.000 10.000 50.216 10.112 50.358 30.547 10.187 10.092 50.156 60.055 40.296 40.252 30.143 10.000 30.014 40.398 30.000 10.028 50.173 10.000 50.265 50.348 40.415 60.179 20.019 50.218 20.000 10.597 20.274 60.565 30.000 10.012 30.000 30.039 50.022 20.000 30.117 40.000 30.000 20.000 20.000 10.000 10.324 50.000 10.384 40.000 10.000 40.251 60.000 10.566 30.000 10.000 10.066 40.404 10.886 50.199 10.000 20.000 10.059 20.000 10.136 10.540 10.127 60.295 40.085 40.143 50.514 20.413 40.000 40.000 10.498 20.000 10.000 20.000 40.623 30.000 20.000 10.000 10.132 50.000 20.000 20.000 30.000 1
David Rozenberszki, Or Litany, Angela Dai: Language-Grounded Indoor 3D Semantic Segmentation in the Wild. arXiv
CSC-Pretrainpermissive0.249 60.455 60.171 50.079 60.766 60.659 40.930 60.494 30.542 60.700 60.314 60.215 60.430 60.121 10.697 60.441 50.683 50.235 40.609 60.895 50.476 60.816 50.770 60.186 30.634 20.216 60.734 30.340 50.471 50.307 50.293 60.591 60.542 60.076 30.205 50.464 40.000 10.484 60.832 60.766 30.052 60.000 50.413 50.059 50.418 50.222 50.318 60.609 50.206 60.112 20.743 30.625 30.076 20.579 60.548 40.590 50.371 50.552 60.081 50.003 20.142 30.201 60.638 60.233 50.686 60.000 10.142 30.444 60.375 20.247 60.198 30.000 10.128 60.454 60.019 20.097 10.000 10.000 30.553 50.557 50.373 20.545 50.164 60.014 60.547 50.174 40.000 30.002 10.000 10.037 20.000 20.063 50.664 60.000 10.000 50.130 20.170 30.152 60.335 60.079 40.110 40.175 30.098 30.175 60.166 50.045 60.207 10.014 40.465 20.000 10.001 60.001 60.046 20.299 40.327 50.537 30.033 50.012 60.186 40.000 10.205 50.377 30.463 50.000 10.058 20.000 30.055 40.041 10.000 30.105 50.000 30.000 20.000 20.000 10.000 10.398 40.000 10.308 60.000 10.000 40.319 40.000 10.543 40.000 10.000 10.062 50.004 50.862 60.000 40.000 20.000 10.000 30.000 10.123 20.316 50.225 40.250 50.094 10.180 40.332 50.441 30.000 40.000 10.310 60.000 10.000 20.000 40.592 40.000 20.000 10.000 10.203 10.000 20.000 20.000 30.000 1
Ji Hou, Benjamin Graham, Matthias Nie├čner, Saining Xie: Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts. CVPR 2021
Minkowski 34Dpermissive0.253 50.463 50.154 60.102 50.771 50.650 50.932 40.483 50.571 50.710 50.331 50.250 50.492 40.044 30.703 50.419 60.606 60.227 60.621 50.865 60.531 20.771 60.813 30.291 20.484 50.242 50.612 60.282 60.440 60.351 40.299 40.622 50.593 40.027 50.293 40.310 60.000 10.757 30.858 40.737 50.150 30.164 10.368 60.084 10.381 60.142 60.357 40.720 30.214 50.092 50.724 50.596 60.056 50.655 20.525 50.581 60.352 60.594 50.056 60.000 30.014 60.224 50.772 40.205 60.720 50.000 10.159 20.531 50.163 60.294 50.136 60.000 10.169 50.589 50.000 40.000 30.000 10.002 10.663 20.466 60.265 60.582 40.337 30.016 50.559 40.084 60.000 30.000 30.000 10.036 30.000 20.125 30.670 50.000 10.102 10.071 30.164 40.406 20.386 40.046 60.068 60.159 40.117 10.284 50.111 60.094 50.000 30.000 60.197 60.000 10.044 40.013 40.002 40.228 60.307 60.588 20.025 60.545 10.134 60.000 10.655 10.302 40.282 60.000 10.060 10.000 30.035 60.000 40.000 30.097 60.000 30.000 20.005 10.000 10.000 10.096 60.000 10.334 50.000 10.000 40.274 50.000 10.513 60.000 10.000 10.280 20.194 30.897 30.000 40.000 20.000 10.000 30.000 10.108 40.279 60.189 50.141 60.059 50.272 20.307 60.445 20.003 20.000 10.353 50.000 10.026 10.000 40.581 50.001 10.000 10.000 10.093 60.002 10.000 20.000 30.000 1
C. Choy, J. Gwak, S. Savarese: 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks. CVPR 2019