The 3D semantic labeling task involves predicting a semantic labeling of a 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the PASCAL VOC intersection-over-union metric (IoU). IoU = TP/(TP+FP+FN), where TP, FP, and FN are the numbers of true positive, false positive, and false negative pixels, respectively. Predicted labels are evaluated per-vertex over the respective 3D scan mesh; for 3D approaches that operate on other representations like grids or points, the predicted labels should be mapped onto the mesh vertices (e.g., one such example for grid to mesh vertices is provided in the evaluation helpers).



This table lists the benchmark results for the 3D semantic label scenario.


Method Infoavg ioubathtubbedbookshelfcabinetchaircountercurtaindeskdoorfloorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwallwindow
sorted bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort by
SparseConvNet0.725 10.647 50.821 10.846 10.721 10.869 10.533 10.754 40.603 20.614 20.955 10.572 10.325 10.710 10.870 20.724 10.823 10.628 20.934 10.865 10.683 1
KP-FCNN0.694 20.849 10.770 30.810 20.685 20.813 30.438 30.791 20.566 30.616 10.944 30.500 30.216 60.559 40.880 10.690 20.758 30.627 30.922 20.832 20.613 3
MinkowskiNet340.679 30.811 20.734 40.739 40.641 30.804 40.413 60.759 30.696 10.545 40.938 70.518 20.141 150.623 20.757 30.680 30.723 40.684 10.896 30.821 30.651 2
joint point-based0.634 40.614 70.778 20.667 60.633 40.825 20.420 50.804 10.467 60.561 30.951 20.494 40.291 20.566 30.458 70.579 40.764 20.559 40.838 40.814 40.598 4
TextureNet0.566 50.672 30.664 70.671 50.494 60.719 60.445 20.678 60.411 110.396 80.935 80.356 80.225 40.412 90.535 60.565 50.636 80.464 90.794 80.680 120.568 5
DVVNet0.562 60.648 40.700 50.770 30.586 50.687 100.333 80.650 70.514 40.475 50.906 140.359 70.223 50.340 110.442 80.422 130.668 50.501 70.708 110.779 50.534 8
PointConv0.556 70.636 60.640 100.574 110.472 80.739 50.430 40.433 100.418 100.445 70.944 30.372 60.185 100.464 70.575 50.540 60.639 70.505 60.827 50.762 60.515 9
PanopticFusion-label0.529 80.491 140.688 60.604 90.386 110.632 140.225 180.705 50.434 80.293 130.815 170.348 90.241 30.499 60.669 40.507 70.649 60.442 120.796 70.602 160.561 6
3DMV, FTSDF0.501 90.558 110.608 130.424 170.478 70.690 90.246 140.586 80.468 50.450 60.911 120.394 50.160 120.438 80.212 140.432 120.541 120.475 80.742 100.727 80.477 11
PCNN0.498 100.559 100.644 90.560 120.420 100.711 80.229 160.414 110.436 70.352 100.941 60.324 100.155 130.238 150.387 90.493 80.529 130.509 50.813 60.751 70.504 10
3DMV0.484 110.484 150.538 150.643 70.424 90.606 170.310 90.574 90.433 90.378 90.796 180.301 110.214 70.537 50.208 150.472 110.507 160.413 150.693 120.602 160.539 7
Angela Dai, Matthias Niessner: 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation. ECCV'18
PointCNN with RGBpermissive0.458 120.577 90.611 120.356 190.321 160.715 70.299 110.376 140.328 160.319 110.944 30.285 130.164 110.216 170.229 130.484 100.545 110.456 100.755 90.709 90.475 12
Yangyan Li, Rui Bu, Mingchao Sun, Baoquan Chen: PointCNN. NeurIPS 2018
SurfaceConvPF0.442 130.505 130.622 110.380 180.342 150.654 120.227 170.397 130.367 140.276 150.924 100.240 150.198 80.359 100.262 110.366 150.581 90.435 130.640 140.668 130.398 14
Hao Pan, Shilin Liu, Yang Liu, Xin Tong: Convolutional Neural Networks on 3D Surfaces Using Parallel Frames.
PNET20.442 130.548 120.548 140.597 100.363 130.628 150.300 100.292 150.374 130.307 120.881 160.268 140.186 90.238 150.204 160.407 140.506 170.449 110.667 130.620 150.462 13
Tangent Convolutionspermissive0.438 150.437 170.646 80.474 140.369 120.645 130.353 70.258 170.282 180.279 140.918 110.298 120.147 140.283 120.294 100.487 90.562 100.427 140.619 150.633 140.352 16
Maxim Tatarchenko, Jaesik Park, Vladlen Koltun, Qian-Yi Zhou: Tangent convolutions for dense prediction in 3d. CVPR 2018
SPLAT Netcopyleft0.393 160.472 160.511 160.606 80.311 170.656 110.245 150.405 120.328 160.197 180.927 90.227 170.000 200.001 200.249 120.271 200.510 140.383 170.593 160.699 100.267 18
Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, Jan Kautz: SPLATNet: Sparse Lattice Networks for Point Cloud Processing. CVPR 2018
ScanNet+FTSDF0.383 170.297 190.491 170.432 160.358 140.612 160.274 120.116 190.411 110.265 160.904 150.229 160.079 180.250 130.185 170.320 180.510 140.385 160.548 170.597 180.394 15
PointNet++permissive0.339 180.584 80.478 180.458 150.256 190.360 200.250 130.247 180.278 190.261 170.677 200.183 180.117 160.212 180.145 190.364 160.346 200.232 200.548 170.523 190.252 19
Charles R. Qi, Li Yi, Hao Su, Leonidas J. Guibas: pointnet++: deep hierarchical feature learning on point sets in a metric space.
SSC-UNetpermissive0.308 190.353 180.290 200.278 200.166 200.553 180.169 200.286 160.147 200.148 200.908 130.182 190.064 190.023 190.018 200.354 170.363 180.345 180.546 190.685 110.278 17
ScanNetpermissive0.306 200.203 200.366 190.501 130.311 170.524 190.211 190.002 200.342 150.189 190.786 190.145 200.102 170.245 140.152 180.318 190.348 190.300 190.460 200.437 200.182 20
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nie├čner: ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. CVPR'17