The 3D semantic labeling task involves predicting a semantic labeling of a 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the PASCAL VOC intersection-over-union metric (IoU). IoU = TP/(TP+FP+FN), where TP, FP, and FN are the numbers of true positive, false positive, and false negative pixels, respectively. Predicted labels are evaluated per-vertex over the respective 3D scan mesh; for 3D approaches that operate on other representations like grids or points, the predicted labels should be mapped onto the mesh vertices (e.g., one such example for grid to mesh vertices is provided in the evaluation helpers).



This table lists the benchmark results for the 3D semantic label scenario.


Method Infoavg ioubathtubbedbookshelfcabinetchaircountercurtaindeskdoorfloorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwallwindow
sorted bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort by
SparseConvNet0.726 10.629 60.801 10.858 10.713 10.884 10.505 10.799 10.636 20.628 10.956 10.602 10.299 10.712 10.858 10.717 10.808 10.629 20.929 10.858 10.694 1
MinkowskiNet340.679 20.811 10.734 30.739 30.641 20.804 20.413 40.759 30.696 10.545 20.938 50.518 20.141 100.623 20.757 20.680 20.723 30.684 10.896 20.821 20.651 2
joint point-based0.621 30.645 40.746 20.612 60.571 40.795 30.386 50.798 20.485 40.539 30.943 40.445 30.287 20.520 40.418 60.635 30.744 20.570 30.859 30.795 30.628 3
TextureNet0.566 40.672 20.664 50.671 40.494 50.719 50.445 20.678 40.411 80.396 70.935 60.356 70.225 30.412 70.535 40.565 40.636 60.464 80.794 60.680 100.568 4
DVVNet0.562 50.648 30.700 40.770 20.586 30.687 80.333 70.650 50.514 30.475 40.906 120.359 60.223 40.340 90.442 50.422 100.668 40.501 60.708 80.779 40.534 6
PointConv0.556 60.636 50.640 70.574 80.472 70.739 40.430 30.433 80.418 70.445 60.944 20.372 50.185 70.464 50.575 30.540 50.639 50.505 50.827 40.762 50.515 7
3DMV, FTSDF0.501 70.558 80.608 90.424 130.478 60.690 70.246 110.586 60.468 50.450 50.911 100.394 40.160 80.438 60.212 110.432 90.541 90.475 70.742 70.727 70.477 9
3DMV0.484 80.484 110.538 110.643 50.424 80.606 130.310 80.574 70.433 60.378 80.796 140.301 80.214 50.537 30.208 120.472 80.507 130.413 110.693 90.602 130.539 5
Angela Dai, Matthias Niessner: 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation. ECCV'18
PointCNN with RGBpermissive0.479 90.510 90.583 100.417 140.414 90.708 60.241 130.367 110.405 100.323 90.944 20.300 90.132 110.226 130.417 70.534 60.525 100.511 40.806 50.743 60.479 8
Yangyan Li, Rui Bu, Mingchao Sun, Baoquan Chen: PointCNN. NIPS 2018
SurfaceConvPF0.442 100.505 100.622 80.380 150.342 120.654 100.227 140.397 100.367 110.276 110.924 80.240 110.198 60.359 80.262 90.366 110.581 70.435 90.640 100.668 110.398 10
Hao Pan, Shilin Liu, Yang Liu, Xin Tong: Convolutional Neural Networks on 3D Surfaces Using Parallel Frames.
Tangent Convolutionspermissive0.438 110.437 130.646 60.474 100.369 100.645 110.353 60.258 130.282 140.279 100.918 90.298 100.147 90.283 100.294 80.487 70.562 80.427 100.619 110.633 120.352 12
Maxim Tatarchenko, Jaesik Park, Vladlen Koltun, Qian-Yi Zhou: Tangent convolutions for dense prediction in 3d. CVPR 2018
SPLAT Netcopyleft0.393 120.472 120.511 120.606 70.311 130.656 90.245 120.405 90.328 130.197 140.927 70.227 130.000 160.001 160.249 100.271 160.510 110.383 130.593 120.699 80.267 14
Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, Jan Kautz: SPLATNet: Sparse Lattice Networks for Point Cloud Processing. CVPR 2018
ScanNet+FTSDF0.383 130.297 150.491 130.432 120.358 110.612 120.274 90.116 150.411 80.265 120.904 130.229 120.079 140.250 110.185 130.320 140.510 110.385 120.548 130.597 140.394 11
PointNet++permissive0.339 140.584 70.478 140.458 110.256 150.360 160.250 100.247 140.278 150.261 130.677 160.183 140.117 120.212 140.145 150.364 120.346 160.232 160.548 130.523 150.252 15
Charles R. Qi, Li Yi, Hao Su, Leonidas J. Guibas: pointnet++: deep hierarchical feature learning on point sets in a metric space.
SSC-UNetpermissive0.308 150.353 140.290 160.278 160.166 160.553 140.169 160.286 120.147 160.148 160.908 110.182 150.064 150.023 150.018 160.354 130.363 140.345 140.546 150.685 90.278 13
ScanNetpermissive0.306 160.203 160.366 150.501 90.311 130.524 150.211 150.002 160.342 120.189 150.786 150.145 160.102 130.245 120.152 140.318 150.348 150.300 150.460 160.437 160.182 16
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nie├čner: ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. CVPR'17