The 3D semantic instance prediction task involves detecting and segmenting the object in an 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the average precision for each class. We report the mean average precision AP at overlap 0.25 (AP 25%), overlap 0.5 (AP 50%), and over overlaps in the range [0.5:0.95:0.05] (AP). Note that multiple predictions of the same ground truth instance are penalized as false positives.



This table lists the benchmark results for the 3D semantic instance scenario.




Method Infoavg ap 50%bathtubbedbookshelfcabinetchaircountercurtaindeskdoorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwindow
sorted bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort by
PointGroup0.636 11.000 10.765 40.624 40.505 30.797 20.116 50.696 30.384 20.441 20.559 10.476 10.596 21.000 10.666 10.756 20.556 50.997 50.513 2
OccuSeg0.634 21.000 10.902 10.771 10.461 40.814 10.282 10.583 70.328 30.472 10.471 20.295 60.600 11.000 10.650 20.664 60.587 21.000 10.537 1
MPA0.595 31.000 10.833 20.668 30.517 10.748 30.131 40.539 80.466 10.423 30.420 50.362 30.589 30.857 50.424 60.755 30.558 41.000 10.420 4
GICN0.586 41.000 10.716 60.769 20.438 50.644 50.140 20.714 20.292 50.371 60.305 90.353 40.550 41.000 10.525 30.789 10.531 71.000 10.412 5
MTML0.549 51.000 10.807 30.588 70.327 80.647 40.004 140.815 10.180 80.418 40.364 80.182 80.445 61.000 10.442 50.688 50.571 31.000 10.396 6
Jean Lahoud, Bernard Ghanem, Marc Pollefeys, Martin R. Oswald: 3D Instance Segmentation via Multi-task Metric Learning. ICCV 2019 [oral]
Occipital-SCS0.512 61.000 10.716 60.509 80.506 20.611 70.092 70.602 60.177 90.346 80.383 70.165 90.442 70.850 80.386 90.618 80.543 60.889 100.389 7
3D-BoNet0.488 71.000 10.672 100.590 60.301 90.484 120.098 60.620 40.306 40.341 90.259 110.125 110.434 80.796 90.402 80.499 130.513 80.909 90.439 3
Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, Niki Trigoni: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds. NeurIPS 2019 Spotlight
PanopticFusion-inst0.478 80.667 100.712 90.595 50.259 110.550 110.000 170.613 50.175 100.250 120.434 30.437 20.411 100.857 50.485 40.591 110.267 140.944 70.359 8
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
ResNet-backbone0.459 91.000 10.737 50.159 150.259 100.587 90.138 30.475 100.217 70.416 50.408 60.128 100.315 110.714 100.411 70.536 120.590 10.873 120.304 9
MASCpermissive0.447 100.528 130.555 120.381 90.382 60.633 60.002 150.509 90.260 60.361 70.432 40.327 50.451 50.571 110.367 100.639 70.386 90.980 60.276 10
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
3D-SISpermissive0.382 111.000 10.432 130.245 120.190 120.577 100.013 120.263 120.033 150.320 100.240 120.075 130.422 90.857 50.117 140.699 40.271 130.883 110.235 12
Ji Hou, Angela Dai, Matthias Niessner: 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans. CVPR 2019
UNet-backbone0.319 120.667 100.715 80.233 130.189 130.479 130.008 130.218 130.067 140.201 130.173 130.107 120.123 130.438 120.150 120.615 90.355 100.916 80.093 16
R-PointNet0.306 130.500 140.405 140.311 100.348 70.589 80.054 80.068 150.126 110.283 110.290 100.028 140.219 120.214 150.331 110.396 140.275 120.821 140.245 11
3D-BEVIS0.248 140.667 100.566 110.076 160.035 170.394 140.027 100.035 160.098 120.099 150.030 160.025 150.098 140.375 130.126 130.604 100.181 150.854 130.171 13
Cathrin Elich, Francis Engelmann, Jonas Schult, Theodora Kontogianni, Bastian Leibe: 3D-BEVIS: Birds-Eye-View Instance Segmentation.
Seg-Clusterpermissive0.215 150.370 150.337 160.285 110.105 140.325 150.025 110.282 110.085 130.105 140.107 140.007 170.079 150.317 140.114 150.309 160.304 110.587 150.123 15
Sgpn_scannet0.143 160.208 170.390 150.169 140.065 150.275 160.029 90.069 140.000 160.087 160.043 150.014 160.027 170.000 160.112 160.351 150.168 160.438 160.138 14
MaskRCNN 2d->3d Proj0.058 170.333 160.002 170.000 170.053 160.002 170.002 160.021 170.000 160.045 170.024 170.238 70.065 160.000 160.014 170.107 170.020 170.110 170.006 17