The 3D semantic instance prediction task involves detecting and segmenting the object in an 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the average precision for each class. We report the mean average precision AP at overlap 0.25 (AP 25%), overlap 0.5 (AP 50%), and over overlaps in the range [0.5:0.95:0.05] (AP). Note that multiple predictions of the same ground truth instance are penalized as false positives.



This table lists the benchmark results for the 3D semantic instance scenario.




Method Infoavg ap 50%bathtubbedbookshelfcabinetchaircountercurtaindeskdoorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwindow
sorted bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort by
GICN0.638 11.000 10.895 20.800 10.480 40.676 50.144 20.737 20.354 30.447 20.400 80.365 40.700 11.000 10.569 30.836 10.599 21.000 10.473 3
PointGroup0.636 21.000 10.765 50.624 40.505 30.797 20.116 60.696 40.384 20.441 30.559 10.476 10.596 51.000 10.666 10.756 40.556 60.997 50.513 2
OccuSeg0.634 31.000 10.902 10.771 20.461 50.814 10.282 10.583 90.328 50.472 10.471 20.295 70.600 31.000 10.650 20.664 70.587 41.000 10.537 1
MPA0.610 41.000 10.833 30.765 30.546 10.750 40.140 40.588 80.478 10.433 50.454 30.376 30.650 20.857 50.429 60.765 30.537 81.000 10.378 7
SSEN0.568 51.000 10.747 60.449 90.371 70.760 30.143 30.706 30.336 40.439 40.430 60.306 60.600 30.857 50.407 80.831 20.611 10.944 70.283 10
MTML0.549 61.000 10.807 40.588 70.327 90.647 60.004 150.815 10.180 90.418 60.364 100.182 90.445 71.000 10.442 50.688 60.571 51.000 10.396 5
Jean Lahoud, Bernard Ghanem, Marc Pollefeys, Martin R. Oswald: 3D Instance Segmentation via Multi-task Metric Learning. ICCV 2019 [oral]
Occipital-SCS0.512 71.000 10.716 80.509 80.506 20.611 80.092 80.602 70.177 100.346 90.383 90.165 100.442 80.850 90.386 100.618 90.543 70.889 110.389 6
3D-BoNet0.488 81.000 10.672 110.590 60.301 100.484 130.098 70.620 50.306 60.341 100.259 120.125 120.434 90.796 100.402 90.499 140.513 90.909 100.439 4
Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, Niki Trigoni: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds. NeurIPS 2019 Spotlight
PanopticFusion-inst0.478 90.667 110.712 100.595 50.259 120.550 120.000 180.613 60.175 110.250 130.434 40.437 20.411 110.857 50.485 40.591 120.267 150.944 70.359 8
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
ResNet-backbone0.459 101.000 10.737 70.159 160.259 110.587 100.138 50.475 110.217 80.416 70.408 70.128 110.315 120.714 110.411 70.536 130.590 30.873 130.304 9
MASCpermissive0.447 110.528 140.555 130.381 100.382 60.633 70.002 160.509 100.260 70.361 80.432 50.327 50.451 60.571 120.367 110.639 80.386 100.980 60.276 11
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
3D-SISpermissive0.382 121.000 10.432 140.245 130.190 130.577 110.013 130.263 130.033 160.320 110.240 130.075 140.422 100.857 50.117 150.699 50.271 140.883 120.235 13
Ji Hou, Angela Dai, Matthias Niessner: 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans. CVPR 2019
UNet-backbone0.319 130.667 110.715 90.233 140.189 140.479 140.008 140.218 140.067 150.201 140.173 140.107 130.123 140.438 130.150 130.615 100.355 110.916 90.093 17
R-PointNet0.306 140.500 150.405 150.311 110.348 80.589 90.054 90.068 160.126 120.283 120.290 110.028 150.219 130.214 160.331 120.396 150.275 130.821 150.245 12
3D-BEVIS0.248 150.667 110.566 120.076 170.035 180.394 150.027 110.035 170.098 130.099 160.030 170.025 160.098 150.375 140.126 140.604 110.181 160.854 140.171 14
Cathrin Elich, Francis Engelmann, Jonas Schult, Theodora Kontogianni, Bastian Leibe: 3D-BEVIS: Birds-Eye-View Instance Segmentation.
Seg-Clusterpermissive0.215 160.370 160.337 170.285 120.105 150.325 160.025 120.282 120.085 140.105 150.107 150.007 180.079 160.317 150.114 160.309 170.304 120.587 160.123 16
Sgpn_scannet0.143 170.208 180.390 160.169 150.065 160.275 170.029 100.069 150.000 170.087 170.043 160.014 170.027 180.000 170.112 170.351 160.168 170.438 170.138 15
MaskRCNN 2d->3d Proj0.058 180.333 170.002 180.000 180.053 170.002 180.002 170.021 180.000 170.045 180.024 180.238 80.065 170.000 170.014 180.107 180.020 180.110 180.006 18