The 3D semantic instance prediction task involves detecting and segmenting the object in an 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the average precision for each class. We report the mean average precision AP at overlap 0.25 (AP 25%), overlap 0.5 (AP 50%), and over overlaps in the range [0.5:0.95:0.05] (AP). Note that multiple predictions of the same ground truth instance are penalized as false positives.



This table lists the benchmark results for the 3D semantic instance scenario.




Method Infoavg ap 50%bathtubbedbookshelfcabinetchaircountercurtaindeskdoorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwindow
sorted bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort by
OccuSeg+instance0.672 11.000 10.758 60.682 30.576 10.842 10.477 10.504 100.524 20.567 10.585 10.451 20.557 41.000 10.751 10.797 20.563 51.000 10.467 3
GICN0.638 21.000 10.895 10.800 10.480 50.676 50.144 20.737 20.354 50.447 30.400 80.365 50.700 11.000 10.569 30.836 10.599 21.000 10.473 2
PointGroup0.636 31.000 10.765 40.624 40.505 40.797 20.116 50.696 30.384 40.441 40.559 20.476 10.596 31.000 10.666 20.756 40.556 70.997 60.513 1
MPA0.611 41.000 10.833 20.765 20.526 20.756 40.136 40.588 70.470 30.438 50.432 60.358 60.650 20.857 60.429 60.765 30.557 61.000 10.430 5
SSEN0.575 51.000 10.761 50.473 90.477 60.795 30.066 80.529 80.658 10.460 20.461 30.380 40.331 110.859 50.401 90.692 60.653 11.000 10.348 9
MTML0.549 61.000 10.807 30.588 70.327 90.647 60.004 150.815 10.180 90.418 60.364 100.182 90.445 61.000 10.442 50.688 70.571 41.000 10.396 6
Jean Lahoud, Bernard Ghanem, Marc Pollefeys, Martin R. Oswald: 3D Instance Segmentation via Multi-task Metric Learning. ICCV 2019 [oral]
Occipital-SCS0.512 71.000 10.716 80.509 80.506 30.611 80.092 70.602 60.177 100.346 90.383 90.165 100.442 70.850 90.386 100.618 90.543 80.889 110.389 7
3D-BoNet0.488 81.000 10.672 110.590 60.301 100.484 140.098 60.620 40.306 60.341 100.259 120.125 120.434 80.796 100.402 80.499 140.513 90.909 100.439 4
Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, Niki Trigoni: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds. NeurIPS 2019 Spotlight
PanopticFusion-inst0.478 90.667 110.712 100.595 50.259 120.550 120.000 180.613 50.175 110.250 130.434 40.437 30.411 100.857 60.485 40.591 120.267 160.944 80.359 8
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
ResNet-backbone0.459 101.000 10.737 70.159 170.259 110.587 100.138 30.475 110.217 80.416 70.408 70.128 110.315 120.714 110.411 70.536 130.590 30.873 130.304 10
MASCpermissive0.447 110.528 150.555 130.381 100.382 70.633 70.002 160.509 90.260 70.361 80.432 50.327 70.451 50.571 120.367 110.639 80.386 100.980 70.276 11
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
3D-SISpermissive0.382 121.000 10.432 150.245 130.190 130.577 110.013 130.263 130.033 170.320 110.240 130.075 140.422 90.857 60.117 150.699 50.271 150.883 120.235 13
Ji Hou, Angela Dai, Matthias Niessner: 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans. CVPR 2019
UNet-backbone0.319 130.667 110.715 90.233 140.189 140.479 150.008 140.218 140.067 160.201 140.173 140.107 130.123 150.438 130.150 130.615 100.355 110.916 90.093 18
R-PointNet0.306 140.500 160.405 160.311 110.348 80.589 90.054 90.068 170.126 120.283 120.290 110.028 160.219 130.214 160.331 120.396 160.275 140.821 150.245 12
RegionNet0.248 150.667 110.437 140.188 150.153 150.491 130.000 180.208 150.094 140.153 150.099 160.057 150.217 140.119 170.039 180.466 150.302 130.640 160.140 15
3D-BEVIS0.248 150.667 110.566 120.076 180.035 190.394 160.027 110.035 180.098 130.099 170.030 180.025 170.098 160.375 140.126 140.604 110.181 170.854 140.171 14
Cathrin Elich, Francis Engelmann, Jonas Schult, Theodora Kontogianni, Bastian Leibe: 3D-BEVIS: Birds-Eye-View Instance Segmentation.
Seg-Clusterpermissive0.215 170.370 170.337 180.285 120.105 160.325 170.025 120.282 120.085 150.105 160.107 150.007 190.079 170.317 150.114 160.309 180.304 120.587 170.123 17
Sgpn_scannet0.143 180.208 190.390 170.169 160.065 170.275 180.029 100.069 160.000 180.087 180.043 170.014 180.027 190.000 180.112 170.351 170.168 180.438 180.138 16
MaskRCNN 2d->3d Proj0.058 190.333 180.002 190.000 190.053 180.002 190.002 170.021 190.000 180.045 190.024 190.238 80.065 180.000 180.014 190.107 190.020 190.110 190.006 19