The 3D semantic instance prediction task involves detecting and segmenting the object in an 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the average precision for each class. We report the mean average precision AP at overlap 0.25 (AP 25%), overlap 0.5 (AP 50%), and over overlaps in the range [0.5:0.95:0.05] (AP). Note that multiple predictions of the same ground truth instance are penalized as false positives.



This table lists the benchmark results for the 3D semantic instance scenario.




Method Infoavg ap 50%bathtubbedbookshelfcabinetchaircountercurtaindeskdoorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwindow
sorted bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort by
Occipital-SCS0.512 11.000 10.716 20.509 30.506 10.611 30.092 30.602 30.177 50.346 40.383 40.165 40.442 20.850 40.386 40.618 30.543 20.889 60.389 2
3D-BoNet0.488 21.000 10.672 50.590 20.301 40.484 80.098 20.620 10.306 10.341 50.259 70.125 60.434 30.796 50.402 30.499 90.513 30.909 50.439 1
MTML0.481 31.000 10.666 60.377 50.272 50.709 10.001 120.579 40.254 30.361 30.318 50.095 80.432 41.000 10.184 70.601 60.487 40.938 30.384 3
PanopticFusion-inst0.478 40.667 60.712 40.595 10.259 70.550 70.000 130.613 20.175 60.250 80.434 10.437 10.411 60.857 20.485 10.591 70.267 100.944 20.359 4
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. arXiv
ResNet-backbone0.459 51.000 10.737 10.159 110.259 60.587 50.138 10.475 60.217 40.416 10.408 30.128 50.315 70.714 60.411 20.536 80.590 10.873 80.304 5
MASCpermissive0.447 60.528 90.555 80.381 40.382 20.633 20.002 100.509 50.260 20.361 20.432 20.327 20.451 10.571 70.367 50.639 20.386 50.980 10.276 6
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
3D-SISpermissive0.382 71.000 10.432 90.245 80.190 80.577 60.013 80.263 80.033 110.320 60.240 80.075 90.422 50.857 20.117 100.699 10.271 90.883 70.235 8
Ji Hou, Angela Dai, Matthias Niessner: 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans. CVPR 2019
UNet-backbone0.319 80.667 60.715 30.233 90.189 90.479 90.008 90.218 90.067 100.201 90.173 90.107 70.123 90.438 80.150 80.615 40.355 60.916 40.093 12
R-PointNet0.306 90.500 100.405 100.311 60.348 30.589 40.054 40.068 110.126 70.283 70.290 60.028 100.219 80.214 110.331 60.396 100.275 80.821 100.245 7
3D-BEVIS0.248 100.667 60.566 70.076 120.035 130.394 100.027 60.035 120.098 80.099 110.030 120.025 110.098 100.375 90.126 90.604 50.181 110.854 90.171 9
Cathrin Elich, Francis Engelmann, Jonas Schult, Theodora Kontogianni, Bastian Leibe: 3D-BEVIS: Birds-Eye-View Instance Segmentation.
Seg-Clusterpermissive0.215 110.370 110.337 120.285 70.105 100.325 110.025 70.282 70.085 90.105 100.107 100.007 130.079 110.317 100.114 110.309 120.304 70.587 110.123 11
Sgpn_scannet0.143 120.208 130.390 110.169 100.065 110.275 120.029 50.069 100.000 120.087 120.043 110.014 120.027 130.000 120.112 120.351 110.168 120.438 120.138 10
MaskRCNN 2d->3d Proj0.058 130.333 120.002 130.000 130.053 120.002 130.002 110.021 130.000 120.045 130.024 130.238 30.065 120.000 120.014 130.107 130.020 130.110 130.006 13