The 3D semantic instance prediction task involves detecting and segmenting the object in an 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the average precision for each class. We report the mean average precision AP at overlap 0.25 (AP 25%), overlap 0.5 (AP 50%), and over overlaps in the range [0.5:0.95:0.05] (AP). Note that multiple predictions of the same ground truth instance are penalized as false positives.



This table lists the benchmark results for the 3D semantic instance scenario.




Method Infoavg ap 50%bathtubbedbookshelfcabinetchaircountercurtaindeskdoorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwindow
sorted bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort by
PanopticFusion-inst0.478 10.667 30.712 30.595 10.259 40.550 50.000 100.613 10.175 30.250 50.434 10.437 10.411 30.857 10.485 10.591 50.267 70.944 20.359 1
ResNet-backbone0.459 21.000 10.737 10.159 90.259 30.587 30.138 10.475 30.217 20.416 10.408 30.128 40.315 40.714 30.411 20.536 60.590 10.873 50.304 2
MASCpermissive0.447 30.528 70.555 60.381 20.382 10.633 10.002 80.509 20.260 10.361 20.432 20.327 20.451 10.571 40.367 30.639 20.386 20.980 10.276 3
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
3D-SIS0.382 41.000 10.432 70.245 60.190 50.577 40.013 60.263 50.033 80.320 30.240 50.075 60.422 20.857 10.117 70.699 10.271 60.883 40.235 5
UNet-backbone0.319 50.667 30.715 20.233 70.189 60.479 60.008 70.218 60.067 70.201 60.173 60.107 50.123 70.438 50.150 50.615 30.355 30.916 30.093 9
R-PointNet0.306 60.500 80.405 80.311 40.348 20.589 20.054 20.068 90.126 40.283 40.290 40.028 70.219 50.214 80.331 40.396 80.275 50.821 70.245 4
3D-BEVIS0.248 70.667 30.566 50.076 100.035 100.394 70.027 40.035 100.098 50.099 90.030 90.025 80.098 80.375 60.126 60.604 40.181 80.854 60.171 6
Seg-Clusterpermissive0.215 80.370 90.337 100.285 50.105 70.325 90.025 50.282 40.085 60.105 70.107 70.007 100.079 90.317 70.114 80.309 100.304 40.587 80.123 8
MTML0.212 90.667 30.614 40.337 30.027 110.390 80.000 100.118 70.001 90.100 80.028 100.000 110.167 60.143 90.046 100.500 70.105 100.570 90.003 11
Sgpn_scannet0.143 100.208 110.390 90.169 80.065 80.275 100.029 30.069 80.000 100.087 100.043 80.014 90.027 110.000 100.112 90.351 90.168 90.438 100.138 7
MaskRCNN 2d->3d Proj0.058 110.333 100.002 110.000 110.053 90.002 110.002 90.021 110.000 100.045 110.024 110.238 30.065 100.000 100.014 110.107 110.020 110.110 110.006 10