The 3D semantic instance prediction task involves detecting and segmenting the object in an 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the average precision for each class. We report the mean average precision AP at overlap 0.25 (AP 25%), overlap 0.5 (AP 50%), and over overlaps in the range [0.5:0.95:0.05] (AP). Note that multiple predictions of the same ground truth instance are penalized as false positives.



This table lists the benchmark results for the 3D semantic instance scenario.




Method Infoavg ap 50%bathtubbedbookshelfcabinetchaircountercurtaindeskdoorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwindow
sorted bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort by
R-PointNet0.306 10.500 40.405 40.311 20.348 10.589 10.054 10.068 40.126 20.283 10.290 10.028 30.219 10.214 30.331 10.396 40.275 10.821 20.245 1
3D-SIS0.271 20.528 30.558 30.241 30.272 20.488 20.000 50.093 20.167 10.179 20.231 20.040 20.182 20.286 20.168 20.546 20.155 40.555 40.185 2
3D-BEVIS0.248 30.667 10.566 20.076 50.035 50.394 30.027 30.035 50.098 30.099 40.030 40.025 40.098 40.375 10.126 30.604 10.181 20.854 10.171 3
MTML0.212 40.667 10.614 10.337 10.027 60.390 40.000 50.118 10.001 40.100 30.028 50.000 60.167 30.143 40.046 50.500 30.105 50.570 30.003 6
Sgpn_scannet0.143 50.208 60.390 50.169 40.065 30.275 50.029 20.069 30.000 50.087 50.043 30.014 50.027 60.000 50.112 40.351 50.168 30.438 50.138 4
MaskRCNN 2d->3d Proj0.058 60.333 50.002 60.000 60.053 40.002 60.002 40.021 60.000 50.045 60.024 60.238 10.065 50.000 50.014 60.107 60.020 60.110 60.006 5