The 3D semantic instance prediction task involves detecting and segmenting the object in an 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the average precision for each class. We report the mean average precision AP at overlap 0.25 (AP 25%), overlap 0.5 (AP 50%), and over overlaps in the range [0.5:0.95:0.05] (AP). Note that multiple predictions of the same ground truth instance are penalized as false positives.



This table lists the benchmark results for the 3D semantic instance scenario.




Method Infoavg ap 50%bathtubbedbookshelfcabinetchaircountercurtaindeskdoorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwindow
sorted bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort by
OccuSeg+instance0.672 11.000 10.758 70.682 30.576 10.842 10.477 10.504 110.524 20.567 10.585 10.451 20.557 61.000 10.751 10.797 20.563 51.000 10.467 4
GICN0.638 21.000 10.895 10.800 10.480 50.676 70.144 30.737 20.354 60.447 40.400 100.365 60.700 11.000 10.569 30.836 10.599 21.000 10.473 3
PointGroup0.636 31.000 10.765 50.624 40.505 40.797 30.116 60.696 30.384 50.441 50.559 20.476 10.596 41.000 10.666 20.756 40.556 80.997 60.513 1
Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi-Wing Fu, Jiaya Jia: PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation. CVPR 2020 [oral]
3D-MPA0.611 41.000 10.833 20.765 20.526 20.756 60.136 50.588 80.470 30.438 60.432 80.358 70.650 20.857 60.429 70.765 30.557 71.000 10.430 6
Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, Matthias Nie├čner: 3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation. CVPR 2020
PCJC0.578 51.000 10.810 30.583 80.449 80.813 20.042 110.603 60.341 70.490 20.465 40.410 40.650 20.835 100.264 140.694 60.561 60.889 130.504 2
SSEN0.575 61.000 10.761 60.473 100.477 60.795 40.066 90.529 90.658 10.460 30.461 50.380 50.331 130.859 50.401 100.692 70.653 11.000 10.348 10
MTML0.549 71.000 10.807 40.588 70.327 110.647 80.004 170.815 10.180 110.418 70.364 120.182 100.445 81.000 10.442 60.688 80.571 41.000 10.396 7
Jean Lahoud, Bernard Ghanem, Marc Pollefeys, Martin R. Oswald: 3D Instance Segmentation via Multi-task Metric Learning. ICCV 2019 [oral]
Sparse R-CNN0.515 81.000 10.538 150.282 140.468 70.790 50.173 20.345 130.429 40.413 90.484 30.176 110.595 50.591 130.522 40.668 90.476 110.986 70.327 11
Occipital-SCS0.512 91.000 10.716 90.509 90.506 30.611 100.092 80.602 70.177 120.346 110.383 110.165 120.442 90.850 90.386 110.618 110.543 90.889 130.389 8
3D-BoNet0.488 101.000 10.672 120.590 60.301 120.484 170.098 70.620 40.306 80.341 120.259 140.125 140.434 100.796 110.402 90.499 160.513 100.909 120.439 5
Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, Niki Trigoni: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds. NeurIPS 2019 Spotlight
PanopticFusion-inst0.478 110.667 130.712 110.595 50.259 140.550 150.000 200.613 50.175 130.250 150.434 60.437 30.411 120.857 60.485 50.591 140.267 180.944 90.359 9
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
ResNet-backbone0.459 121.000 10.737 80.159 190.259 130.587 130.138 40.475 120.217 100.416 80.408 90.128 130.315 140.714 120.411 80.536 150.590 30.873 160.304 12
MASCpermissive0.447 130.528 170.555 140.381 110.382 90.633 90.002 180.509 100.260 90.361 100.432 70.327 80.451 70.571 140.367 120.639 100.386 120.980 80.276 13
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
3D-SISpermissive0.382 141.000 10.432 170.245 150.190 150.577 140.013 150.263 150.033 190.320 130.240 150.075 160.422 110.857 60.117 170.699 50.271 170.883 150.235 15
Ji Hou, Angela Dai, Matthias Niessner: 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans. CVPR 2019
UNet-backbone0.319 150.667 130.715 100.233 160.189 160.479 180.008 160.218 160.067 180.201 160.173 160.107 150.123 170.438 160.150 150.615 120.355 130.916 110.093 20
R-PointNet0.306 160.500 190.405 180.311 120.348 100.589 120.054 100.068 190.126 140.283 140.290 130.028 180.219 150.214 190.331 130.396 180.275 160.821 180.245 14
Region0.248 170.667 130.437 160.188 170.153 170.491 160.000 200.208 170.094 160.153 180.099 180.057 170.217 160.119 200.039 210.466 170.302 150.640 190.140 17
3D-BEVIS0.248 170.667 130.566 130.076 200.035 220.394 190.027 130.035 200.098 150.099 200.030 210.025 190.098 180.375 170.126 160.604 130.181 200.854 170.171 16
Cathrin Elich, Francis Engelmann, Jonas Schult, Theodora Kontogianni, Bastian Leibe: 3D-BEVIS: Birds-Eye-View Instance Segmentation.
Seg-Clusterpermissive0.215 190.370 200.337 200.285 130.105 180.325 200.025 140.282 140.085 170.105 190.107 170.007 220.079 190.317 180.114 180.309 200.304 140.587 200.123 19
PE0.188 200.528 170.046 210.017 210.062 200.600 110.000 200.003 220.030 200.177 170.032 200.013 210.000 220.487 150.090 200.093 220.261 190.916 100.029 21
Sgpn_scannet0.143 210.208 220.390 190.169 180.065 190.275 210.029 120.069 180.000 210.087 210.043 190.014 200.027 210.000 210.112 190.351 190.168 210.438 210.138 18
MaskRCNN 2d->3d Proj0.058 220.333 210.002 220.000 220.053 210.002 220.002 190.021 210.000 210.045 220.024 220.238 90.065 200.000 210.014 220.107 210.020 220.110 220.006 22