The 3D semantic instance prediction task involves detecting and segmenting the object in an 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the average precision for each class. We report the mean average precision AP at overlap 0.25 (AP 25%), overlap 0.5 (AP 50%), and over overlaps in the range [0.5:0.95:0.05] (AP). Note that multiple predictions of the same ground truth instance are penalized as false positives.



This table lists the benchmark results for the 3D semantic instance scenario.




Method Infoavg ap 50%bathtubbedbookshelfcabinetchaircountercurtaindeskdoorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwindow
sorted bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort by
OccuSeg+instance0.672 11.000 10.758 70.682 30.576 20.842 10.477 10.504 120.524 20.567 10.585 10.451 20.557 61.000 10.751 10.797 20.563 61.000 10.467 4
GICN0.638 21.000 10.895 10.800 10.480 60.676 80.144 40.737 20.354 70.447 50.400 110.365 60.700 11.000 10.569 30.836 10.599 31.000 10.473 3
PointGroup0.636 31.000 10.765 50.624 50.505 50.797 40.116 70.696 30.384 60.441 60.559 20.476 10.596 41.000 10.666 20.756 50.556 90.997 60.513 1
Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi-Wing Fu, Jiaya Jia: PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation. CVPR 2020 [oral]
3D-MPA0.611 41.000 10.833 20.765 20.526 30.756 70.136 60.588 80.470 40.438 70.432 90.358 70.650 20.857 60.429 80.765 40.557 81.000 10.430 6
Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, Matthias Nie├čner: 3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation. CVPR 2020
PCJC0.578 51.000 10.810 30.583 90.449 90.813 20.042 120.603 60.341 80.490 30.465 40.410 40.650 20.835 100.264 150.694 70.561 70.889 130.504 2
SSEN0.575 61.000 10.761 60.473 110.477 70.795 50.066 100.529 90.658 10.460 40.461 50.380 50.331 140.859 50.401 110.692 80.653 11.000 10.348 10
Dongsu Zhang, Junha Chun, Sang Kyun Cha, Young Min Kim: Spatial Semantic Embedding Network: Fast 3D Instance Segmentation with Deep Metric Learning. Arxiv
PE0.560 70.667 130.722 90.673 40.577 10.806 30.202 20.520 100.524 30.491 20.450 60.061 170.553 70.688 130.507 50.777 30.604 20.941 100.322 12
MTML0.549 81.000 10.807 40.588 80.327 120.647 90.004 180.815 10.180 120.418 80.364 130.182 100.445 91.000 10.442 70.688 90.571 51.000 10.396 7
Jean Lahoud, Bernard Ghanem, Marc Pollefeys, Martin R. Oswald: 3D Instance Segmentation via Multi-task Metric Learning. ICCV 2019 [oral]
Sparse R-CNN0.515 91.000 10.538 160.282 150.468 80.790 60.173 30.345 140.429 50.413 100.484 30.176 110.595 50.591 140.522 40.668 100.476 120.986 70.327 11
Occipital-SCS0.512 101.000 10.716 100.509 100.506 40.611 110.092 90.602 70.177 130.346 120.383 120.165 120.442 100.850 90.386 120.618 120.543 100.889 130.389 8
3D-BoNet0.488 111.000 10.672 130.590 70.301 130.484 170.098 80.620 40.306 90.341 130.259 150.125 140.434 110.796 110.402 100.499 170.513 110.909 120.439 5
Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, Niki Trigoni: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds. NeurIPS 2019 Spotlight
PanopticFusion-inst0.478 120.667 130.712 120.595 60.259 150.550 150.000 210.613 50.175 140.250 160.434 70.437 30.411 130.857 60.485 60.591 150.267 190.944 90.359 9
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
SALoss-ResNet0.459 131.000 10.737 80.159 200.259 140.587 130.138 50.475 130.217 110.416 90.408 100.128 130.315 150.714 120.411 90.536 160.590 40.873 160.304 13
Zhidong Liang, Ming Yang, Hao Li, Chunxiang Wang: 3D Instance Embedding Learning With a Structure-Aware Loss Function for Point Cloud Segmentation. IEEE Robotics and Automation Letters (IROS2020)
MASCpermissive0.447 140.528 180.555 150.381 120.382 100.633 100.002 190.509 110.260 100.361 110.432 80.327 80.451 80.571 150.367 130.639 110.386 130.980 80.276 14
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
3D-SISpermissive0.382 151.000 10.432 180.245 160.190 160.577 140.013 160.263 160.033 200.320 140.240 160.075 160.422 120.857 60.117 180.699 60.271 180.883 150.235 16
Ji Hou, Angela Dai, Matthias Niessner: 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans. CVPR 2019
UNet-backbone0.319 160.667 130.715 110.233 170.189 170.479 180.008 170.218 170.067 190.201 170.173 170.107 150.123 180.438 160.150 160.615 130.355 140.916 110.093 21
R-PointNet0.306 170.500 190.405 190.311 130.348 110.589 120.054 110.068 200.126 150.283 150.290 140.028 190.219 160.214 190.331 140.396 190.275 170.821 180.245 15
Region0.248 180.667 130.437 170.188 180.153 180.491 160.000 210.208 180.094 170.153 180.099 190.057 180.217 170.119 200.039 210.466 180.302 160.640 190.140 18
3D-BEVIS0.248 180.667 130.566 140.076 210.035 220.394 190.027 140.035 210.098 160.099 200.030 210.025 200.098 190.375 170.126 170.604 140.181 200.854 170.171 17
Cathrin Elich, Francis Engelmann, Jonas Schult, Theodora Kontogianni, Bastian Leibe: 3D-BEVIS: Birds-Eye-View Instance Segmentation.
Seg-Clusterpermissive0.215 200.370 200.337 210.285 140.105 190.325 200.025 150.282 150.085 180.105 190.107 180.007 220.079 200.317 180.114 190.309 210.304 150.587 200.123 20
Sgpn_scannet0.143 210.208 220.390 200.169 190.065 200.275 210.029 130.069 190.000 210.087 210.043 200.014 210.027 220.000 210.112 200.351 200.168 210.438 210.138 19
MaskRCNN 2d->3d Proj0.058 220.333 210.002 220.000 220.053 210.002 220.002 200.021 220.000 210.045 220.024 220.238 90.065 210.000 210.014 220.107 220.020 220.110 220.006 22