3D Semantic Instance Results - ScanNet Benchmark

The 3D semantic instance prediction task involves detecting and segmenting the object in an 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the average precision for each class. We report the mean average precision AP at overlap 0.25 (AP 25%), overlap 0.5 (AP 50%), and over overlaps in the range [0.5:0.95:0.05] (AP). Note that multiple predictions of the same ground truth instance are penalized as false positives.

This table lists the benchmark results for the 3D semantic instance scenario.

Method	avg ap 50%	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	window

GICN	0.638 1	1.000 1	0.895 2	0.800 1	0.480 4	0.676 5	0.144 2	0.737 2	0.354 3	0.447 2	0.400 8	0.365 4	0.700 1	1.000 1	0.569 3	0.836 1	0.599 2	1.000 1	0.473 3

PointGroup	0.636 2	1.000 1	0.765 5	0.624 4	0.505 3	0.797 2	0.116 6	0.696 4	0.384 2	0.441 3	0.559 1	0.476 1	0.596 5	1.000 1	0.666 1	0.756 4	0.556 6	0.997 5	0.513 2

OccuSeg	0.634 3	1.000 1	0.902 1	0.771 2	0.461 5	0.814 1	0.282 1	0.583 9	0.328 5	0.472 1	0.471 2	0.295 7	0.600 3	1.000 1	0.650 2	0.664 7	0.587 4	1.000 1	0.537 1

MPA	0.610 4	1.000 1	0.833 3	0.765 3	0.546 1	0.750 4	0.140 4	0.588 8	0.478 1	0.433 5	0.454 3	0.376 3	0.650 2	0.857 5	0.429 6	0.765 3	0.537 8	1.000 1	0.378 7

SSEN	0.568 5	1.000 1	0.747 6	0.449 9	0.371 7	0.760 3	0.143 3	0.706 3	0.336 4	0.439 4	0.430 6	0.306 6	0.600 3	0.857 5	0.407 8	0.831 2	0.611 1	0.944 7	0.283 10

MTML	0.549 6	1.000 1	0.807 4	0.588 7	0.327 9	0.647 6	0.004 15	0.815 1	0.180 9	0.418 6	0.364 10	0.182 9	0.445 7	1.000 1	0.442 5	0.688 6	0.571 5	1.000 1	0.396 5
Jean Lahoud, Bernard Ghanem, Marc Pollefeys, Martin R. Oswald: 3D Instance Segmentation via Multi-task Metric Learning. ICCV 2019 [oral]
Occipital-SCS	0.512 7	1.000 1	0.716 8	0.509 8	0.506 2	0.611 8	0.092 8	0.602 7	0.177 10	0.346 9	0.383 9	0.165 10	0.442 8	0.850 9	0.386 10	0.618 9	0.543 7	0.889 11	0.389 6

3D-BoNet	0.488 8	1.000 1	0.672 11	0.590 6	0.301 10	0.484 13	0.098 7	0.620 5	0.306 6	0.341 10	0.259 12	0.125 12	0.434 9	0.796 10	0.402 9	0.499 14	0.513 9	0.909 10	0.439 4
Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, Niki Trigoni: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds. NeurIPS 2019 Spotlight
PanopticFusion-inst	0.478 9	0.667 11	0.712 10	0.595 5	0.259 12	0.550 12	0.000 18	0.613 6	0.175 11	0.250 13	0.434 4	0.437 2	0.411 11	0.857 5	0.485 4	0.591 12	0.267 15	0.944 7	0.359 8
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
ResNet-backbone	0.459 10	1.000 1	0.737 7	0.159 16	0.259 11	0.587 10	0.138 5	0.475 11	0.217 8	0.416 7	0.408 7	0.128 11	0.315 12	0.714 11	0.411 7	0.536 13	0.590 3	0.873 13	0.304 9

MASC	0.447 11	0.528 14	0.555 13	0.381 10	0.382 6	0.633 7	0.002 16	0.509 10	0.260 7	0.361 8	0.432 5	0.327 5	0.451 6	0.571 12	0.367 11	0.639 8	0.386 10	0.980 6	0.276 11
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
3D-SIS	0.382 12	1.000 1	0.432 14	0.245 13	0.190 13	0.577 11	0.013 13	0.263 13	0.033 16	0.320 11	0.240 13	0.075 14	0.422 10	0.857 5	0.117 15	0.699 5	0.271 14	0.883 12	0.235 13
Ji Hou, Angela Dai, Matthias Niessner: 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans. CVPR 2019
UNet-backbone	0.319 13	0.667 11	0.715 9	0.233 14	0.189 14	0.479 14	0.008 14	0.218 14	0.067 15	0.201 14	0.173 14	0.107 13	0.123 14	0.438 13	0.150 13	0.615 10	0.355 11	0.916 9	0.093 17

R-PointNet	0.306 14	0.500 15	0.405 15	0.311 11	0.348 8	0.589 9	0.054 9	0.068 16	0.126 12	0.283 12	0.290 11	0.028 15	0.219 13	0.214 16	0.331 12	0.396 15	0.275 13	0.821 15	0.245 12

3D-BEVIS	0.248 15	0.667 11	0.566 12	0.076 17	0.035 18	0.394 15	0.027 11	0.035 17	0.098 13	0.099 16	0.030 17	0.025 16	0.098 15	0.375 14	0.126 14	0.604 11	0.181 16	0.854 14	0.171 14
Cathrin Elich, Francis Engelmann, Jonas Schult, Theodora Kontogianni, Bastian Leibe: 3D-BEVIS: Birds-Eye-View Instance Segmentation.
Seg-Cluster	0.215 16	0.370 16	0.337 17	0.285 12	0.105 15	0.325 16	0.025 12	0.282 12	0.085 14	0.105 15	0.107 15	0.007 18	0.079 16	0.317 15	0.114 16	0.309 17	0.304 12	0.587 16	0.123 16

Sgpn_scannet	0.143 17	0.208 18	0.390 16	0.169 15	0.065 16	0.275 17	0.029 10	0.069 15	0.000 17	0.087 17	0.043 16	0.014 17	0.027 18	0.000 17	0.112 17	0.351 16	0.168 17	0.438 17	0.138 15

MaskRCNN 2d->3d Proj	0.058 18	0.333 17	0.002 18	0.000 18	0.053 17	0.002 18	0.002 17	0.021 18	0.000 17	0.045 18	0.024 18	0.238 8	0.065 17	0.000 17	0.014 18	0.107 18	0.020 18	0.110 18	0.006 18

3D Semantic instance benchmark