3D Semantic Instance Results - ScanNet Benchmark

The 3D semantic instance prediction task involves detecting and segmenting the object in an 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the average precision for each class. We report the mean average precision AP at overlap 0.25 (AP 25%), overlap 0.5 (AP 50%), and over overlaps in the range [0.5:0.95:0.05] (AP). Note that multiple predictions of the same ground truth instance are penalized as false positives.

This table lists the benchmark results for the 3D semantic instance scenario.

Method	avg ap 50%	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	window

Occipital-SCS	0.512 1	1.000 1	0.716 2	0.509 3	0.506 1	0.611 3	0.092 3	0.602 3	0.177 5	0.346 4	0.383 4	0.165 4	0.442 2	0.850 4	0.386 4	0.618 3	0.543 2	0.889 6	0.389 2

3D-BoNet	0.488 2	1.000 1	0.672 5	0.590 2	0.301 4	0.484 8	0.098 2	0.620 1	0.306 1	0.341 5	0.259 7	0.125 6	0.434 3	0.796 5	0.402 3	0.499 9	0.513 3	0.909 5	0.439 1

MTML	0.481 3	1.000 1	0.666 6	0.377 5	0.272 5	0.709 1	0.001 12	0.579 4	0.254 3	0.361 3	0.318 5	0.095 8	0.432 4	1.000 1	0.184 7	0.601 6	0.487 4	0.938 3	0.384 3

PanopticFusion-inst	0.478 4	0.667 6	0.712 4	0.595 1	0.259 7	0.550 7	0.000 13	0.613 2	0.175 6	0.250 8	0.434 1	0.437 1	0.411 6	0.857 2	0.485 1	0.591 7	0.267 10	0.944 2	0.359 4
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. arXiv
ResNet-backbone	0.459 5	1.000 1	0.737 1	0.159 11	0.259 6	0.587 5	0.138 1	0.475 6	0.217 4	0.416 1	0.408 3	0.128 5	0.315 7	0.714 6	0.411 2	0.536 8	0.590 1	0.873 8	0.304 5

MASC	0.447 6	0.528 9	0.555 8	0.381 4	0.382 2	0.633 2	0.002 10	0.509 5	0.260 2	0.361 2	0.432 2	0.327 2	0.451 1	0.571 7	0.367 5	0.639 2	0.386 5	0.980 1	0.276 6
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
3D-SIS	0.382 7	1.000 1	0.432 9	0.245 8	0.190 8	0.577 6	0.013 8	0.263 8	0.033 11	0.320 6	0.240 8	0.075 9	0.422 5	0.857 2	0.117 10	0.699 1	0.271 9	0.883 7	0.235 8
Ji Hou, Angela Dai, Matthias Niessner: 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans. CVPR 2019
UNet-backbone	0.319 8	0.667 6	0.715 3	0.233 9	0.189 9	0.479 9	0.008 9	0.218 9	0.067 10	0.201 9	0.173 9	0.107 7	0.123 9	0.438 8	0.150 8	0.615 4	0.355 6	0.916 4	0.093 12

R-PointNet	0.306 9	0.500 10	0.405 10	0.311 6	0.348 3	0.589 4	0.054 4	0.068 11	0.126 7	0.283 7	0.290 6	0.028 10	0.219 8	0.214 11	0.331 6	0.396 10	0.275 8	0.821 10	0.245 7

3D-BEVIS	0.248 10	0.667 6	0.566 7	0.076 12	0.035 13	0.394 10	0.027 6	0.035 12	0.098 8	0.099 11	0.030 12	0.025 11	0.098 10	0.375 9	0.126 9	0.604 5	0.181 11	0.854 9	0.171 9
Cathrin Elich, Francis Engelmann, Jonas Schult, Theodora Kontogianni, Bastian Leibe: 3D-BEVIS: Birds-Eye-View Instance Segmentation.
Seg-Cluster	0.215 11	0.370 11	0.337 12	0.285 7	0.105 10	0.325 11	0.025 7	0.282 7	0.085 9	0.105 10	0.107 10	0.007 13	0.079 11	0.317 10	0.114 11	0.309 12	0.304 7	0.587 11	0.123 11

Sgpn_scannet	0.143 12	0.208 13	0.390 11	0.169 10	0.065 11	0.275 12	0.029 5	0.069 10	0.000 12	0.087 12	0.043 11	0.014 12	0.027 13	0.000 12	0.112 12	0.351 11	0.168 12	0.438 12	0.138 10

MaskRCNN 2d->3d Proj	0.058 13	0.333 12	0.002 13	0.000 13	0.053 12	0.002 13	0.002 11	0.021 13	0.000 12	0.045 13	0.024 13	0.238 3	0.065 12	0.000 12	0.014 13	0.107 13	0.020 13	0.110 13	0.006 13

3D Semantic instance benchmark