3D Semantic Instance Results - ScanNet Benchmark

The 3D semantic instance prediction task involves detecting and segmenting the object in an 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the average precision for each class. We report the mean average precision AP at overlap 0.25 (AP 25%), overlap 0.5 (AP 50%), and over overlaps in the range [0.5:0.95:0.05] (AP). Note that multiple predictions of the same ground truth instance are penalized as false positives.

This table lists the benchmark results for the 3D semantic instance scenario.

Method	avg ap 50%	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	window

HAIS	0.699 1	1.000 1	0.849 2	0.820 3	0.675 1	0.808 7	0.279 3	0.757 5	0.465 6	0.517 3	0.596 1	0.559 3	0.600 7	1.000 1	0.654 5	0.767 4	0.676 2	0.994 13	0.560 3

SSTNet	0.698 2	1.000 1	0.697 19	0.888 2	0.556 4	0.803 8	0.387 2	0.626 9	0.417 9	0.556 2	0.585 3	0.702 1	0.600 7	1.000 1	0.824 1	0.720 11	0.692 1	1.000 1	0.509 5

OccuSeg+instance	0.672 3	1.000 1	0.758 14	0.682 10	0.576 3	0.842 1	0.477 1	0.504 19	0.524 3	0.567 1	0.585 4	0.451 7	0.557 13	1.000 1	0.751 2	0.797 3	0.563 12	1.000 1	0.467 10

Mask-Group	0.664 4	1.000 1	0.822 5	0.764 8	0.616 2	0.815 4	0.139 7	0.694 8	0.597 2	0.459 6	0.566 5	0.599 2	0.600 7	0.516 23	0.715 3	0.819 2	0.635 6	1.000 1	0.603 1

CSC-Pretrained	0.648 5	1.000 1	0.810 6	0.768 6	0.523 9	0.813 5	0.143 6	0.819 1	0.389 10	0.422 13	0.511 10	0.443 8	0.650 2	1.000 1	0.624 6	0.732 10	0.634 7	1.000 1	0.375 16

PE	0.645 6	1.000 1	0.773 10	0.798 5	0.538 6	0.786 13	0.088 14	0.799 3	0.350 14	0.435 12	0.547 7	0.545 4	0.646 6	0.933 10	0.562 9	0.761 7	0.556 16	0.997 10	0.501 7
Biao Zhang, Peter Wonka: Point Cloud Instance Segmentation using Probabilistic Embeddings. CVPR 2021
RPGN	0.643 7	1.000 1	0.758 13	0.582 16	0.539 5	0.826 3	0.046 17	0.765 4	0.372 12	0.436 11	0.588 2	0.539 5	0.650 2	1.000 1	0.577 7	0.750 9	0.653 5	0.997 10	0.495 8

Dyco3D	0.641 8	1.000 1	0.841 3	0.893 1	0.531 7	0.802 9	0.115 11	0.588 14	0.448 7	0.438 9	0.537 8	0.430 10	0.550 14	0.857 12	0.534 10	0.764 6	0.657 3	0.987 14	0.568 2

GICN	0.638 9	1.000 1	0.895 1	0.800 4	0.480 12	0.676 15	0.144 5	0.737 6	0.354 13	0.447 7	0.400 18	0.365 13	0.700 1	1.000 1	0.569 8	0.836 1	0.599 9	1.000 1	0.473 9

PointGroup	0.636 10	1.000 1	0.765 11	0.624 11	0.505 11	0.797 10	0.116 10	0.696 7	0.384 11	0.441 8	0.559 6	0.476 6	0.596 10	1.000 1	0.666 4	0.756 8	0.556 15	0.997 10	0.513 4
Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi-Wing Fu, Jiaya Jia: PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation. CVPR 2020 [oral]
3D-MPA	0.611 11	1.000 1	0.833 4	0.765 7	0.526 8	0.756 14	0.136 9	0.588 14	0.470 5	0.438 10	0.432 16	0.358 14	0.650 2	0.857 12	0.429 14	0.765 5	0.557 14	1.000 1	0.430 13
Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, Matthias Nießner: 3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation. CVPR 2020
SphereNet	0.606 12	1.000 1	0.776 9	0.745 9	0.436 16	0.834 2	0.035 19	0.587 16	0.518 4	0.338 20	0.534 9	0.352 15	0.594 12	1.000 1	0.391 18	0.696 14	0.624 8	1.000 1	0.451 11

PCJC	0.578 13	1.000 1	0.810 7	0.583 15	0.449 15	0.813 6	0.042 18	0.603 12	0.341 15	0.490 4	0.465 12	0.410 11	0.650 2	0.835 17	0.264 22	0.694 15	0.561 13	0.889 20	0.504 6

SSEN	0.575 14	1.000 1	0.761 12	0.473 18	0.477 13	0.795 11	0.066 15	0.529 17	0.658 1	0.460 5	0.461 13	0.380 12	0.331 22	0.859 11	0.401 17	0.692 16	0.653 4	1.000 1	0.348 18
Dongsu Zhang, Junha Chun, Sang Kyun Cha, Young Min Kim: Spatial Semantic Embedding Network: Fast 3D Instance Segmentation with Deep Metric Learning. Arxiv
MTML	0.549 15	1.000 1	0.807 8	0.588 14	0.327 19	0.647 16	0.004 24	0.815 2	0.180 19	0.418 14	0.364 20	0.182 18	0.445 17	1.000 1	0.442 13	0.688 17	0.571 11	1.000 1	0.396 14
Jean Lahoud, Bernard Ghanem, Marc Pollefeys, Martin R. Oswald: 3D Instance Segmentation via Multi-task Metric Learning. ICCV 2019 [oral]
Sparse R-CNN	0.515 16	1.000 1	0.538 24	0.282 21	0.468 14	0.790 12	0.173 4	0.345 21	0.429 8	0.413 16	0.484 11	0.176 19	0.595 11	0.591 21	0.522 11	0.668 18	0.476 19	0.986 15	0.327 19

Occipital-SCS	0.512 17	1.000 1	0.716 16	0.509 17	0.506 10	0.611 18	0.092 13	0.602 13	0.177 20	0.346 18	0.383 19	0.165 20	0.442 18	0.850 16	0.386 19	0.618 20	0.543 17	0.889 20	0.389 15

3D-BoNet	0.488 18	1.000 1	0.672 20	0.590 13	0.301 20	0.484 25	0.098 12	0.620 10	0.306 16	0.341 19	0.259 22	0.125 22	0.434 19	0.796 18	0.402 16	0.499 25	0.513 18	0.909 19	0.439 12
Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, Niki Trigoni: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds. NeurIPS 2019 Spotlight
PanopticFusion-inst	0.478 19	0.667 21	0.712 18	0.595 12	0.259 22	0.550 23	0.000 27	0.613 11	0.175 21	0.250 24	0.434 14	0.437 9	0.411 21	0.857 12	0.485 12	0.591 23	0.267 26	0.944 17	0.359 17
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
SALoss-ResNet	0.459 20	1.000 1	0.737 15	0.159 27	0.259 21	0.587 20	0.138 8	0.475 20	0.217 18	0.416 15	0.408 17	0.128 21	0.315 23	0.714 19	0.411 15	0.536 24	0.590 10	0.873 23	0.304 20
Zhidong Liang, Ming Yang, Hao Li, Chunxiang Wang: 3D Instance Embedding Learning With a Structure-Aware Loss Function for Point Cloud Segmentation. IEEE Robotics and Automation Letters (IROS2020)
MASC	0.447 21	0.528 26	0.555 22	0.381 19	0.382 17	0.633 17	0.002 25	0.509 18	0.260 17	0.361 17	0.432 15	0.327 16	0.451 16	0.571 22	0.367 20	0.639 19	0.386 20	0.980 16	0.276 21
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
3D-SIS	0.382 22	1.000 1	0.432 26	0.245 23	0.190 23	0.577 21	0.013 22	0.263 22	0.033 26	0.320 21	0.240 23	0.075 24	0.422 20	0.857 12	0.117 25	0.699 12	0.271 25	0.883 22	0.235 23
Ji Hou, Angela Dai, Matthias Niessner: 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans. CVPR 2019
Hier3D	0.323 23	0.667 21	0.542 23	0.264 22	0.157 25	0.550 22	0.000 27	0.205 25	0.009 27	0.270 23	0.218 24	0.075 24	0.500 15	0.688 20	0.007 29	0.698 13	0.301 23	0.459 27	0.200 24
Tan: HCFS3D: Hierarchical Coupled Feature Selection Network for 3D Semantic and Instance Segmentation.
UNet-backbone	0.319 24	0.667 21	0.715 17	0.233 24	0.189 24	0.479 26	0.008 23	0.218 23	0.067 25	0.201 25	0.173 25	0.107 23	0.123 26	0.438 24	0.150 23	0.615 21	0.355 21	0.916 18	0.093 28

R-PointNet	0.306 25	0.500 27	0.405 27	0.311 20	0.348 18	0.589 19	0.054 16	0.068 27	0.126 22	0.283 22	0.290 21	0.028 27	0.219 24	0.214 26	0.331 21	0.396 27	0.275 24	0.821 25	0.245 22

3D-BEVIS	0.248 26	0.667 21	0.566 21	0.076 28	0.035 29	0.394 27	0.027 21	0.035 28	0.098 23	0.099 27	0.030 28	0.025 28	0.098 27	0.375 25	0.126 24	0.604 22	0.181 27	0.854 24	0.171 25
Cathrin Elich, Francis Engelmann, Jonas Schult, Theodora Kontogianni, Bastian Leibe: 3D-BEVIS: Birds-Eye-View Instance Segmentation.
Region	0.248 26	0.667 21	0.437 25	0.188 25	0.153 26	0.491 24	0.000 27	0.208 24	0.094 24	0.153 26	0.099 26	0.057 26	0.217 25	0.119 27	0.039 27	0.466 26	0.302 22	0.640 26	0.140 26

Sgpn_scannet	0.143 28	0.208 29	0.390 28	0.169 26	0.065 27	0.275 28	0.029 20	0.069 26	0.000 28	0.087 28	0.043 27	0.014 29	0.027 29	0.000 28	0.112 26	0.351 28	0.168 28	0.438 28	0.138 27

MaskRCNN 2d->3d Proj	0.058 29	0.333 28	0.002 29	0.000 29	0.053 28	0.002 29	0.002 26	0.021 29	0.000 28	0.045 29	0.024 29	0.238 17	0.065 28	0.000 28	0.014 28	0.107 29	0.020 29	0.110 29	0.006 29

3D Semantic instance benchmark