3D Semantic Instance Results - ScanNet Benchmark

The 3D semantic instance prediction task involves detecting and segmenting the object in an 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the average precision for each class. We report the mean average precision AP at overlap 0.25 (AP 25%), overlap 0.5 (AP 50%), and over overlaps in the range [0.5:0.95:0.05] (AP). Note that multiple predictions of the same ground truth instance are penalized as false positives.

This table lists the benchmark results for the 3D semantic instance scenario.

Method	avg ap 50%	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	window

SPFormer	0.770 1	0.903 32	0.903 1	0.806 8	0.609 11	0.886 2	0.568 1	0.815 6	0.705 3	0.711 1	0.655 4	0.652 7	0.685 8	1.000 1	0.789 3	0.809 9	0.776 3	1.000 1	0.583 6

SoftGroup++	0.769 2	1.000 1	0.803 17	0.937 1	0.684 3	0.865 4	0.213 15	0.870 2	0.664 4	0.571 6	0.758 1	0.702 4	0.807 2	1.000 1	0.653 14	0.902 1	0.792 2	1.000 1	0.626 1

Mask3D	0.765 3	1.000 1	0.893 3	0.721 20	0.682 4	0.855 8	0.449 3	0.710 17	0.775 1	0.601 4	0.707 3	0.646 8	0.811 1	1.000 1	0.817 2	0.807 10	0.725 8	1.000 1	0.578 7

SoftGroup	0.761 4	1.000 1	0.808 14	0.845 6	0.716 1	0.862 6	0.243 12	0.824 3	0.655 6	0.620 3	0.734 2	0.699 5	0.791 4	0.981 21	0.716 6	0.844 4	0.769 4	1.000 1	0.594 5
Thang Vu, Kookhoi Kim, Tung M. Luu, Xuan Thanh Nguyen, Chang D. Yoo: SoftGroup for 3D Instance Segmentaiton on Point Clouds. CVPR 2022 [Oral]
GraphCut	0.732 5	1.000 1	0.788 20	0.724 19	0.642 7	0.859 7	0.248 11	0.787 10	0.618 10	0.596 5	0.653 6	0.722 2	0.583 25	1.000 1	0.766 4	0.861 2	0.825 1	1.000 1	0.504 17

IPCA-Inst	0.731 6	1.000 1	0.788 21	0.884 5	0.698 2	0.788 22	0.252 10	0.760 12	0.646 7	0.511 14	0.637 8	0.665 6	0.804 3	1.000 1	0.644 15	0.778 12	0.747 6	1.000 1	0.561 12

TopoSeg	0.725 7	1.000 1	0.806 16	0.933 2	0.668 6	0.758 25	0.272 8	0.734 16	0.630 8	0.549 10	0.654 5	0.606 9	0.697 7	0.966 23	0.612 18	0.839 5	0.754 5	1.000 1	0.573 8

DKNet	0.718 8	1.000 1	0.814 11	0.782 11	0.619 8	0.872 3	0.224 13	0.751 14	0.569 12	0.677 2	0.585 11	0.724 1	0.633 17	0.981 21	0.515 25	0.819 7	0.736 7	1.000 1	0.617 2
Yizheng Wu, Min Shi, Shuaiyuan Du, Hao Lu, Zhiguo Cao, Weicai Zhong: 3D Instances as 1D Kernels. ECCV 2022
SSEC	0.700 9	1.000 1	0.848 6	0.763 17	0.609 12	0.792 20	0.262 9	0.824 3	0.627 9	0.535 12	0.547 19	0.493 15	0.600 19	1.000 1	0.712 8	0.731 24	0.689 12	1.000 1	0.563 11

HAIS	0.699 10	1.000 1	0.849 5	0.820 7	0.675 5	0.808 15	0.279 6	0.757 13	0.465 17	0.517 13	0.596 9	0.559 11	0.600 19	1.000 1	0.654 13	0.767 14	0.676 13	0.994 28	0.560 13
Shaoyu Chen, Jiemin Fang, Qian Zhang, Wenyu Liu, Xinggang Wang: Hierarchical Aggregation for 3D Instance Segmentation. ICCV 2021
SSTNet	0.698 11	1.000 1	0.697 36	0.888 4	0.556 18	0.803 16	0.387 4	0.626 24	0.417 21	0.556 9	0.585 12	0.702 3	0.600 19	1.000 1	0.824 1	0.720 26	0.692 10	1.000 1	0.509 16
Zhihao Liang, Zhihao Li, Songcen Xu, Mingkui Tan, Kui Jia: Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks. ICCV2021
SphereSeg	0.680 12	1.000 1	0.856 4	0.744 18	0.618 9	0.893 1	0.151 17	0.651 22	0.713 2	0.537 11	0.579 14	0.430 23	0.651 9	1.000 1	0.389 34	0.744 21	0.697 9	0.991 29	0.601 4

Box2Mask	0.677 13	1.000 1	0.847 7	0.771 13	0.509 24	0.816 11	0.277 7	0.558 31	0.482 14	0.562 8	0.640 7	0.448 19	0.700 5	1.000 1	0.666 9	0.852 3	0.578 24	0.997 23	0.488 21
Julian Chibane, Francis Engelmann, Tuan Anh Tran, Gerard Pons-Moll: Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation Using Bounding Boxes. ECCV 2022
OccuSeg+instance	0.672 14	1.000 1	0.758 28	0.682 22	0.576 16	0.842 9	0.477 2	0.504 34	0.524 13	0.567 7	0.585 13	0.451 18	0.557 26	1.000 1	0.751 5	0.797 11	0.563 27	1.000 1	0.467 24
Lei Han, Tian Zheng, Lan Xu, Lu Fang: OccuSeg: Occupancy-aware 3D Instance Segmentation. CVPR2020
Mask-Group	0.664 15	1.000 1	0.822 10	0.764 16	0.616 10	0.815 12	0.139 21	0.694 19	0.597 11	0.459 19	0.566 15	0.599 10	0.600 19	0.516 40	0.715 7	0.819 8	0.635 17	1.000 1	0.603 3
Min Zhong, Xinghao Chen, Xiaokang Chen, Gang Zeng, Yunhe Wang: MaskGroup: Hierarchical Point Grouping and Masking for 3D Instance Segmentation. ICME 2022
INS-Conv-instance	0.657 16	1.000 1	0.760 26	0.667 24	0.581 14	0.863 5	0.323 5	0.655 21	0.477 15	0.473 17	0.549 17	0.432 22	0.650 10	1.000 1	0.655 12	0.738 22	0.585 23	0.944 33	0.472 23

CSC-Pretrained	0.648 17	1.000 1	0.810 12	0.768 14	0.523 23	0.813 13	0.143 20	0.819 5	0.389 22	0.422 26	0.511 22	0.443 20	0.650 10	1.000 1	0.624 17	0.732 23	0.634 18	1.000 1	0.375 30

PE	0.645 18	1.000 1	0.773 23	0.798 10	0.538 20	0.786 23	0.088 28	0.799 9	0.350 26	0.435 25	0.547 18	0.545 12	0.646 16	0.933 24	0.562 21	0.761 17	0.556 32	0.997 23	0.501 19
Biao Zhang, Peter Wonka: Point Cloud Instance Segmentation using Probabilistic Embeddings. CVPR 2021
RPGN	0.643 19	1.000 1	0.758 27	0.582 32	0.539 19	0.826 10	0.046 32	0.765 11	0.372 24	0.436 24	0.588 10	0.539 14	0.650 10	1.000 1	0.577 19	0.750 19	0.653 16	0.997 23	0.495 20

Dyco3D	0.641 20	1.000 1	0.841 8	0.893 3	0.531 21	0.802 17	0.115 25	0.588 29	0.448 18	0.438 22	0.537 21	0.430 24	0.550 27	0.857 26	0.534 23	0.764 16	0.657 14	0.987 30	0.568 9
Tong He; Chunhua Shen; Anton van den Hengel: DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolution. CVPR2021
GICN	0.638 21	1.000 1	0.895 2	0.800 9	0.480 27	0.676 29	0.144 19	0.737 15	0.354 25	0.447 20	0.400 31	0.365 29	0.700 5	1.000 1	0.569 20	0.836 6	0.599 20	1.000 1	0.473 22

PointGroup	0.636 22	1.000 1	0.765 24	0.624 26	0.505 26	0.797 18	0.116 24	0.696 18	0.384 23	0.441 21	0.559 16	0.476 16	0.596 23	1.000 1	0.666 9	0.756 18	0.556 31	0.997 23	0.513 15
Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi-Wing Fu, Jiaya Jia: PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation. CVPR 2020 [oral]
DD-UNet+Group	0.635 23	0.667 33	0.797 19	0.714 21	0.562 17	0.774 24	0.146 18	0.810 8	0.429 20	0.476 16	0.546 20	0.399 26	0.633 17	1.000 1	0.632 16	0.722 25	0.609 19	1.000 1	0.514 14
H. Liu, R. Liu, K. Yang, J. Zhang, K. Peng, R. Stiefelhagen: HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensor. ICCVW 2021
DENet	0.629 24	1.000 1	0.797 18	0.608 27	0.589 13	0.627 33	0.219 14	0.882 1	0.310 28	0.402 30	0.383 33	0.396 27	0.650 10	1.000 1	0.663 11	0.543 41	0.691 11	1.000 1	0.568 10

3D-MPA	0.611 25	1.000 1	0.833 9	0.765 15	0.526 22	0.756 26	0.136 23	0.588 29	0.470 16	0.438 23	0.432 29	0.358 30	0.650 10	0.857 26	0.429 30	0.765 15	0.557 30	1.000 1	0.430 26
Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, Matthias Nießner: 3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation. CVPR 2020
PCJC	0.578 26	1.000 1	0.810 13	0.583 31	0.449 30	0.813 14	0.042 33	0.603 27	0.341 27	0.490 15	0.465 25	0.410 25	0.650 10	0.835 32	0.264 39	0.694 30	0.561 28	0.889 37	0.504 18

SSEN	0.575 27	1.000 1	0.761 25	0.473 34	0.477 28	0.795 19	0.066 29	0.529 32	0.658 5	0.460 18	0.461 26	0.380 28	0.331 39	0.859 25	0.401 33	0.692 31	0.653 15	1.000 1	0.348 32
Dongsu Zhang, Junha Chun, Sang Kyun Cha, Young Min Kim: Spatial Semantic Embedding Network: Fast 3D Instance Segmentation with Deep Metric Learning. Arxiv
RWSeg	0.567 28	0.528 42	0.708 35	0.626 25	0.580 15	0.745 27	0.063 30	0.627 23	0.240 32	0.400 31	0.497 23	0.464 17	0.515 28	1.000 1	0.475 27	0.745 20	0.571 25	1.000 1	0.429 27

MTML	0.549 29	1.000 1	0.807 15	0.588 30	0.327 35	0.647 31	0.004 38	0.815 7	0.180 34	0.418 27	0.364 35	0.182 34	0.445 32	1.000 1	0.442 29	0.688 32	0.571 26	1.000 1	0.396 28
Jean Lahoud, Bernard Ghanem, Marc Pollefeys, Martin R. Oswald: 3D Instance Segmentation via Multi-task Metric Learning. ICCV 2019 [oral]
One_Thing_One_Click	0.529 30	0.667 33	0.718 31	0.777 12	0.399 31	0.683 28	0.000 41	0.669 20	0.138 37	0.391 32	0.374 34	0.539 13	0.360 38	0.641 37	0.556 22	0.774 13	0.593 21	0.997 23	0.251 37
Zhengzhe Liu, Xiaojuan Qi, Chi-Wing Fu: One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation. CVPR 2021
Sparse R-CNN	0.515 31	1.000 1	0.538 43	0.282 37	0.468 29	0.790 21	0.173 16	0.345 39	0.429 19	0.413 29	0.484 24	0.176 35	0.595 24	0.591 38	0.522 24	0.668 33	0.476 36	0.986 31	0.327 33

Occipital-SCS	0.512 32	1.000 1	0.716 32	0.509 33	0.506 25	0.611 34	0.092 27	0.602 28	0.177 35	0.346 35	0.383 32	0.165 36	0.442 33	0.850 31	0.386 35	0.618 37	0.543 33	0.889 37	0.389 29

3D-BoNet	0.488 33	1.000 1	0.672 38	0.590 29	0.301 37	0.484 44	0.098 26	0.620 25	0.306 29	0.341 36	0.259 39	0.125 38	0.434 35	0.796 33	0.402 32	0.499 43	0.513 35	0.909 36	0.439 25
Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, Niki Trigoni: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds. NeurIPS 2019 Spotlight
PanopticFusion-inst	0.478 34	0.667 33	0.712 34	0.595 28	0.259 40	0.550 40	0.000 41	0.613 26	0.175 36	0.250 41	0.434 27	0.437 21	0.411 37	0.857 26	0.485 26	0.591 40	0.267 46	0.944 33	0.359 31
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
SPG_WSIS	0.470 35	0.667 33	0.685 37	0.677 23	0.372 33	0.562 38	0.000 41	0.482 35	0.244 31	0.316 38	0.298 36	0.052 44	0.442 34	0.857 26	0.267 38	0.702 27	0.559 29	1.000 1	0.287 35

SALoss-ResNet	0.459 36	1.000 1	0.737 30	0.159 47	0.259 39	0.587 36	0.138 22	0.475 36	0.217 33	0.416 28	0.408 30	0.128 37	0.315 40	0.714 34	0.411 31	0.536 42	0.590 22	0.873 40	0.304 34
Zhidong Liang, Ming Yang, Hao Li, Chunxiang Wang: 3D Instance Embedding Learning With a Structure-Aware Loss Function for Point Cloud Segmentation. IEEE Robotics and Automation Letters (IROS2020)
MASC	0.447 37	0.528 42	0.555 41	0.381 35	0.382 32	0.633 32	0.002 39	0.509 33	0.260 30	0.361 34	0.432 28	0.327 31	0.451 31	0.571 39	0.367 36	0.639 35	0.386 37	0.980 32	0.276 36
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
SegGroup_ins	0.445 38	0.667 33	0.773 22	0.185 44	0.317 36	0.656 30	0.000 41	0.407 38	0.134 38	0.381 33	0.267 38	0.217 33	0.476 30	0.714 34	0.452 28	0.629 36	0.514 34	1.000 1	0.222 40
An Tao, Yueqi Duan, Yi Wei, Jiwen Lu, Jie Zhou: SegGroup: Seg-Level Supervision for 3D Instance and Semantic Segmentation. TIP 2022
3D-SIS	0.382 39	1.000 1	0.432 45	0.245 39	0.190 41	0.577 37	0.013 36	0.263 41	0.033 44	0.320 37	0.240 40	0.075 40	0.422 36	0.857 26	0.117 43	0.699 28	0.271 45	0.883 39	0.235 39
Ji Hou, Angela Dai, Matthias Niessner: 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans. CVPR 2019
Hier3D	0.323 40	0.667 33	0.542 42	0.264 38	0.157 44	0.550 39	0.000 41	0.205 44	0.009 45	0.270 40	0.218 41	0.075 40	0.500 29	0.688 36	0.007 49	0.698 29	0.301 42	0.459 46	0.200 41
Tan: HCFS3D: Hierarchical Coupled Feature Selection Network for 3D Semantic and Instance Segmentation.
UNet-backbone	0.319 41	0.667 33	0.715 33	0.233 40	0.189 42	0.479 45	0.008 37	0.218 42	0.067 43	0.201 43	0.173 42	0.107 39	0.123 45	0.438 41	0.150 41	0.615 38	0.355 38	0.916 35	0.093 48

R-PointNet	0.306 42	0.500 44	0.405 46	0.311 36	0.348 34	0.589 35	0.054 31	0.068 47	0.126 39	0.283 39	0.290 37	0.028 45	0.219 43	0.214 44	0.331 37	0.396 47	0.275 43	0.821 42	0.245 38

Region-18class	0.284 43	0.250 48	0.751 29	0.228 42	0.270 38	0.521 41	0.000 41	0.468 37	0.008 47	0.205 42	0.127 43	0.000 49	0.068 47	0.070 47	0.262 40	0.652 34	0.323 40	0.740 43	0.173 42

SemRegionNet-20cls	0.250 44	0.333 45	0.613 39	0.229 41	0.163 43	0.493 42	0.000 41	0.304 40	0.107 40	0.147 45	0.100 44	0.052 43	0.231 41	0.119 45	0.039 45	0.445 45	0.325 39	0.654 44	0.141 44

3D-BEVIS	0.248 45	0.667 33	0.566 40	0.076 48	0.035 49	0.394 47	0.027 35	0.035 48	0.098 41	0.099 47	0.030 48	0.025 46	0.098 46	0.375 43	0.126 42	0.604 39	0.181 47	0.854 41	0.171 43
Cathrin Elich, Francis Engelmann, Jonas Schult, Theodora Kontogianni, Bastian Leibe: 3D-BEVIS: Birds-Eye-View Instance Segmentation.
tmp	0.248 45	0.667 33	0.437 44	0.188 43	0.153 45	0.491 43	0.000 41	0.208 43	0.094 42	0.153 44	0.099 45	0.057 42	0.217 44	0.119 45	0.039 45	0.466 44	0.302 41	0.640 45	0.140 45

ASIS	0.199 47	0.333 45	0.253 48	0.167 46	0.140 46	0.438 46	0.000 41	0.177 45	0.008 46	0.121 46	0.069 46	0.004 48	0.231 42	0.429 42	0.036 47	0.445 46	0.273 44	0.333 48	0.119 47

Sgpn_scannet	0.143 48	0.208 49	0.390 47	0.169 45	0.065 47	0.275 48	0.029 34	0.069 46	0.000 48	0.087 48	0.043 47	0.014 47	0.027 49	0.000 48	0.112 44	0.351 48	0.168 48	0.438 47	0.138 46

MaskRCNN 2d->3d Proj	0.058 49	0.333 45	0.002 49	0.000 49	0.053 48	0.002 49	0.002 40	0.021 49	0.000 48	0.045 49	0.024 49	0.238 32	0.065 48	0.000 48	0.014 48	0.107 49	0.020 49	0.110 49	0.006 49

3D Semantic Instance Benchmark