3D Semantic Instance Results - ScanNet Benchmark

The 3D semantic instance prediction task involves detecting and segmenting the object in an 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the average precision for each class. We report the mean average precision AP at overlap 0.25 (AP 25%), overlap 0.5 (AP 50%), and over overlaps in the range [0.5:0.95:0.05] (AP). Note that multiple predictions of the same ground truth instance are penalized as false positives.

This table lists the benchmark results for the 3D semantic instance scenario.

Method	avg ap 50%	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	window

Mask3D	0.780 1	1.000 1	0.786 22	0.716 21	0.696 4	0.885 3	0.500 2	0.714 17	0.810 1	0.672 3	0.715 3	0.679 6	0.809 1	1.000 1	0.831 1	0.833 8	0.787 3	1.000 1	0.602 4
Jonas Schult, Francis Engelmann, Alexander Hermans, Or Litany, Siyu Tang, Bastian Leibe: Mask3D for 3D Semantic Instance Segmentation.
SPFormer	0.770 2	0.903 33	0.903 1	0.806 9	0.609 12	0.886 2	0.568 1	0.815 6	0.705 3	0.711 1	0.655 4	0.652 8	0.685 8	1.000 1	0.789 3	0.809 11	0.776 4	1.000 1	0.583 7
Sun Jiahao, Qing Chunmei, Tan Junpeng, Xu Xiangmin: Superpoint Transformer for 3D Scene Instance Segmentation. AAAI 2023
SoftGroup++	0.769 3	1.000 1	0.803 17	0.937 1	0.684 5	0.865 5	0.213 16	0.870 2	0.664 4	0.571 6	0.758 1	0.702 4	0.807 2	1.000 1	0.653 15	0.902 1	0.792 2	1.000 1	0.626 1

SoftGroup	0.761 4	1.000 1	0.808 14	0.845 6	0.716 1	0.862 7	0.243 13	0.824 3	0.655 6	0.620 4	0.734 2	0.699 5	0.791 4	0.981 22	0.716 6	0.844 4	0.769 5	1.000 1	0.594 6
Thang Vu, Kookhoi Kim, Tung M. Luu, Xuan Thanh Nguyen, Chang D. Yoo: SoftGroup for 3D Instance Segmentaiton on Point Clouds. CVPR 2022 [Oral]
PBNet	0.734 5	1.000 1	0.810 11	0.824 7	0.704 2	0.830 10	0.492 3	0.696 18	0.601 11	0.562 8	0.593 10	0.587 11	0.650 10	1.000 1	0.698 9	0.844 5	0.762 6	1.000 1	0.556 14

GraphCut	0.732 6	1.000 1	0.788 20	0.724 20	0.642 8	0.859 8	0.248 12	0.787 10	0.618 10	0.596 5	0.653 6	0.722 2	0.583 26	1.000 1	0.766 4	0.861 2	0.825 1	1.000 1	0.504 18

IPCA-Inst	0.731 7	1.000 1	0.788 21	0.884 5	0.698 3	0.788 23	0.252 11	0.760 12	0.646 7	0.511 15	0.637 8	0.665 7	0.804 3	1.000 1	0.644 16	0.778 13	0.747 8	1.000 1	0.561 12

TopoSeg	0.725 8	1.000 1	0.806 16	0.933 2	0.668 7	0.758 26	0.272 9	0.734 16	0.630 8	0.549 11	0.654 5	0.606 9	0.697 7	0.966 24	0.612 19	0.839 6	0.754 7	1.000 1	0.573 8

DKNet	0.718 9	1.000 1	0.814 10	0.782 12	0.619 9	0.872 4	0.224 14	0.751 14	0.569 13	0.677 2	0.585 12	0.724 1	0.633 18	0.981 22	0.515 26	0.819 9	0.736 9	1.000 1	0.617 2
Yizheng Wu, Min Shi, Shuaiyuan Du, Hao Lu, Zhiguo Cao, Weicai Zhong: 3D Instances as 1D Kernels. ECCV 2022
SSEC	0.700 10	1.000 1	0.848 5	0.763 18	0.609 13	0.792 21	0.262 10	0.824 3	0.627 9	0.535 13	0.547 20	0.493 16	0.600 20	1.000 1	0.712 8	0.731 25	0.689 13	1.000 1	0.563 11

HAIS	0.699 11	1.000 1	0.849 4	0.820 8	0.675 6	0.808 16	0.279 7	0.757 13	0.465 18	0.517 14	0.596 9	0.559 12	0.600 20	1.000 1	0.654 14	0.767 15	0.676 14	0.994 29	0.560 13
Shaoyu Chen, Jiemin Fang, Qian Zhang, Wenyu Liu, Xinggang Wang: Hierarchical Aggregation for 3D Instance Segmentation. ICCV 2021
SSTNet	0.698 12	1.000 1	0.697 37	0.888 4	0.556 19	0.803 17	0.387 5	0.626 25	0.417 22	0.556 10	0.585 13	0.702 3	0.600 20	1.000 1	0.824 2	0.720 27	0.692 11	1.000 1	0.509 17
Zhihao Liang, Zhihao Li, Songcen Xu, Mingkui Tan, Kui Jia: Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks. ICCV2021
SphereSeg	0.680 13	1.000 1	0.856 3	0.744 19	0.618 10	0.893 1	0.151 18	0.651 23	0.713 2	0.537 12	0.579 15	0.430 24	0.651 9	1.000 1	0.389 35	0.744 22	0.697 10	0.991 30	0.601 5

Box2Mask	0.677 14	1.000 1	0.847 6	0.771 14	0.509 25	0.816 12	0.277 8	0.558 32	0.482 15	0.562 9	0.640 7	0.448 20	0.700 5	1.000 1	0.666 10	0.852 3	0.578 25	0.997 24	0.488 22
Julian Chibane, Francis Engelmann, Tuan Anh Tran, Gerard Pons-Moll: Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation Using Bounding Boxes. ECCV 2022
OccuSeg+instance	0.672 15	1.000 1	0.758 29	0.682 23	0.576 17	0.842 9	0.477 4	0.504 35	0.524 14	0.567 7	0.585 14	0.451 19	0.557 27	1.000 1	0.751 5	0.797 12	0.563 28	1.000 1	0.467 25
Lei Han, Tian Zheng, Lan Xu, Lu Fang: OccuSeg: Occupancy-aware 3D Instance Segmentation. CVPR2020
Mask-Group	0.664 16	1.000 1	0.822 9	0.764 17	0.616 11	0.815 13	0.139 22	0.694 20	0.597 12	0.459 20	0.566 16	0.599 10	0.600 20	0.516 41	0.715 7	0.819 10	0.635 18	1.000 1	0.603 3
Min Zhong, Xinghao Chen, Xiaokang Chen, Gang Zeng, Yunhe Wang: MaskGroup: Hierarchical Point Grouping and Masking for 3D Instance Segmentation. ICME 2022
INS-Conv-instance	0.657 17	1.000 1	0.760 27	0.667 25	0.581 15	0.863 6	0.323 6	0.655 22	0.477 16	0.473 18	0.549 18	0.432 23	0.650 10	1.000 1	0.655 13	0.738 23	0.585 24	0.944 34	0.472 24

CSC-Pretrained	0.648 18	1.000 1	0.810 12	0.768 15	0.523 24	0.813 14	0.143 21	0.819 5	0.389 23	0.422 27	0.511 23	0.443 21	0.650 10	1.000 1	0.624 18	0.732 24	0.634 19	1.000 1	0.375 31

PE	0.645 19	1.000 1	0.773 24	0.798 11	0.538 21	0.786 24	0.088 29	0.799 9	0.350 27	0.435 26	0.547 19	0.545 13	0.646 17	0.933 25	0.562 22	0.761 18	0.556 33	0.997 24	0.501 20
Biao Zhang, Peter Wonka: Point Cloud Instance Segmentation using Probabilistic Embeddings. CVPR 2021
RPGN	0.643 20	1.000 1	0.758 28	0.582 33	0.539 20	0.826 11	0.046 33	0.765 11	0.372 25	0.436 25	0.588 11	0.539 15	0.650 10	1.000 1	0.577 20	0.750 20	0.653 17	0.997 24	0.495 21
Shichao Dong, Guosheng Lin, Tzu-Yi Hung: Learning Regional Purity for Instance Segmentation on 3D Point Clouds. ECCV 2022
Dyco3D	0.641 21	1.000 1	0.841 7	0.893 3	0.531 22	0.802 18	0.115 26	0.588 30	0.448 19	0.438 23	0.537 22	0.430 25	0.550 28	0.857 27	0.534 24	0.764 17	0.657 15	0.987 31	0.568 9
Tong He; Chunhua Shen; Anton van den Hengel: DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolution. CVPR2021
GICN	0.638 22	1.000 1	0.895 2	0.800 10	0.480 28	0.676 30	0.144 20	0.737 15	0.354 26	0.447 21	0.400 32	0.365 30	0.700 5	1.000 1	0.569 21	0.836 7	0.599 21	1.000 1	0.473 23

PointGroup	0.636 23	1.000 1	0.765 25	0.624 27	0.505 27	0.797 19	0.116 25	0.696 18	0.384 24	0.441 22	0.559 17	0.476 17	0.596 24	1.000 1	0.666 10	0.756 19	0.556 32	0.997 24	0.513 16
Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi-Wing Fu, Jiaya Jia: PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation. CVPR 2020 [oral]
DD-UNet+Group	0.635 24	0.667 34	0.797 19	0.714 22	0.562 18	0.774 25	0.146 19	0.810 8	0.429 21	0.476 17	0.546 21	0.399 27	0.633 18	1.000 1	0.632 17	0.722 26	0.609 20	1.000 1	0.514 15
H. Liu, R. Liu, K. Yang, J. Zhang, K. Peng, R. Stiefelhagen: HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensor. ICCVW 2021
DENet	0.629 25	1.000 1	0.797 18	0.608 28	0.589 14	0.627 34	0.219 15	0.882 1	0.310 29	0.402 31	0.383 34	0.396 28	0.650 10	1.000 1	0.663 12	0.543 42	0.691 12	1.000 1	0.568 10

3D-MPA	0.611 26	1.000 1	0.833 8	0.765 16	0.526 23	0.756 27	0.136 24	0.588 30	0.470 17	0.438 24	0.432 30	0.358 31	0.650 10	0.857 27	0.429 31	0.765 16	0.557 31	1.000 1	0.430 27
Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, Matthias Nießner: 3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation. CVPR 2020
PCJC	0.578 27	1.000 1	0.810 13	0.583 32	0.449 31	0.813 15	0.042 34	0.603 28	0.341 28	0.490 16	0.465 26	0.410 26	0.650 10	0.835 33	0.264 40	0.694 31	0.561 29	0.889 38	0.504 19

SSEN	0.575 28	1.000 1	0.761 26	0.473 35	0.477 29	0.795 20	0.066 30	0.529 33	0.658 5	0.460 19	0.461 27	0.380 29	0.331 40	0.859 26	0.401 34	0.692 32	0.653 16	1.000 1	0.348 33
Dongsu Zhang, Junha Chun, Sang Kyun Cha, Young Min Kim: Spatial Semantic Embedding Network: Fast 3D Instance Segmentation with Deep Metric Learning. Arxiv
RWSeg	0.567 29	0.528 43	0.708 36	0.626 26	0.580 16	0.745 28	0.063 31	0.627 24	0.240 33	0.400 32	0.497 24	0.464 18	0.515 29	1.000 1	0.475 28	0.745 21	0.571 26	1.000 1	0.429 28

MTML	0.549 30	1.000 1	0.807 15	0.588 31	0.327 36	0.647 32	0.004 39	0.815 7	0.180 35	0.418 28	0.364 36	0.182 35	0.445 33	1.000 1	0.442 30	0.688 33	0.571 27	1.000 1	0.396 29
Jean Lahoud, Bernard Ghanem, Marc Pollefeys, Martin R. Oswald: 3D Instance Segmentation via Multi-task Metric Learning. ICCV 2019 [oral]
One_Thing_One_Click	0.529 31	0.667 34	0.718 32	0.777 13	0.399 32	0.683 29	0.000 42	0.669 21	0.138 38	0.391 33	0.374 35	0.539 14	0.360 39	0.641 38	0.556 23	0.774 14	0.593 22	0.997 24	0.251 38
Zhengzhe Liu, Xiaojuan Qi, Chi-Wing Fu: One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation. CVPR 2021
Sparse R-CNN	0.515 32	1.000 1	0.538 44	0.282 38	0.468 30	0.790 22	0.173 17	0.345 40	0.429 20	0.413 30	0.484 25	0.176 36	0.595 25	0.591 39	0.522 25	0.668 34	0.476 37	0.986 32	0.327 34

Occipital-SCS	0.512 33	1.000 1	0.716 33	0.509 34	0.506 26	0.611 35	0.092 28	0.602 29	0.177 36	0.346 36	0.383 33	0.165 37	0.442 34	0.850 32	0.386 36	0.618 38	0.543 34	0.889 38	0.389 30

3D-BoNet	0.488 34	1.000 1	0.672 39	0.590 30	0.301 38	0.484 45	0.098 27	0.620 26	0.306 30	0.341 37	0.259 40	0.125 39	0.434 36	0.796 34	0.402 33	0.499 44	0.513 36	0.909 37	0.439 26
Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, Niki Trigoni: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds. NeurIPS 2019 Spotlight
PanopticFusion-inst	0.478 35	0.667 34	0.712 35	0.595 29	0.259 41	0.550 41	0.000 42	0.613 27	0.175 37	0.250 42	0.434 28	0.437 22	0.411 38	0.857 27	0.485 27	0.591 41	0.267 47	0.944 34	0.359 32
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
SPG_WSIS	0.470 36	0.667 34	0.685 38	0.677 24	0.372 34	0.562 39	0.000 42	0.482 36	0.244 32	0.316 39	0.298 37	0.052 45	0.442 35	0.857 27	0.267 39	0.702 28	0.559 30	1.000 1	0.287 36

SALoss-ResNet	0.459 37	1.000 1	0.737 31	0.159 48	0.259 40	0.587 37	0.138 23	0.475 37	0.217 34	0.416 29	0.408 31	0.128 38	0.315 41	0.714 35	0.411 32	0.536 43	0.590 23	0.873 41	0.304 35
Zhidong Liang, Ming Yang, Hao Li, Chunxiang Wang: 3D Instance Embedding Learning With a Structure-Aware Loss Function for Point Cloud Segmentation. IEEE Robotics and Automation Letters (IROS2020)
MASC	0.447 38	0.528 43	0.555 42	0.381 36	0.382 33	0.633 33	0.002 40	0.509 34	0.260 31	0.361 35	0.432 29	0.327 32	0.451 32	0.571 40	0.367 37	0.639 36	0.386 38	0.980 33	0.276 37
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
SegGroup_ins	0.445 39	0.667 34	0.773 23	0.185 45	0.317 37	0.656 31	0.000 42	0.407 39	0.134 39	0.381 34	0.267 39	0.217 34	0.476 31	0.714 35	0.452 29	0.629 37	0.514 35	1.000 1	0.222 41
An Tao, Yueqi Duan, Yi Wei, Jiwen Lu, Jie Zhou: SegGroup: Seg-Level Supervision for 3D Instance and Semantic Segmentation. TIP 2022
3D-SIS	0.382 40	1.000 1	0.432 46	0.245 40	0.190 42	0.577 38	0.013 37	0.263 42	0.033 45	0.320 38	0.240 41	0.075 41	0.422 37	0.857 27	0.117 44	0.699 29	0.271 46	0.883 40	0.235 40
Ji Hou, Angela Dai, Matthias Niessner: 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans. CVPR 2019
Hier3D	0.323 41	0.667 34	0.542 43	0.264 39	0.157 45	0.550 40	0.000 42	0.205 45	0.009 46	0.270 41	0.218 42	0.075 41	0.500 30	0.688 37	0.007 50	0.698 30	0.301 43	0.459 47	0.200 42
Tan: HCFS3D: Hierarchical Coupled Feature Selection Network for 3D Semantic and Instance Segmentation.
UNet-backbone	0.319 42	0.667 34	0.715 34	0.233 41	0.189 43	0.479 46	0.008 38	0.218 43	0.067 44	0.201 44	0.173 43	0.107 40	0.123 46	0.438 42	0.150 42	0.615 39	0.355 39	0.916 36	0.093 49

R-PointNet	0.306 43	0.500 45	0.405 47	0.311 37	0.348 35	0.589 36	0.054 32	0.068 48	0.126 40	0.283 40	0.290 38	0.028 46	0.219 44	0.214 45	0.331 38	0.396 48	0.275 44	0.821 43	0.245 39

Region-18class	0.284 44	0.250 49	0.751 30	0.228 43	0.270 39	0.521 42	0.000 42	0.468 38	0.008 48	0.205 43	0.127 44	0.000 50	0.068 48	0.070 48	0.262 41	0.652 35	0.323 41	0.740 44	0.173 43

SemRegionNet-20cls	0.250 45	0.333 46	0.613 40	0.229 42	0.163 44	0.493 43	0.000 42	0.304 41	0.107 41	0.147 46	0.100 45	0.052 44	0.231 42	0.119 46	0.039 46	0.445 46	0.325 40	0.654 45	0.141 45

3D-BEVIS	0.248 46	0.667 34	0.566 41	0.076 49	0.035 50	0.394 48	0.027 36	0.035 49	0.098 42	0.099 48	0.030 49	0.025 47	0.098 47	0.375 44	0.126 43	0.604 40	0.181 48	0.854 42	0.171 44
Cathrin Elich, Francis Engelmann, Jonas Schult, Theodora Kontogianni, Bastian Leibe: 3D-BEVIS: Birds-Eye-View Instance Segmentation.
tmp	0.248 46	0.667 34	0.437 45	0.188 44	0.153 46	0.491 44	0.000 42	0.208 44	0.094 43	0.153 45	0.099 46	0.057 43	0.217 45	0.119 46	0.039 46	0.466 45	0.302 42	0.640 46	0.140 46

ASIS	0.199 48	0.333 46	0.253 49	0.167 47	0.140 47	0.438 47	0.000 42	0.177 46	0.008 47	0.121 47	0.069 47	0.004 49	0.231 43	0.429 43	0.036 48	0.445 47	0.273 45	0.333 49	0.119 48

Sgpn_scannet	0.143 49	0.208 50	0.390 48	0.169 46	0.065 48	0.275 49	0.029 35	0.069 47	0.000 49	0.087 49	0.043 48	0.014 48	0.027 50	0.000 49	0.112 45	0.351 49	0.168 49	0.438 48	0.138 47

MaskRCNN 2d->3d Proj	0.058 50	0.333 46	0.002 50	0.000 50	0.053 49	0.002 50	0.002 41	0.021 50	0.000 49	0.045 50	0.024 50	0.238 33	0.065 49	0.000 49	0.014 49	0.107 50	0.020 50	0.110 50	0.006 50

3D Semantic Instance Benchmark