The 3D semantic instance prediction task involves detecting and segmenting the object in an 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the average precision for each class. We report the mean average precision AP at overlap 0.25 (AP 25%), overlap 0.5 (AP 50%), and over overlaps in the range [0.5:0.95:0.05] (AP). Note that multiple predictions of the same ground truth instance are penalized as false positives.



This table lists the benchmark results for the 3D semantic instance scenario.




Method Infoavg apbathtubbedbookshelfcabinetchaircountercurtaindeskdoorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwindow
sorted bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort by
Volt-SPFormerScanNetpermissive0.640 10.920 120.665 120.634 10.582 10.794 50.242 190.559 210.496 10.535 20.646 10.737 20.709 60.731 290.509 40.796 20.566 160.944 100.457 2
Kadir Yilmaz, Adrian Kruse, Tristan Höfer, Daan de Geus, Bastian Leibe: Volume Transformer: Revisiting Vanilla Transformers for 3D Scene Understanding.
PointComp0.629 20.787 260.679 100.574 60.502 40.824 10.378 10.480 400.483 40.480 160.601 20.744 10.682 90.809 80.460 210.819 10.643 20.935 140.449 4
PointRel0.622 30.926 70.710 30.541 120.502 30.772 100.314 50.598 110.425 110.504 120.565 40.650 90.716 20.809 70.476 130.747 70.618 30.963 30.364 22
: Relation3D: Enhancing Relation Modeling for Point Cloud Instance Segmentation. CVPR 2025
Competitor-MAFT0.618 40.866 160.724 10.628 20.484 60.803 30.300 90.509 330.496 20.539 10.547 80.703 30.668 100.708 350.463 180.708 200.595 40.959 50.418 9
SIM3D0.617 50.952 30.629 200.539 130.426 180.768 140.302 80.681 20.425 120.473 180.511 180.701 40.717 10.821 60.467 160.774 30.559 170.914 210.448 5
Spherical Mask(CtoF)0.616 60.946 40.654 150.555 80.434 150.769 130.271 140.604 80.447 60.505 100.549 50.698 50.716 20.775 170.480 100.747 80.575 120.925 160.436 7
EV3D0.615 70.946 40.652 160.555 80.433 160.773 90.271 150.604 80.447 60.506 90.544 90.698 50.716 20.775 170.480 100.747 80.572 140.925 160.435 8
DCD0.614 80.892 130.633 190.434 300.495 50.810 20.292 100.501 340.408 130.525 60.582 30.688 70.625 120.801 90.608 10.672 230.649 10.965 20.476 1
ExtMask3D0.598 90.852 170.692 80.433 330.461 90.791 60.264 160.488 370.493 30.508 80.528 170.594 150.706 70.791 110.483 80.734 120.595 50.911 230.437 6
MAFT0.596 100.889 140.721 20.448 250.460 100.768 150.251 180.558 220.408 140.504 110.539 110.616 120.618 140.858 30.482 90.684 220.551 200.931 150.450 3
MG-Former0.587 110.852 170.639 180.454 240.393 240.758 180.338 30.572 160.480 50.527 40.491 250.671 80.527 240.867 10.485 70.601 340.590 80.938 130.390 14
InsSSM0.586 121.000 10.593 240.440 280.480 70.771 110.345 20.437 430.444 90.495 150.548 70.579 190.621 130.720 310.409 250.712 150.593 60.960 40.395 12
Lei Yao, Yi Wang, Moyun Liu, Lap-Pui Chau: SGIFormer: Semantic-guided and Geometric-enhanced Interleaving Transformer for 3D Instance Segmentation. TCSVT, 2024
Queryformer0.583 130.926 70.702 50.393 390.504 20.733 240.276 130.527 280.373 200.479 170.534 130.533 260.697 80.720 320.436 230.745 100.592 70.958 60.363 23
KmaxOneFormerNetpermissive0.581 140.745 310.692 90.551 100.458 110.798 40.264 170.531 270.369 220.513 70.531 160.632 100.494 270.798 100.567 30.648 270.558 190.950 80.362 25
Competitor-SPFormer0.580 150.721 380.705 40.593 50.444 140.786 80.286 110.564 190.376 190.498 140.534 140.546 240.390 480.785 130.577 20.708 190.579 100.954 70.388 15
VDG-Uni3DSeg0.576 160.833 210.699 60.483 180.412 220.767 160.313 60.461 420.446 80.526 50.498 230.584 160.551 200.743 260.464 170.766 40.538 240.919 190.363 24
PBNetpermissive0.573 170.926 70.575 300.619 30.472 80.736 220.239 210.487 380.383 180.459 220.506 210.533 250.585 160.767 190.404 270.717 140.559 180.969 10.381 18
Weiguang Zhao, Yuyao Yan, Chaolong Yang, Jianan Ye, Xi Yang, Kaizhu Huang: Divide and Conquer: 3D Instance Segmentation With Point-Wise Binarization. ICCV 2023
TST3D0.569 180.778 280.675 110.598 40.451 130.727 250.280 120.476 410.395 150.472 190.457 310.583 170.580 180.777 140.462 200.735 110.547 220.919 200.333 31
Duc Tran Dang Trung, Byeongkeun Kang, Yeejin Lee: MSTA3D: Multi-scale Twin-attention for 3D Instance Segmentation. ACM Multimedia 2024
Mask3D0.566 190.926 70.597 230.408 360.420 200.737 210.239 200.598 110.386 170.458 230.549 50.568 220.716 20.601 480.480 100.646 280.575 120.922 180.364 21
Jonas Schult, Francis Engelmann, Alexander Hermans, Or Litany, Siyu Tang, Bastian Leibe: Mask3D for 3D Semantic Instance Segmentation. ICRA 2023
OneFormer3Dcopyleft0.566 190.781 270.697 70.562 70.431 170.770 120.331 40.400 490.373 210.529 30.504 220.568 210.475 310.732 280.470 140.762 50.550 210.871 380.379 19
Maxim Kolodiazhnyi, Anna Vorontsova, Anton Konushin, Danila Rukhovich: OneFormer3D: One Transformer for Unified Point Cloud Segmentation.
UniPerception0.560 210.815 220.659 130.388 410.453 120.786 70.212 220.526 290.441 100.471 200.539 100.607 130.442 360.671 420.406 260.731 130.577 110.944 110.411 10
ISBNetpermissive0.559 220.939 60.655 140.383 430.426 190.763 170.180 240.534 260.386 160.499 130.509 200.621 110.427 420.704 370.467 150.649 260.571 150.948 90.401 11
Tuan Duc Ngo, Binh-Son Hua, Khoi Nguyen: ISBNet: a 3D Point Cloud Instance Segmentation Network with Instance-aware Sampling and Box-aware Dynamic Convolution. CVPR 2023
GraphCut0.552 231.000 10.611 220.438 290.392 250.714 260.139 280.598 130.327 260.389 260.510 190.598 140.427 430.754 220.463 190.761 60.588 90.903 260.329 33
SPFormerpermissive0.549 240.745 310.640 170.484 170.395 230.739 200.311 70.566 180.335 240.468 210.492 240.555 230.478 300.747 240.436 220.712 160.540 230.893 300.343 30
Sun Jiahao, Qing Chunmei, Tan Junpeng, Xu Xiangmin: Superpoint Transformer for 3D Scene Instance Segmentation. AAAI 2023 [Oral]
DKNet0.532 250.815 220.624 210.517 140.377 270.749 190.107 300.509 320.304 280.437 240.475 260.581 180.539 220.775 160.339 330.640 300.506 270.901 270.385 17
Yizheng Wu, Min Shi, Shuaiyuan Du, Hao Lu, Zhiguo Cao, Weicai Zhong: 3D Instances as 1D Kernels. ECCV 2022
IPCA-Inst0.520 260.889 140.551 340.548 110.418 210.665 360.064 390.585 140.260 360.277 410.471 280.500 270.644 110.785 120.369 290.591 380.511 250.878 350.362 26
SoftGroup++0.513 270.704 400.578 290.398 380.363 330.704 270.061 400.647 50.297 330.378 290.537 120.343 310.614 150.828 50.295 380.710 180.505 290.875 370.394 13
SSTNetpermissive0.506 280.738 350.549 350.497 160.316 390.693 300.178 250.377 530.198 420.330 320.463 300.576 200.515 250.857 40.494 50.637 310.457 330.943 120.290 42
Zhihao Liang, Zhihao Li, Songcen Xu, Mingkui Tan, Kui Jia: Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks. ICCV2021
DANCENET0.504 290.926 70.579 260.472 200.367 300.626 460.165 260.432 440.221 380.408 250.449 330.411 290.564 190.746 250.421 240.707 210.438 360.846 460.288 43
SoftGrouppermissive0.504 290.667 470.579 270.372 450.381 260.694 290.072 360.677 30.303 290.387 270.531 150.319 350.582 170.754 210.318 340.643 290.492 300.907 250.388 16
Thang Vu, Kookhoi Kim, Tung M. Luu, Xuan Thanh Nguyen, Chang D. Yoo: SoftGroup for 3D Instance Segmentaiton on Point Clouds. CVPR 2022 [Oral]
TD3Dpermissive0.489 310.852 170.511 440.434 310.322 380.735 230.101 330.512 310.355 230.349 310.468 290.283 390.514 260.676 410.268 430.671 240.510 260.908 240.329 34
Maksim Kolodiazhnyi, Anna Vorontsova, Anton Konushin, Danila Rukhovich: Top-Down Beats Bottom-Up in 3D Instance Segmentation. WACV 2024
OccuSeg+instance0.486 320.802 250.536 370.428 340.369 290.702 280.205 230.331 580.301 300.379 280.474 270.327 320.437 370.862 20.485 60.601 350.394 440.846 480.273 46
Lei Han, Tian Zheng, Lan Xu, Lu Fang: OccuSeg: Occupancy-aware 3D Instance Segmentation. CVPR2020
TopoSeg0.479 330.704 400.564 310.467 220.366 310.633 440.068 370.554 230.262 350.328 330.447 340.323 330.534 230.722 300.288 400.614 320.482 310.912 220.358 28
DualGroup0.469 340.815 220.552 330.398 370.374 280.683 320.130 290.539 250.310 270.327 340.407 370.276 400.447 350.535 520.342 320.659 250.455 340.900 290.301 38
SSEC0.465 350.667 470.578 280.502 150.362 340.641 430.035 490.605 70.291 340.323 350.451 320.296 370.417 460.677 400.245 470.501 560.506 280.900 280.366 20
ODIN - Inspermissive0.463 360.738 350.589 250.344 490.358 350.560 550.139 270.393 520.331 250.373 300.392 400.496 280.493 280.709 340.377 280.599 360.359 500.752 580.332 32
Ayush Jain, Pushkal Katara, Nikolaos Gkanatsios, Adam W. Harley, Gabriel Sarch, Kriti Aggarwal, Vishrav Chaudhary, Katerina Fragkiadaki: ODIN: A Single Model for 2D and 3D Segmentation. CVPR 2024
HAISpermissive0.457 370.704 400.561 320.457 230.364 320.673 330.046 480.547 240.194 430.308 360.426 350.288 380.454 340.711 330.262 440.563 460.434 380.889 320.344 29
Shaoyu Chen, Jiemin Fang, Qian Zhang, Wenyu Liu, Xinggang Wang: Hierarchical Aggregation for 3D Instance Segmentation. ICCV 2021
DD-UNet+Group0.436 380.630 550.508 470.480 190.310 410.624 480.065 380.638 60.174 440.256 450.384 420.194 520.428 400.759 200.289 390.574 430.400 420.849 450.291 41
H. Liu, R. Liu, K. Yang, J. Zhang, K. Peng, R. Stiefelhagen: HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensor. ICCVW 2021
INS-Conv-instance0.435 390.716 390.495 490.355 470.331 360.689 310.102 320.394 510.208 410.280 390.395 390.250 430.544 210.741 270.309 360.536 520.391 450.842 510.258 50
Mask-Group0.434 400.778 280.516 420.471 210.330 370.658 370.029 510.526 300.249 370.256 440.400 380.309 360.384 510.296 680.368 300.575 420.425 390.877 360.362 27
Min Zhong, Xinghao Chen, Xiaokang Chen, Gang Zeng, Yunhe Wang: MaskGroup: Hierarchical Point Grouping and Masking for 3D Instance Segmentation. ICME 2022
Box2Mask0.433 410.741 330.463 540.433 320.283 440.625 470.103 310.298 630.125 530.260 430.424 360.322 340.472 320.701 380.363 310.711 170.309 620.882 330.272 48
Julian Chibane, Francis Engelmann, Tuan Anh Tran, Gerard Pons-Moll: Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation Using Bounding Boxes. ECCV 2022
RPGN0.428 420.630 550.508 460.367 460.249 510.658 380.016 590.673 40.131 510.234 480.383 430.270 410.434 380.748 230.274 420.609 330.406 410.842 500.267 49
Shichao Dong, Guosheng Lin, Tzu-Yi Hung: Learning Regional Purity for Instance Segmentation on 3D Point Clouds. ECCV 2022
DENet0.413 430.741 330.520 390.237 590.284 430.523 580.097 340.691 10.138 480.209 580.229 600.238 460.390 490.707 360.310 350.448 630.470 320.892 310.310 36
PointGroup0.407 440.639 540.496 480.415 350.243 530.645 420.021 560.570 170.114 540.211 560.359 450.217 500.428 410.660 430.256 450.562 470.341 540.860 410.291 40
Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi-Wing Fu, Jiaya Jia: PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation. CVPR 2020 [oral]
CSC-Pretrained0.405 450.738 350.465 530.331 520.205 570.655 390.051 440.601 100.092 580.211 570.329 480.198 510.459 330.775 150.195 540.524 540.400 430.878 340.184 59
PE0.396 460.667 470.467 520.446 270.243 520.624 490.022 550.577 150.106 550.219 510.340 460.239 450.487 290.475 590.225 490.541 510.350 520.818 530.273 47
Biao Zhang, Peter Wonka: Point Cloud Instance Segmentation using Probabilistic Embeddings. CVPR 2021
Dyco3Dcopyleft0.395 470.642 530.518 410.447 260.259 500.666 350.050 450.251 680.166 450.231 490.362 440.232 470.331 540.535 510.229 480.587 390.438 370.850 430.317 35
Tong He; Chunhua Shen; Anton van den Hengel: DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolution. CVPR2021
OSIS0.392 480.778 280.530 380.220 610.278 450.567 540.083 350.330 590.299 310.270 420.310 510.143 580.260 580.624 460.277 410.568 450.361 490.865 400.301 37
AOIA0.387 490.704 400.515 430.385 420.225 560.669 340.005 660.482 390.126 520.181 610.269 570.221 490.426 440.478 580.218 500.592 370.371 470.851 420.242 52
SSEN0.384 500.852 170.494 500.192 620.226 550.648 410.022 540.398 500.299 320.277 400.317 500.231 480.194 650.514 550.196 520.586 400.444 350.843 490.184 58
Dongsu Zhang, Junha Chun, Sang Kyun Cha, Young Min Kim: Spatial Semantic Embedding Network: Fast 3D Instance Segmentation with Deep Metric Learning. Arxiv
Mask3D_evaluation0.382 510.593 570.520 400.390 400.314 400.600 500.018 580.287 660.151 470.281 380.387 410.169 560.429 390.654 440.172 580.578 410.384 460.670 650.278 45
PCJC0.375 520.704 400.542 360.284 560.197 590.649 400.006 630.426 450.138 490.242 460.304 520.183 550.388 500.629 450.141 650.546 500.344 530.738 600.283 44
ClickSeg_Instance0.366 530.654 510.375 580.184 630.302 420.592 520.050 460.300 620.093 570.283 370.277 540.249 440.426 450.615 470.299 370.504 550.367 480.832 520.191 57
SphereSeg0.357 540.651 520.411 560.345 480.264 490.630 450.059 410.289 650.212 390.240 470.336 470.158 570.305 550.557 490.159 610.455 620.341 550.726 620.294 39
3D-MPA0.355 550.457 670.484 510.299 540.277 460.591 530.047 470.332 560.212 400.217 520.278 530.193 530.413 470.410 620.195 530.574 440.352 510.849 440.213 55
Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, Matthias Nießner: 3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation. CVPR 2020
NeuralBF0.353 560.593 570.511 450.375 440.264 480.597 510.008 610.332 570.160 460.229 500.274 560.000 790.206 620.678 390.155 620.485 580.422 400.816 540.254 51
Weiwei Sun, Daniel Rebain, Renjie Liao, Vladimir Tankovich, Soroosh Yazdani, Kwang Moo Yi, Andrea Tagliasacchi: NeuralBF: Neural Bilateral Filtering for Top-down Instance Segmentation on Point Clouds. WACV 2023
RWSeg0.348 570.475 640.456 550.320 530.275 470.476 600.020 570.491 360.056 650.212 550.320 490.261 420.302 560.520 530.182 560.557 480.285 640.867 390.197 56
GICN0.341 580.580 590.371 590.344 500.198 580.469 610.052 430.564 200.093 560.212 540.212 620.127 600.347 530.537 500.206 510.525 530.329 570.729 610.241 53
One_Thing_One_Clickpermissive0.326 590.472 650.361 600.232 600.183 600.555 560.000 720.498 350.038 670.195 590.226 610.362 300.168 660.469 600.251 460.553 490.335 560.846 470.117 67
Zhengzhe Liu, Xiaojuan Qi, Chi-Wing Fu: One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation. CVPR 2021
Occipital-SCS0.320 600.679 460.352 610.334 510.229 540.436 620.025 520.412 480.058 630.161 660.240 590.085 620.262 570.496 570.187 550.467 600.328 580.775 550.231 54
Sparse R-CNN0.292 610.704 400.213 710.153 650.154 620.551 570.053 420.212 690.132 500.174 630.274 550.070 640.363 520.441 610.176 570.424 650.234 660.758 570.161 63
MTML0.282 620.577 600.380 570.182 640.107 680.430 630.001 690.422 460.057 640.179 620.162 650.070 650.229 600.511 560.161 590.491 570.313 590.650 680.162 61
Jean Lahoud, Bernard Ghanem, Marc Pollefeys, Martin R. Oswald: 3D Instance Segmentation via Multi-task Metric Learning. ICCV 2019 [oral]
SALoss-ResNet0.262 630.667 470.335 620.067 720.123 660.427 640.022 530.280 670.058 620.216 530.211 630.039 680.142 680.519 540.106 690.338 690.310 610.721 630.138 64
Zhidong Liang, Ming Yang, Hao Li, Chunxiang Wang: 3D Instance Embedding Learning With a Structure-Aware Loss Function for Point Cloud Segmentation. IEEE Robotics and Automation Letters (IROS2020)
MASCpermissive0.254 640.463 660.249 700.113 660.167 610.412 660.000 710.374 540.073 590.173 640.243 580.130 590.228 610.368 640.160 600.356 670.208 670.711 640.136 65
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
3D-BoNet0.253 650.519 620.324 650.251 580.137 650.345 710.031 500.419 470.069 600.162 650.131 670.052 660.202 640.338 660.147 640.301 720.303 630.651 670.178 60
Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, Niki Trigoni: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds. NeurIPS 2019 Spotlight
SPG_WSIS0.251 660.380 690.274 680.289 550.144 630.413 650.000 720.311 600.065 610.113 680.130 680.029 710.204 630.388 630.108 680.459 610.311 600.769 560.127 66
SegGroup_inspermissive0.246 670.556 610.335 630.062 740.115 670.490 590.000 720.297 640.018 710.186 600.142 660.083 630.233 590.216 700.153 630.469 590.251 650.744 590.083 70
An Tao, Yueqi Duan, Yi Wei, Jiwen Lu, Jie Zhou: SegGroup: Seg-Level Supervision for 3D Instance and Semantic Segmentation. TIP 2022
PanopticFusion-inst0.214 680.250 740.330 640.275 570.103 690.228 770.000 720.345 550.024 690.088 700.203 640.186 540.167 670.367 650.125 660.221 750.112 770.666 660.162 62
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
UNet-backbone0.161 690.519 620.259 690.084 680.059 710.325 730.002 670.093 740.009 730.077 720.064 710.045 670.044 750.161 720.045 710.331 700.180 690.566 690.033 79
3D-SISpermissive0.161 690.407 680.155 760.068 710.043 750.346 700.001 680.134 710.005 740.088 690.106 700.037 690.135 700.321 670.028 750.339 680.116 760.466 720.093 69
Ji Hou, Angela Dai, Matthias Niessner: 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans. CVPR 2019
R-PointNet0.158 710.356 700.173 740.113 670.140 640.359 670.012 600.023 770.039 660.134 670.123 690.008 750.089 710.149 730.117 670.221 740.128 740.563 700.094 68
Region-18class0.146 720.175 780.321 660.080 690.062 700.357 680.000 720.307 610.002 760.066 730.044 730.000 790.018 770.036 780.054 700.447 640.133 720.472 710.060 74
SemRegionNet-20cls0.121 730.296 720.203 720.071 700.058 720.349 690.000 720.150 700.019 700.054 750.034 760.017 740.052 730.042 770.013 780.209 760.183 680.371 730.057 75
3D-BEVIS0.117 740.250 740.308 670.020 780.009 800.269 760.006 640.008 780.029 680.037 780.014 790.003 770.036 760.147 740.042 730.381 660.118 750.362 740.069 73
Cathrin Elich, Francis Engelmann, Jonas Schult, Theodora Kontogianni, Bastian Leibe: 3D-BEVIS: Birds-Eye-View Instance Segmentation.
Hier3Dcopyleft0.117 740.222 760.161 750.054 760.027 770.289 740.000 720.124 720.001 780.079 710.061 720.027 720.141 690.240 690.005 790.310 710.129 730.153 790.081 71
Tan: HCFS3D: Hierarchical Coupled Feature Selection Network for 3D Semantic and Instance Segmentation.
tmp0.113 760.333 710.151 770.056 750.053 730.344 720.000 720.105 730.016 720.049 760.035 750.020 730.053 720.048 760.013 770.183 780.173 700.344 760.054 76
Sem_Recon_ins0.098 770.295 730.187 730.015 790.036 760.213 780.005 650.038 760.003 750.056 740.037 740.036 700.015 780.051 750.044 720.209 770.098 780.354 750.071 72
ASIS0.085 780.037 790.080 790.066 730.047 740.282 750.000 720.052 750.002 770.047 770.026 770.001 780.046 740.194 710.031 740.264 730.140 710.167 780.047 78
Sgpn_scannet0.049 790.023 800.134 780.031 770.013 790.144 790.006 620.008 790.000 790.028 790.017 780.003 760.009 800.000 790.021 760.122 790.095 790.175 770.054 77
MaskRCNN 2d->3d Proj0.022 800.185 770.000 800.000 800.015 780.000 800.000 700.006 800.000 790.010 800.006 800.107 610.012 790.000 790.002 800.027 800.004 800.022 800.001 80