Benchmark Results - ScanNet Data Efficient

This table lists the benchmark results for the 3D semantic label with limited annotations scenario.

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

Q2E	0.721 1	0.984 1	0.785 1	0.684 2	0.693 2	0.879 1	0.563 1	0.822 1	0.640 1	0.659 1	0.965 2	0.493 1	0.147 5	0.711 1	0.866 1	0.631 3	0.797 1	0.663 1	0.932 2	0.849 1	0.660 1

GaIA	0.638 5	0.536 12	0.783 2	0.651 6	0.600 5	0.840 3	0.413 5	0.728 4	0.490 7	0.520 6	0.948 4	0.475 2	0.299 1	0.518 6	0.680 4	0.629 4	0.729 6	0.573 4	0.906 3	0.815 4	0.626 3
Min Seok Lee, Seok Woo Yang, and Sung Won Han: GaIA: Graphical Information gain based Attention Network for Weakly Supervised 3D Point Cloud Semantic Segmentation. WACV 2023
ActiveST	0.703 2	0.977 2	0.776 3	0.657 5	0.707 1	0.874 2	0.541 2	0.744 2	0.605 2	0.610 2	0.968 1	0.442 4	0.126 6	0.705 2	0.785 2	0.742 1	0.791 2	0.586 2	0.940 1	0.839 2	0.645 2
Gengxin Liu, Oliver van Kaick, Hui Huang, Ruizhen Hu: Active Self-Training for Weakly Supervised 3D Scene Semantic Segmentation.
WS3D_LA_Sem	0.662 3	0.812 4	0.762 4	0.742 1	0.635 3	0.828 6	0.474 3	0.736 3	0.588 3	0.546 3	0.947 5	0.450 3	0.174 4	0.536 5	0.752 3	0.668 2	0.735 5	0.583 3	0.902 6	0.797 6	0.573 5
Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022
LE	0.608 6	0.791 5	0.726 5	0.651 6	0.589 6	0.779 9	0.346 9	0.662 8	0.493 6	0.524 5	0.923 13	0.430 5	0.234 3	0.572 3	0.638 7	0.411 10	0.708 7	0.533 8	0.855 7	0.782 7	0.508 7

DE-3DLearner LA	0.639 4	0.839 3	0.723 6	0.681 3	0.629 4	0.839 5	0.424 4	0.728 4	0.538 5	0.526 4	0.945 6	0.427 6	0.120 7	0.511 7	0.643 6	0.547 6	0.781 3	0.566 5	0.905 4	0.809 5	0.607 4
Ping-Chung Yu, Cheng Sun, Min Sun: Data Efficient 3D Learner via Knowledge Transferred from 2D Model. ECCV 2022
One-Thing-One-Click	0.594 7	0.756 6	0.722 7	0.494 12	0.546 8	0.795 7	0.371 6	0.725 6	0.559 4	0.488 7	0.957 3	0.367 8	0.261 2	0.547 4	0.575 12	0.225 12	0.671 10	0.543 6	0.904 5	0.826 3	0.557 6
Zhengzhe Liu, Xiaojuan Qi, Chi-Wing Fu: One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation. CVPR 2021
PointContrast_LA_SEM	0.550 9	0.735 9	0.676 8	0.601 9	0.475 9	0.794 8	0.288 11	0.621 10	0.378 12	0.430 9	0.940 8	0.303 10	0.089 10	0.379 10	0.580 11	0.531 7	0.689 9	0.422 11	0.852 8	0.758 8	0.468 9

SQN_LA	0.486 12	0.587 11	0.649 9	0.527 11	0.372 12	0.718 12	0.320 10	0.510 12	0.393 11	0.325 12	0.924 12	0.290 11	0.095 9	0.287 11	0.607 9	0.356 11	0.626 12	0.416 12	0.672 12	0.680 13	0.359 12

CSC_LA_SEM	0.531 11	0.659 10	0.638 10	0.578 10	0.417 11	0.775 10	0.254 12	0.537 11	0.396 10	0.439 8	0.939 10	0.284 12	0.083 11	0.414 8	0.599 10	0.488 9	0.698 8	0.444 10	0.785 11	0.747 10	0.440 10

VIBUS	0.586 8	0.736 8	0.623 11	0.664 4	0.559 7	0.840 3	0.358 7	0.666 7	0.447 8	0.429 10	0.944 7	0.421 7	0.000 13	0.411 9	0.629 8	0.614 5	0.745 4	0.541 7	0.848 9	0.758 8	0.493 8
Beiwen Tian,Liyi Luo,Hao Zhao,Guyue Zhou: VIBUS: Data-efficient 3D Scene Parsing with VIewpoint Bottleneck and Uncertainty-Spectrum Modeling. ISPRS Journal of Photogrammetry and Remote Sensing
Scratch_LA_SEM	0.382 13	0.389 13	0.606 12	0.401 13	0.303 13	0.705 13	0.169 13	0.460 13	0.292 13	0.282 13	0.939 10	0.207 13	0.004 12	0.147 13	0.201 13	0.184 13	0.592 13	0.389 13	0.409 13	0.714 12	0.250 13

Viewpoint_BN_LA_AIR	0.548 10	0.747 7	0.574 13	0.631 8	0.456 10	0.762 11	0.355 8	0.639 9	0.412 9	0.404 11	0.940 8	0.335 9	0.107 8	0.277 12	0.645 5	0.495 8	0.666 11	0.517 9	0.818 10	0.740 11	0.431 11
Liyi Luo, Beiwen Tian, Hao Zhao, Guyue Zhou: Pointly-supervised 3D Scene Parsing with Viewpoint Bottleneck.

Method

Info

avg ap

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WS3D_LA_Ins

0.336 1

0.630 1

0.386 1

0.242 2

0.173 1

0.575 1

0.007 1

0.550 1

0.050 1

0.164 1

0.232 1

0.127 2

0.424 1

0.614 1

0.198 1

0.420 2

0.314 1

0.771 1

0.164 1

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

Scratch_LA_INS

0.089 5

0.306 4

0.319 2

0.025 5

0.035 5

0.274 4

0.000 3

0.157 5

0.001 5

0.006 5

0.031 5

0.000 5

0.095 4

0.017 5

0.172 5

0.077 4

0.094 5

0.002 5

CSC_LA_INS

0.159 3

0.343 3

0.247 3

0.226 3

0.063 3

0.264 5

0.000 3

0.268 2

0.004 3

0.018 4

0.064 4

0.007 3

0.009 4

0.041 5

0.091 2

0.279 3

0.194 2

0.712 2

0.037 3

Box2Mask_LA

0.235 2

0.407 2

0.199 4

0.427 1

0.107 2

0.414 2

0.007 2

0.229 3

0.022 2

0.098 2

0.226 2

0.152 1

0.261 2

0.492 2

0.039 4

0.427 1

0.113 3

0.463 4

0.150 2

Julian Chibane, Francis Engelmann, Tuan Anh Tran, Gerard Pons-Moll: Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation Using Bounding Boxes. ECCV 2022

PointContrast_LA_INS

0.124 4

0.037 5

0.109 5

0.085 4

0.052 4

0.287 3

0.000 3

0.219 4

0.001 4

0.040 3

0.104 3

0.002 4

0.074 3

0.211 3

0.055 3

0.239 4

0.036 5

0.661 3

0.020 4

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WS3D_LA_OD

0.341 1

1.000 1

0.629 1

0.426 1

0.070 1

0.608 1

0.063 1

0.176 1

0.503 1

0.132 1

0.084 1

0.001 1

0.337 1

0.220 1

0.103 1

0.628 1

0.282 1

0.739 1

0.131 1

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

CSC_LA_DET

0.135 2

0.444 3

0.336 2

0.029 4

0.001 4

0.356 2

0.008 2

0.000 2

0.011 4

0.045 3

0.010 4

0.000 2

0.032 2

0.011 3

0.043 2

0.458 3

0.028 3

0.602 2

0.010 3

PointContrast_LA_DET

0.135 2

0.667 2

0.296 3

0.145 2

0.002 3

0.330 3

0.000 4

0.000 3

0.050 3

0.049 2

0.023 2

0.000 3

0.002 4

0.006 4

0.034 3

0.472 2

0.052 2

0.285 4

0.011 2

Scratch_LA_DET

0.098 4

0.167 4

0.253 4

0.074 3

0.002 2

0.257 4

0.004 3

0.000 4

0.080 2

0.038 4

0.013 3

0.000 3

0.006 3

0.143 2

0.002 4

0.243 4

0.017 4

0.473 3

0.001 4

Method

Info

avg iou

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

floor

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

wall

window

CSC_LR_SEM

0.612 5

0.739 7

0.794 1

0.687 8

0.564 4

0.850 1

0.347 8

0.590 7

0.587 2

0.521 4

0.945 6

0.358 7

0.140 6

0.522 6

0.496 8

0.627 3

0.725 5

0.598 2

0.850 4

0.792 4

0.508 5

NWSYY

0.670 2

0.745 4

0.784 2

0.780 3

0.611 3

0.829 5

0.463 1

0.692 2

0.585 3

0.542 3

0.948 2

0.448 2

0.219 2

0.554 2

0.828 1

0.702 2

0.758 1

0.573 3

0.844 7

0.825 2

0.674 1

DE-3DLearner LR

0.663 3

0.851 3

0.770 3

0.760 4

0.615 2

0.830 4

0.439 3

0.670 3

0.546 5

0.587 2

0.955 1

0.406 4

0.177 4

0.627 1

0.758 3

0.606 4

0.740 3

0.549 4

0.888 3

0.840 1

0.652 2

Ping-Chung Yu, Cheng Sun, Min Sun: Data Efficient 3D Learner via Knowledge Transferred from 2D Model. ECCV 2022

WS3D_LR_Sem

0.682 1

0.864 2

0.766 4

0.785 1

0.645 1

0.805 7

0.443 2

0.793 1

0.600 1

0.588 1

0.944 7

0.457 1

0.222 1

0.537 3

0.773 2

0.724 1

0.755 2

0.622 1

0.907 2

0.821 3

0.597 3

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

CSG_3DSegNet

0.594 8

0.732 8

0.755 5

0.697 7

0.548 5

0.790 8

0.363 7

0.432 8

0.576 4

0.448 8

0.933 8

0.414 3

0.168 5

0.486 7

0.621 6

0.574 6

0.727 4

0.539 6

0.807 8

0.779 8

0.498 6

Viewpoint_BN_LR_AIR

0.625 4

0.873 1

0.727 6

0.709 5

0.535 7

0.820 6

0.402 6

0.643 5

0.540 6

0.501 6

0.946 4

0.352 8

0.181 3

0.535 4

0.594 7

0.596 5

0.685 8

0.543 5

0.927 1

0.792 4

0.592 4

Scratch_LR_SEM

0.596 7

0.745 4

0.722 7

0.783 2

0.486 8

0.834 3

0.414 5

0.667 4

0.398 8

0.492 7

0.948 2

0.359 6

0.124 7

0.406 8

0.636 5

0.574 6

0.708 6

0.503 8

0.849 6

0.784 6

0.490 7

PointContrast_LR_SEM

0.603 6

0.740 6

0.700 8

0.700 6

0.546 6

0.843 2

0.419 4

0.592 6

0.462 7

0.513 5

0.946 4

0.374 5

0.104 8

0.530 5

0.687 4

0.571 8

0.694 7

0.519 7

0.850 4

0.781 7

0.484 8

Method

Info

avg ap

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

InstTeacher3D

0.473 1

0.778 3

0.635 1

0.331 4

0.322 2

0.732 1

0.089 1

0.502 1

0.200 1

0.350 1

0.440 1

0.507 1

0.489 1

0.493 3

0.327 1

0.655 1

0.469 1

0.924 1

0.268 2

Yizheng Wu, Zhiyu Pan, Kewei Wang, Xingyi Li, Jiahao Cui, Liwen Xiao, Guosheng Lin, Zhiguo Cao: Instance Consistency Regularization for Semi-Supervised 3D Instance Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence

WS3D_LR_Ins

0.440 2

0.704 4

0.533 2

0.485 1

0.335 1

0.659 2

0.025 5

0.460 2

0.156 2

0.315 2

0.412 2

0.254 2

0.383 2

0.638 2

0.278 2

0.653 2

0.431 2

0.869 2

0.330 1

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

Scratch_LR_INS

0.287 6

0.485 5

0.438 3

0.362 3

0.154 5

0.614 4

0.028 2

0.218 6

0.058 5

0.191 4

0.247 6

0.065 4

0.272 6

0.271 6

0.109 6

0.433 6

0.355 4

0.766 5

0.102 5

CSC_LR_INS

0.322 4

0.824 1

0.433 4

0.288 5

0.169 4

0.610 5

0.028 2

0.302 3

0.084 4

0.174 5

0.274 4

0.061 5

0.370 3

0.358 5

0.142 5

0.507 4

0.367 3

0.701 6

0.097 6

PointContrast_LR_INS

0.304 5

0.233 6

0.418 5

0.389 2

0.136 6

0.596 6

0.006 6

0.274 5

0.038 6

0.140 6

0.265 5

0.052 6

0.314 5

0.647 1

0.149 4

0.533 3

0.350 5

0.806 4

0.132 4

TWIST+CSC

0.342 3

0.796 2

0.412 6

0.233 6

0.170 3

0.637 3

0.028 2

0.292 4

0.089 3

0.214 3

0.320 3

0.133 3

0.337 4

0.444 4

0.242 3

0.458 5

0.328 6

0.852 3

0.179 3

Ruihang Chu: TWIST: Two-Way Inter-label Self-Training for Semi-supervised 3D Instance Segmentation. CVPR 2022

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WS3D_LR_OD

0.374 1

0.867 1

0.797 1

0.655 1

0.104 1

0.678 1

0.046 1

0.215 1

0.406 1

0.186 1

0.219 1

0.034 1

0.354 1

0.160 1

0.101 1

0.741 1

0.306 1

0.679 1

0.181 1

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

PointContrast_LR_DET

0.187 3

0.667 2

0.523 2

0.109 3

0.027 3

0.435 2

0.005 3

0.013 2

0.199 2

0.070 2

0.035 3

0.000 4

0.183 2

0.033 2

0.003 4

0.497 3

0.078 3

0.488 2

0.005 3

CSC_LR_DET

0.191 2

0.667 2

0.468 3

0.226 2

0.036 2

0.420 3

0.025 2

0.010 3

0.081 3

0.066 3

0.045 2

0.000 2

0.162 3

0.010 3

0.017 2

0.657 2

0.109 2

0.420 3

0.013 2

Scratch_LR_DET

0.076 4

0.667 2

0.099 4

0.015 4

0.005 4

0.190 4

0.000 4

0.033 4

0.007 4

0.001 4

0.000 3

0.000 4

0.010 4

0.004 3

0.094 4

0.014 4

0.237 4

0.000 4

3D Semantic Label with Limited Annotations Benchmark

3D Semantic Instance with Limited Annotations Benchmark

3D Object Detection with Limited Annotations Benchmark

3D Semantic Label with Limited Reconstructions Benchmark

3D Semantic Instance with Limited Reconstructions Benchmark

3D Object Detection with Limited Reconstructions Benchmark