Benchmark Results - ScanNet Data Efficient

This table lists the benchmark results for the 3D semantic label with limited annotations scenario.

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

Q2E	0.741 1	0.984 1	0.821 2	0.757 4	0.739 1	0.868 2	0.600 1	0.849 1	0.595 6	0.659 1	0.971 2	0.490 2	0.299 2	0.689 4	0.822 3	0.749 1	0.788 4	0.641 1	0.935 2	0.860 1	0.699 2

ActiveST	0.735 2	0.983 2	0.769 4	0.798 1	0.701 2	0.852 5	0.527 2	0.801 2	0.680 1	0.629 2	0.973 1	0.447 10	0.312 1	0.757 1	0.799 4	0.747 2	0.795 3	0.632 2	0.952 1	0.855 2	0.684 3
Gengxin Liu, Oliver van Kaick, Hui Huang, Ruizhen Hu: Active Self-Training for Weakly Supervised 3D Scene Semantic Segmentation.
WS3D_LA_Sem	0.689 4	0.879 3	0.753 6	0.798 1	0.648 8	0.816 9	0.421 10	0.796 3	0.604 5	0.603 3	0.945 10	0.457 9	0.204 9	0.559 10	0.851 2	0.724 3	0.760 7	0.630 3	0.903 7	0.821 5	0.603 8
Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022
DE-3DLearner LA	0.704 3	0.774 7	0.766 5	0.764 3	0.687 4	0.832 7	0.413 11	0.790 4	0.639 2	0.599 4	0.952 4	0.478 6	0.222 8	0.746 2	0.859 1	0.678 4	0.806 2	0.607 6	0.915 5	0.847 3	0.703 1
Ping-Chung Yu, Cheng Sun, Min Sun: Data Efficient 3D Learner via Knowledge Transferred from 2D Model. ECCV 2022
VIBUS	0.684 5	0.848 4	0.752 7	0.708 9	0.691 3	0.861 3	0.474 5	0.770 5	0.611 4	0.538 9	0.951 5	0.478 6	0.275 4	0.676 5	0.671 11	0.649 8	0.788 4	0.610 5	0.869 9	0.808 10	0.657 4
Beiwen Tian,Liyi Luo,Hao Zhao,Guyue Zhou: VIBUS: Data-efficient 3D Scene Parsing with VIewpoint Bottleneck and Uncertainty-Spectrum Modeling. ISPRS Journal of Photogrammetry and Remote Sensing
LE	0.680 7	0.744 9	0.731 9	0.727 6	0.664 5	0.859 4	0.427 9	0.759 6	0.562 7	0.562 6	0.948 7	0.480 4	0.245 6	0.735 3	0.765 6	0.648 10	0.786 6	0.591 7	0.931 3	0.817 7	0.624 7

One-Thing-One-Click	0.670 8	0.734 10	0.815 3	0.661 13	0.644 9	0.841 6	0.509 3	0.741 7	0.479 12	0.548 7	0.968 3	0.461 8	0.251 5	0.664 6	0.754 7	0.656 7	0.744 10	0.541 11	0.917 4	0.844 4	0.625 6
Zhengzhe Liu, Xiaojuan Qi, Chi-Wing Fu: One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation. CVPR 2021
Viewpoint_BN_LA_AIR	0.650 9	0.778 6	0.731 9	0.688 11	0.617 11	0.812 11	0.446 7	0.739 8	0.618 3	0.540 8	0.945 10	0.415 11	0.204 9	0.623 7	0.676 10	0.594 11	0.744 10	0.576 8	0.868 10	0.811 8	0.582 10
Liyi Luo, Beiwen Tian, Hao Zhao, Guyue Zhou: Pointly-supervised 3D Scene Parsing with Viewpoint Bottleneck.
GaIA	0.682 6	0.731 11	0.846 1	0.713 8	0.657 6	0.869 1	0.475 4	0.705 9	0.452 13	0.569 5	0.951 5	0.563 1	0.290 3	0.544 11	0.799 4	0.677 5	0.810 1	0.618 4	0.900 8	0.821 5	0.642 5
Min Seok Lee, Seok Woo Yang, and Sung Won Han: GaIA: Graphical Information gain based Attention Network for Weakly Supervised 3D Point Cloud Semantic Segmentation. WACV 2023
Scratch_LA_SEM	0.621 12	0.802 5	0.715 11	0.687 12	0.570 12	0.800 12	0.386 12	0.703 10	0.486 11	0.514 12	0.946 8	0.390 12	0.181 12	0.620 8	0.670 12	0.487 13	0.746 9	0.539 12	0.804 12	0.798 12	0.580 11

CSC_LA_SEM	0.644 10	0.761 8	0.707 12	0.703 10	0.642 10	0.813 10	0.436 8	0.659 11	0.502 9	0.516 11	0.945 10	0.487 3	0.238 7	0.538 12	0.678 9	0.659 6	0.739 12	0.568 10	0.915 5	0.811 8	0.566 12

PointContrast_LA_SEM	0.636 11	0.694 12	0.738 8	0.731 5	0.653 7	0.817 8	0.467 6	0.651 12	0.517 8	0.522 10	0.946 8	0.479 5	0.198 11	0.575 9	0.526 13	0.649 8	0.747 8	0.569 9	0.845 11	0.803 11	0.600 9

SQN_LA	0.576 13	0.674 13	0.670 13	0.722 7	0.454 13	0.790 13	0.342 13	0.622 13	0.487 10	0.427 13	0.933 13	0.357 13	0.157 13	0.452 13	0.721 8	0.492 12	0.696 13	0.487 13	0.790 13	0.748 13	0.507 13

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WS3D_LA_Ins

0.579 1

0.667 2

0.758 1

0.655 3

0.498 1

0.750 1

0.042 4

0.779 1

0.163 2

0.429 1

0.472 2

0.408 1

0.625 1

1.000 1

0.469 3

0.680 3

0.612 1

1.000 1

0.404 1

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

Box2Mask_LA

0.565 2

1.000 1

0.719 4

0.649 4

0.407 2

0.742 2

0.135 2

0.598 2

0.298 1

0.368 2

0.509 1

0.343 2

0.503 2

0.981 3

0.347 4

0.755 1

0.521 2

0.983 4

0.315 2

Julian Chibane, Francis Engelmann, Tuan Anh Tran, Gerard Pons-Moll: Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation Using Bounding Boxes. ECCV 2022

PointContrast_LA_INS

0.456 4

0.667 2

0.696 5

0.687 2

0.325 4

0.543 3

0.072 3

0.556 3

0.095 4

0.270 5

0.315 4

0.021 5

0.318 5

1.000 1

0.562 2

0.585 4

0.319 5

0.994 3

0.188 5

CSC_LA_INS

0.460 3

0.528 4

0.754 3

0.709 1

0.314 5

0.542 4

0.139 1

0.472 4

0.120 3

0.302 4

0.373 3

0.090 4

0.389 4

0.714 4

0.631 1

0.531 5

0.467 3

1.000 1

0.203 4

Scratch_LA_INS

0.418 5

0.528 4

0.758 2

0.475 5

0.377 3

0.484 5

0.008 5

0.450 5

0.029 5

0.311 3

0.260 5

0.096 3

0.500 3

0.714 4

0.309 5

0.681 2

0.370 4

0.938 5

0.231 3

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WS3D_LA_OD

0.341 1

1.000 1

0.629 1

0.426 1

0.070 1

0.608 1

0.063 1

0.176 1

0.503 1

0.132 1

0.084 1

0.001 1

0.337 1

0.220 1

0.103 1

0.628 1

0.282 1

0.739 1

0.131 1

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

CSC_LA_DET

0.135 2

0.444 3

0.336 2

0.029 4

0.001 4

0.356 2

0.008 2

0.000 2

0.011 4

0.045 3

0.010 4

0.000 2

0.032 2

0.011 3

0.043 2

0.458 3

0.028 3

0.602 2

0.010 3

PointContrast_LA_DET

0.135 2

0.667 2

0.296 3

0.145 2

0.002 3

0.330 3

0.000 4

0.000 3

0.050 3

0.049 2

0.023 2

0.000 3

0.002 4

0.006 4

0.034 3

0.472 2

0.052 2

0.285 4

0.011 2

Scratch_LA_DET

0.098 4

0.167 4

0.253 4

0.074 3

0.002 2

0.257 4

0.004 3

0.000 4

0.080 2

0.038 4

0.013 3

0.000 3

0.006 3

0.143 2

0.002 4

0.243 4

0.017 4

0.473 3

0.001 4

Method

Info

avg iou

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

floor

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

wall

window

WS3D_LR_Sem

0.682 1

0.864 2

0.766 4

0.785 1

0.645 1

0.805 7

0.443 2

0.793 1

0.600 1

0.588 1

0.944 7

0.457 1

0.222 1

0.537 3

0.773 2

0.724 1

0.755 2

0.622 1

0.907 2

0.821 3

0.597 3

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

NWSYY

0.670 2

0.745 4

0.784 2

0.780 3

0.611 3

0.829 5

0.463 1

0.692 2

0.585 3

0.542 3

0.948 2

0.448 2

0.219 2

0.554 2

0.828 1

0.702 2

0.758 1

0.573 3

0.844 7

0.825 2

0.674 1

DE-3DLearner LR

0.663 3

0.851 3

0.770 3

0.760 4

0.615 2

0.830 4

0.439 3

0.670 3

0.546 5

0.587 2

0.955 1

0.406 4

0.177 4

0.627 1

0.758 3

0.606 4

0.740 3

0.549 4

0.888 3

0.840 1

0.652 2

Ping-Chung Yu, Cheng Sun, Min Sun: Data Efficient 3D Learner via Knowledge Transferred from 2D Model. ECCV 2022

Scratch_LR_SEM

0.596 7

0.745 4

0.722 7

0.783 2

0.486 8

0.834 3

0.414 5

0.667 4

0.398 8

0.492 7

0.948 2

0.359 6

0.124 7

0.406 8

0.636 5

0.574 6

0.708 6

0.503 8

0.849 6

0.784 6

0.490 7

Viewpoint_BN_LR_AIR

0.625 4

0.873 1

0.727 6

0.709 5

0.535 7

0.820 6

0.402 6

0.643 5

0.540 6

0.501 6

0.946 4

0.352 8

0.181 3

0.535 4

0.594 7

0.596 5

0.685 8

0.543 5

0.927 1

0.792 4

0.592 4

PointContrast_LR_SEM

0.603 6

0.740 6

0.700 8

0.700 6

0.546 6

0.843 2

0.419 4

0.592 6

0.462 7

0.513 5

0.946 4

0.374 5

0.104 8

0.530 5

0.687 4

0.571 8

0.694 7

0.519 7

0.850 4

0.781 7

0.484 8

CSC_LR_SEM

0.612 5

0.739 7

0.794 1

0.687 8

0.564 4

0.850 1

0.347 8

0.590 7

0.587 2

0.521 4

0.945 6

0.358 7

0.140 6

0.522 6

0.496 8

0.627 3

0.725 5

0.598 2

0.850 4

0.792 4

0.508 5

CSG_3DSegNet

0.594 8

0.732 8

0.755 5

0.697 7

0.548 5

0.790 8

0.363 7

0.432 8

0.576 4

0.448 8

0.933 8

0.414 3

0.168 5

0.486 7

0.621 6

0.574 6

0.727 4

0.539 6

0.807 8

0.779 8

0.498 6

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

InstTeacher3D

0.646 2

1.000 1

0.764 5

0.699 3

0.563 2

0.849 1

0.177 1

0.694 1

0.449 1

0.509 2

0.561 2

0.575 1

0.550 2

0.714 3

0.595 2

0.777 4

0.650 1

0.997 4

0.513 2

Yizheng Wu, Zhiyu Pan, Kewei Wang, Xingyi Li, Jiahao Cui, Liwen Xiao, Guosheng Lin, Zhiguo Cao: Instance Consistency Regularization for Semi-Supervised 3D Instance Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence

WS3D_LR_Ins

0.656 1

1.000 1

0.766 4

0.736 1

0.656 1

0.793 3

0.091 2

0.561 2

0.389 4

0.533 1

0.582 1

0.531 2

0.592 1

0.981 1

0.622 1

0.782 2

0.638 2

0.997 4

0.558 1

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

Scratch_LR_INS

0.473 6

0.528 5

0.773 1

0.632 4

0.391 5

0.784 5

0.050 3

0.515 3

0.271 5

0.392 4

0.400 6

0.123 5

0.486 5

0.387 6

0.322 6

0.627 6

0.573 3

1.000 1

0.268 5

CSC_LR_INS

0.529 4

1.000 1

0.773 1

0.704 2

0.414 4

0.786 4

0.050 3

0.412 4

0.394 3

0.376 5

0.442 4

0.179 4

0.542 3

0.539 5

0.394 5

0.793 1

0.564 4

0.944 6

0.217 6

TWIST+CSC

0.550 3

1.000 1

0.758 6

0.570 6

0.462 3

0.797 2

0.050 3

0.389 5

0.436 2

0.433 3

0.484 3

0.253 3

0.491 4

0.571 4

0.538 3

0.760 5

0.561 5

1.000 1

0.354 3

Ruihang Chu: TWIST: Two-Way Inter-label Self-Training for Semi-supervised 3D Instance Segmentation. CVPR 2022

PointContrast_LR_INS

0.488 5

0.472 6

0.773 1

0.594 5

0.374 6

0.774 6

0.013 6

0.353 6

0.252 6

0.327 6

0.416 5

0.087 6

0.435 6

0.857 2

0.444 4

0.779 3

0.560 6

1.000 1

0.282 4

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WS3D_LR_OD

0.374 1

0.867 1

0.797 1

0.655 1

0.104 1

0.678 1

0.046 1

0.215 1

0.406 1

0.186 1

0.219 1

0.034 1

0.354 1

0.160 1

0.101 1

0.741 1

0.306 1

0.679 1

0.181 1

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

PointContrast_LR_DET

0.187 3

0.667 2

0.523 2

0.109 3

0.027 3

0.435 2

0.005 3

0.013 2

0.199 2

0.070 2

0.035 3

0.000 4

0.183 2

0.033 2

0.003 4

0.497 3

0.078 3

0.488 2

0.005 3

CSC_LR_DET

0.191 2

0.667 2

0.468 3

0.226 2

0.036 2

0.420 3

0.025 2

0.010 3

0.081 3

0.066 3

0.045 2

0.000 2

0.162 3

0.010 3

0.017 2

0.657 2

0.109 2

0.420 3

0.013 2

Scratch_LR_DET

0.076 4

0.667 2

0.099 4

0.015 4

0.005 4

0.190 4

0.000 4

0.033 4

0.007 4

0.001 4

0.000 3

0.000 4

0.010 4

0.004 3

0.094 4

0.014 4

0.237 4

0.000 4

3D Semantic Label with Limited Annotations Benchmark

3D Semantic Instance with Limited Annotations Benchmark

3D Object Detection with Limited Annotations Benchmark

3D Semantic Label with Limited Reconstructions Benchmark

3D Semantic Instance with Limited Reconstructions Benchmark

3D Object Detection with Limited Reconstructions Benchmark