Benchmark Results - ScanNet Data Efficient

This table lists the benchmark results for the 3D semantic label with limited annotations scenario.

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

ActiveST	0.748 1	0.984 1	0.804 3	0.759 5	0.720 2	0.849 5	0.516 2	0.791 3	0.670 1	0.654 2	0.974 1	0.495 5	0.382 1	0.811 1	0.828 5	0.787 1	0.780 6	0.640 2	0.952 1	0.861 3	0.701 1
Gengxin Liu, Oliver van Kaick, Hui Huang, Ruizhen Hu: Active Self-Training for Weakly Supervised 3D Scene Semantic Segmentation.
Q2E	0.743 2	0.984 1	0.803 4	0.770 1	0.725 1	0.881 1	0.572 1	0.806 2	0.663 2	0.665 1	0.972 2	0.506 3	0.305 2	0.652 6	0.829 4	0.761 2	0.809 2	0.660 1	0.951 2	0.862 2	0.682 2

VIBUS	0.691 6	0.860 5	0.731 12	0.738 10	0.672 7	0.860 2	0.470 5	0.766 5	0.625 3	0.547 11	0.949 5	0.491 6	0.255 5	0.693 4	0.715 10	0.712 4	0.778 7	0.597 8	0.911 5	0.816 9	0.635 7
Beiwen Tian,Liyi Luo,Hao Zhao,Guyue Zhou: VIBUS: Data-efficient 3D Scene Parsing with VIewpoint Bottleneck and Uncertainty-Spectrum Modeling. ISPRS Journal of Photogrammetry and Remote Sensing
DE-3DLearner LA	0.709 3	0.877 4	0.772 8	0.744 9	0.694 3	0.836 7	0.453 6	0.787 4	0.623 4	0.598 4	0.953 4	0.490 7	0.216 11	0.682 5	0.879 1	0.727 3	0.802 3	0.604 5	0.922 3	0.845 4	0.676 3
Ping-Chung Yu, Cheng Sun, Min Sun: Data Efficient 3D Learner via Knowledge Transferred from 2D Model. ECCV 2022
WS3D_LA_Sem	0.694 4	0.895 3	0.743 10	0.767 2	0.675 6	0.826 10	0.496 3	0.817 1	0.612 5	0.613 3	0.947 10	0.460 9	0.254 6	0.558 11	0.811 7	0.710 5	0.776 8	0.616 3	0.874 11	0.822 6	0.603 12
Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022
Viewpoint_BN_LA_AIR	0.669 9	0.847 8	0.732 11	0.724 11	0.613 12	0.827 9	0.443 7	0.742 6	0.562 6	0.551 10	0.947 10	0.441 12	0.218 10	0.650 7	0.753 9	0.621 9	0.765 10	0.601 6	0.905 8	0.814 12	0.618 9
Liyi Luo, Beiwen Tian, Hao Zhao, Guyue Zhou: Pointly-supervised 3D Scene Parsing with Viewpoint Bottleneck.
One-Thing-One-Click	0.694 4	0.760 9	0.815 2	0.706 13	0.684 5	0.840 6	0.492 4	0.701 9	0.557 7	0.596 5	0.972 2	0.497 4	0.281 4	0.709 2	0.757 8	0.689 6	0.789 4	0.600 7	0.907 7	0.864 1	0.671 4
Zhengzhe Liu, Xiaojuan Qi, Chi-Wing Fu: One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation. CVPR 2021
CSC_LA_SEM	0.665 10	0.857 6	0.756 9	0.763 4	0.647 9	0.852 4	0.432 9	0.684 12	0.543 8	0.514 12	0.948 6	0.469 8	0.179 12	0.599 9	0.702 11	0.620 10	0.789 4	0.614 4	0.911 5	0.815 11	0.607 11

LE	0.688 7	0.856 7	0.779 6	0.754 7	0.687 4	0.834 8	0.438 8	0.732 7	0.536 9	0.577 6	0.948 6	0.508 2	0.248 7	0.699 3	0.831 3	0.636 8	0.752 11	0.586 9	0.895 9	0.821 7	0.643 6

GaIA	0.685 8	0.759 10	0.834 1	0.759 5	0.650 8	0.859 3	0.427 10	0.694 10	0.524 10	0.575 7	0.948 6	0.537 1	0.304 3	0.534 12	0.853 2	0.678 7	0.820 1	0.581 10	0.914 4	0.828 5	0.626 8
Min Seok Lee, Seok Woo Yang, and Sung Won Han: GaIA: Graphical Information gain based Attention Network for Weakly Supervised 3D Point Cloud Semantic Segmentation. WACV 2023
SQN_LA	0.598 13	0.741 11	0.681 13	0.766 3	0.482 13	0.805 12	0.389 13	0.658 13	0.499 11	0.437 13	0.936 13	0.386 13	0.243 8	0.422 13	0.663 13	0.552 13	0.700 13	0.519 13	0.809 13	0.750 13	0.515 13

Scratch_LA_SEM	0.643 12	0.699 13	0.793 5	0.718 12	0.636 10	0.816 11	0.411 11	0.707 8	0.490 12	0.574 8	0.948 6	0.448 10	0.173 13	0.559 10	0.689 12	0.604 11	0.722 12	0.556 11	0.853 12	0.820 8	0.651 5

PointContrast_LA_SEM	0.653 11	0.717 12	0.775 7	0.754 7	0.626 11	0.804 13	0.391 12	0.689 11	0.485 13	0.572 9	0.945 12	0.448 10	0.232 9	0.603 8	0.813 6	0.591 12	0.775 9	0.537 12	0.885 10	0.816 9	0.608 10

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WS3D_LA_Ins

0.668 1

1.000 1

0.786 1

0.845 1

0.647 1

0.777 1

0.137 1

0.618 2

0.432 1

0.506 1

0.564 2

0.561 1

0.546 3

1.000 1

0.562 1

0.786 2

0.675 1

1.000 1

0.577 1

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

Box2Mask_LA

0.592 2

1.000 1

0.619 5

0.820 2

0.471 2

0.773 2

0.104 2

0.618 1

0.377 2

0.409 2

0.591 1

0.364 2

0.515 4

0.857 2

0.443 3

0.782 3

0.524 4

1.000 1

0.382 2

Julian Chibane, Francis Engelmann, Tuan Anh Tran, Gerard Pons-Moll: Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation Using Bounding Boxes. ECCV 2022

PointContrast_LA_INS

0.471 4

0.667 3

0.773 2

0.646 5

0.330 5

0.490 4

0.032 4

0.470 5

0.122 3

0.368 4

0.349 5

0.048 5

0.592 1

0.614 5

0.338 5

0.789 1

0.536 3

0.997 4

0.316 3

Scratch_LA_INS

0.464 5

0.222 5

0.763 4

0.714 4

0.464 3

0.469 5

0.027 5

0.591 3

0.079 4

0.303 5

0.395 4

0.075 4

0.550 2

0.777 4

0.458 2

0.712 5

0.512 5

0.997 4

0.243 5

CSC_LA_INS

0.494 3

0.528 4

0.766 3

0.800 3

0.408 4

0.611 3

0.045 3

0.547 4

0.055 5

0.368 3

0.429 3

0.126 3

0.389 5

0.819 3

0.421 4

0.775 4

0.550 2

1.000 1

0.253 4

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WS3D_LA_OD

0.341 1

1.000 1

0.629 1

0.426 1

0.070 1

0.608 1

0.063 1

0.176 1

0.503 1

0.132 1

0.084 1

0.001 1

0.337 1

0.220 1

0.103 1

0.628 1

0.282 1

0.739 1

0.131 1

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

Scratch_LA_DET

0.098 4

0.167 4

0.253 4

0.074 3

0.002 2

0.257 4

0.004 3

0.000 4

0.080 2

0.038 4

0.013 3

0.000 3

0.006 3

0.143 2

0.002 4

0.243 4

0.017 4

0.473 3

0.001 4

PointContrast_LA_DET

0.135 2

0.667 2

0.296 3

0.145 2

0.002 3

0.330 3

0.000 4

0.000 3

0.050 3

0.049 2

0.023 2

0.000 3

0.002 4

0.006 4

0.034 3

0.472 2

0.052 2

0.285 4

0.011 2

CSC_LA_DET

0.135 2

0.444 3

0.336 2

0.029 4

0.001 4

0.356 2

0.008 2

0.000 2

0.011 4

0.045 3

0.010 4

0.000 2

0.032 2

0.011 3

0.043 2

0.458 3

0.028 3

0.602 2

0.010 3

Method

Info

avg iou

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

floor

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

wall

window

WS3D_LR_Sem

0.685 1

0.871 1

0.769 1

0.779 1

0.647 1

0.806 1

0.453 1

0.802 1

0.577 1

0.588 1

0.945 1

0.460 1

0.223 1

0.539 1

0.793 1

0.732 1

0.766 1

0.614 1

0.904 1

0.823 1

0.604 1

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

Viewpoint_BN_LR_AIR

0.256 6

0.000 2

0.479 5

0.377 2

0.204 4

0.551 5

0.205 4

0.219 7

0.235 2

0.224 7

0.903 7

0.092 4

0.088 6

0.122 3

0.000 2

0.003 7

0.354 7

0.354 3

0.000 3

0.676 7

0.034 6

CSC_LR_SEM

0.270 3

0.000 2

0.528 3

0.331 5

0.139 7

0.535 7

0.118 8

0.326 4

0.222 3

0.292 3

0.921 4

0.089 6

0.163 4

0.129 2

0.000 2

0.131 3

0.463 3

0.278 8

0.000 3

0.699 3

0.033 7

DE-3DLearner LR

0.263 4

0.000 2

0.547 2

0.235 8

0.184 5

0.566 3

0.165 5

0.249 6

0.196 4

0.309 2

0.938 2

0.070 8

0.186 3

0.069 7

0.000 2

0.000 8

0.368 6

0.356 2

0.000 3

0.698 4

0.118 4

Ping-Chung Yu, Cheng Sun, Min Sun: Data Efficient 3D Learner via Knowledge Transferred from 2D Model. ECCV 2022

Scratch_LR_SEM

0.251 8

0.000 2

0.457 6

0.238 7

0.205 3

0.528 8

0.123 7

0.419 2

0.195 5

0.246 6

0.905 6

0.086 7

0.048 8

0.103 4

0.000 2

0.132 2

0.331 8

0.308 6

0.000 3

0.675 8

0.015 8

PointContrast_LR_SEM

0.253 7

0.000 2

0.412 7

0.347 4

0.137 8

0.564 4

0.140 6

0.361 3

0.187 6

0.249 5

0.914 5

0.092 4

0.055 7

0.102 5

0.000 2

0.048 6

0.392 5

0.302 7

0.000 3

0.697 5

0.056 5

CSG_3DSegNet

0.258 5

0.000 2

0.335 8

0.368 3

0.169 6

0.549 6

0.229 2

0.158 8

0.182 7

0.208 8

0.898 8

0.105 3

0.190 2

0.093 6

0.000 2

0.093 4

0.448 4

0.342 5

0.000 3

0.679 6

0.119 3

NWSYY

0.286 2

0.000 2

0.515 4

0.322 6

0.247 2

0.618 2

0.219 3

0.304 5

0.174 8

0.268 4

0.926 3

0.117 2

0.162 5

0.065 8

0.000 2

0.079 5

0.514 2

0.345 4

0.028 2

0.701 2

0.125 2

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WS3D_LR_Ins

0.565 1

0.667 1

0.755 1

0.614 1

0.488 1

0.767 1

0.024 2

0.710 1

0.392 1

0.370 1

0.496 1

0.298 1

0.515 1

0.857 1

0.496 1

0.667 1

0.596 1

0.990 1

0.474 1

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

CSC_LR_INS

0.117 5

0.000 2

0.492 5

0.040 4

0.032 6

0.524 5

0.000 3

0.174 5

0.021 2

0.139 4

0.096 4

0.112 3

0.039 4

0.000 2

0.018 3

0.258 5

0.150 6

0.000 2

0.001 3

Scratch_LR_INS

0.101 6

0.000 2

0.488 6

0.000 5

0.035 5

0.491 6

0.000 3

0.160 6

0.009 3

0.111 5

0.025 6

0.002 6

0.009 5

0.000 2

0.001 4

0.265 4

0.219 3

0.000 2

0.001 2

TWIST+CSC

0.186 2

0.000 2

0.545 2

0.153 2

0.148 2

0.667 3

0.050 1

0.420 2

0.007 4

0.210 3

0.175 2

0.016 5

0.218 2

0.000 2

0.000 5

0.511 2

0.227 2

0.000 2

0.000 5

Ruihang Chu: TWIST: Two-Way Inter-label Self-Training for Semi-supervised 3D Instance Segmentation. CVPR 2022

InstTeacher3D

0.162 3

0.000 2

0.500 4

0.000 5

0.120 3

0.738 2

0.000 3

0.404 3

0.005 5

0.291 2

0.155 3

0.153 2

0.008 6

0.000 2

0.059 2

0.289 3

0.192 4

0.000 2

0.000 4

Yizheng Wu, Zhiyu Pan, Kewei Wang, Xingyi Li, Jiahao Cui, Liwen Xiao, Guosheng Lin, Zhiguo Cao: Instance Consistency Regularization for Semi-Supervised 3D Instance Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence

PointContrast_LR_INS

0.119 4

0.000 2

0.534 3

0.041 3

0.047 4

0.560 4

0.000 3

0.307 4

0.000 6

0.083 6

0.071 5

0.068 4

0.064 3

0.000 2

0.000 5

0.186 6

0.176 5

0.000 2

0.000 5

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WS3D_LR_OD

0.374 1

0.867 1

0.797 1

0.655 1

0.104 1

0.678 1

0.046 1

0.215 1

0.406 1

0.186 1

0.219 1

0.034 1

0.354 1

0.160 1

0.101 1

0.741 1

0.306 1

0.679 1

0.181 1

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

PointContrast_LR_DET

0.187 3

0.667 2

0.523 2

0.109 3

0.027 3

0.435 2

0.005 3

0.013 2

0.199 2

0.070 2

0.035 3

0.000 4

0.183 2

0.033 2

0.003 4

0.497 3

0.078 3

0.488 2

0.005 3

CSC_LR_DET

0.191 2

0.667 2

0.468 3

0.226 2

0.036 2

0.420 3

0.025 2

0.010 3

0.081 3

0.066 3

0.045 2

0.000 2

0.162 3

0.010 3

0.017 2

0.657 2

0.109 2

0.420 3

0.013 2

Scratch_LR_DET

0.076 4

0.667 2

0.099 4

0.015 4

0.005 4

0.190 4

0.000 4

0.033 4

0.007 4

0.001 4

0.000 3

0.000 4

0.010 4

0.004 3

0.094 4

0.014 4

0.237 4

0.000 4

3D Semantic Label with Limited Annotations Benchmark

3D Semantic Instance with Limited Annotations Benchmark

3D Object Detection with Limited Annotations Benchmark

3D Semantic Label with Limited Reconstructions Benchmark

3D Semantic Instance with Limited Reconstructions Benchmark

3D Object Detection with Limited Reconstructions Benchmark