Benchmark Results - ScanNet Data Efficient

This table lists the benchmark results for the 3D semantic label with limited annotations scenario.

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

Q2E	0.739 1	0.984 1	0.797 1	0.761 2	0.716 1	0.884 1	0.588 1	0.843 1	0.589 3	0.656 1	0.971 2	0.487 2	0.271 2	0.772 1	0.807 2	0.726 2	0.795 2	0.630 1	0.945 2	0.856 1	0.693 1

ActiveST	0.725 2	0.980 2	0.764 4	0.753 3	0.699 2	0.863 2	0.521 2	0.773 2	0.671 1	0.625 2	0.974 1	0.456 6	0.182 9	0.721 2	0.874 1	0.746 1	0.808 1	0.628 2	0.960 1	0.846 2	0.664 3
Gengxin Liu, Oliver van Kaick, Hui Huang, Ruizhen Hu: Active Self-Training for Weakly Supervised 3D Scene Semantic Segmentation.
WS3D_LA_Sem	0.670 4	0.842 6	0.732 8	0.825 1	0.657 4	0.794 10	0.506 3	0.762 5	0.584 4	0.553 5	0.947 7	0.451 8	0.219 5	0.585 6	0.652 8	0.670 3	0.791 3	0.570 5	0.857 11	0.816 5	0.579 5
Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022
Viewpoint_BN_LA_AIR	0.623 9	0.812 8	0.743 6	0.654 11	0.579 11	0.800 9	0.462 4	0.713 7	0.533 6	0.516 9	0.944 8	0.434 9	0.215 7	0.437 11	0.521 12	0.601 7	0.720 8	0.563 6	0.884 6	0.800 7	0.534 10
Liyi Luo, Beiwen Tian, Hao Zhao, Guyue Zhou: Pointly-supervised 3D Scene Parsing with Viewpoint Bottleneck.
One-Thing-One-Click	0.642 8	0.725 10	0.735 7	0.717 6	0.635 6	0.829 6	0.457 5	0.639 11	0.421 11	0.552 6	0.967 3	0.460 5	0.240 3	0.558 7	0.788 3	0.621 6	0.720 8	0.477 11	0.915 5	0.842 3	0.539 8
Zhengzhe Liu, Xiaojuan Qi, Chi-Wing Fu: One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation. CVPR 2021
LE	0.652 5	0.816 7	0.760 5	0.747 4	0.648 5	0.807 8	0.455 6	0.765 4	0.517 7	0.523 7	0.941 11	0.452 7	0.190 8	0.586 5	0.691 7	0.525 11	0.762 6	0.552 8	0.930 4	0.795 9	0.580 4

DE-3DLearner LA	0.695 3	0.897 3	0.784 2	0.728 5	0.697 3	0.846 4	0.441 7	0.770 3	0.615 2	0.585 3	0.951 4	0.504 1	0.232 4	0.672 3	0.760 4	0.655 4	0.772 5	0.599 3	0.877 7	0.834 4	0.678 2
Ping-Chung Yu, Cheng Sun, Min Sun: Data Efficient 3D Learner via Knowledge Transferred from 2D Model. ECCV 2022
CSC_LA_SEM	0.612 11	0.747 9	0.731 9	0.679 8	0.603 8	0.815 7	0.400 8	0.648 10	0.453 9	0.481 11	0.944 8	0.421 10	0.173 10	0.504 8	0.623 10	0.588 10	0.690 12	0.545 9	0.877 7	0.778 11	0.541 7

GaIA	0.643 7	0.704 11	0.776 3	0.670 10	0.597 9	0.842 5	0.382 9	0.688 9	0.413 12	0.556 4	0.950 5	0.471 3	0.334 1	0.478 10	0.728 6	0.640 5	0.787 4	0.557 7	0.937 3	0.812 6	0.531 11
Min Seok Lee, Seok Woo Yang, and Sung Won Han: GaIA: Graphical Information gain based Attention Network for Weakly Supervised 3D Point Cloud Semantic Segmentation. WACV 2023
SQN_LA	0.542 12	0.568 13	0.674 13	0.618 13	0.462 12	0.772 12	0.351 10	0.567 12	0.443 10	0.378 13	0.931 13	0.335 12	0.173 10	0.392 12	0.623 10	0.455 13	0.688 13	0.466 12	0.769 13	0.720 13	0.450 12

PointContrast_LA_SEM	0.614 10	0.844 5	0.731 9	0.681 7	0.590 10	0.791 11	0.348 11	0.689 8	0.503 8	0.502 10	0.942 10	0.361 11	0.154 12	0.484 9	0.624 9	0.591 8	0.708 10	0.524 10	0.874 10	0.793 10	0.536 9

Scratch_LA_SEM	0.524 13	0.640 12	0.690 12	0.636 12	0.442 13	0.756 13	0.326 12	0.544 13	0.365 13	0.396 12	0.940 12	0.284 13	0.085 13	0.333 13	0.479 13	0.502 12	0.696 11	0.453 13	0.785 12	0.746 12	0.372 13

VIBUS	0.651 6	0.868 4	0.728 11	0.675 9	0.624 7	0.861 3	0.247 13	0.734 6	0.561 5	0.520 8	0.948 6	0.464 4	0.216 6	0.670 4	0.742 5	0.589 9	0.746 7	0.579 4	0.877 7	0.800 7	0.568 6
Beiwen Tian,Liyi Luo,Hao Zhao,Guyue Zhou: VIBUS: Data-efficient 3D Scene Parsing with VIewpoint Bottleneck and Uncertainty-Spectrum Modeling. ISPRS Journal of Photogrammetry and Remote Sensing

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

CSC_LA_INS

0.414 3

0.667 2

0.563 5

0.417 3

0.395 2

0.546 3

0.085 1

0.551 2

0.014 4

0.213 4

0.262 4

0.043 4

0.364 4

0.571 3

0.532 3

0.629 2

0.392 2

0.997 1

0.215 3

Box2Mask_LA

0.498 2

1.000 1

0.658 2

0.675 1

0.264 4

0.675 2

0.065 2

0.446 4

0.261 2

0.308 3

0.441 1

0.304 2

0.488 2

1.000 1

0.135 5

0.576 4

0.335 3

0.921 5

0.410 1

Julian Chibane, Francis Engelmann, Tuan Anh Tran, Gerard Pons-Moll: Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation Using Bounding Boxes. ECCV 2022

WS3D_LA_Ins

0.548 1

0.667 2

0.766 1

0.540 2

0.446 1

0.754 1

0.018 3

0.628 1

0.370 1

0.419 1

0.436 2

0.407 1

0.450 3

0.714 2

0.533 2

0.724 1

0.596 1

0.991 2

0.407 2

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

PointContrast_LA_INS

0.400 4

0.667 2

0.637 4

0.411 4

0.335 3

0.534 4

0.018 4

0.526 3

0.031 3

0.318 2

0.327 3

0.056 3

0.488 1

0.260 5

0.555 1

0.602 3

0.278 5

0.957 3

0.196 4

Scratch_LA_INS

0.311 5

0.667 2

0.651 3

0.411 5

0.262 5

0.486 5

0.006 5

0.257 5

0.006 5

0.134 5

0.168 5

0.000 5

0.060 5

0.343 4

0.296 4

0.459 5

0.321 4

0.944 4

0.126 5

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WS3D_LA_OD

0.341 1

1.000 1

0.629 1

0.426 1

0.070 1

0.608 1

0.063 1

0.176 1

0.503 1

0.132 1

0.084 1

0.001 1

0.337 1

0.220 1

0.103 1

0.628 1

0.282 1

0.739 1

0.131 1

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

CSC_LA_DET

0.135 2

0.444 3

0.336 2

0.029 4

0.001 4

0.356 2

0.008 2

0.000 2

0.011 4

0.045 3

0.010 4

0.000 2

0.032 2

0.011 3

0.043 2

0.458 3

0.028 3

0.602 2

0.010 3

Scratch_LA_DET

0.098 4

0.167 4

0.253 4

0.074 3

0.002 2

0.257 4

0.004 3

0.000 4

0.080 2

0.038 4

0.013 3

0.000 3

0.006 3

0.143 2

0.002 4

0.243 4

0.017 4

0.473 3

0.001 4

PointContrast_LA_DET

0.135 2

0.667 2

0.296 3

0.145 2

0.002 3

0.330 3

0.000 4

0.000 3

0.050 3

0.049 2

0.023 2

0.000 3

0.002 4

0.006 4

0.034 3

0.472 2

0.052 2

0.285 4

0.011 2

Method

Info

avg iou

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

floor

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

wall

window

WS3D_LR_Sem

0.685 1

0.871 1

0.769 1

0.779 1

0.647 1

0.806 1

0.453 1

0.802 1

0.577 1

0.588 1

0.945 1

0.460 1

0.223 1

0.539 1

0.793 1

0.732 1

0.766 1

0.614 1

0.904 1

0.823 1

0.604 1

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

CSG_3DSegNet

0.258 5

0.000 2

0.335 8

0.368 3

0.169 6

0.549 6

0.229 2

0.158 8

0.182 7

0.208 8

0.898 8

0.105 3

0.190 2

0.093 6

0.000 2

0.093 4

0.448 4

0.342 5

0.000 3

0.679 6

0.119 3

NWSYY

0.286 2

0.000 2

0.515 4

0.322 6

0.247 2

0.618 2

0.219 3

0.304 5

0.174 8

0.268 4

0.926 3

0.117 2

0.162 5

0.065 8

0.000 2

0.079 5

0.514 2

0.345 4

0.028 2

0.701 2

0.125 2

Viewpoint_BN_LR_AIR

0.256 6

0.000 2

0.479 5

0.377 2

0.204 4

0.551 5

0.205 4

0.219 7

0.235 2

0.224 7

0.903 7

0.092 4

0.088 6

0.122 3

0.000 2

0.003 7

0.354 7

0.354 3

0.000 3

0.676 7

0.034 6

DE-3DLearner LR

0.263 4

0.000 2

0.547 2

0.235 8

0.184 5

0.566 3

0.165 5

0.249 6

0.196 4

0.309 2

0.938 2

0.070 8

0.186 3

0.069 7

0.000 2

0.000 8

0.368 6

0.356 2

0.000 3

0.698 4

0.118 4

Ping-Chung Yu, Cheng Sun, Min Sun: Data Efficient 3D Learner via Knowledge Transferred from 2D Model. ECCV 2022

PointContrast_LR_SEM

0.253 7

0.000 2

0.412 7

0.347 4

0.137 8

0.564 4

0.140 6

0.361 3

0.187 6

0.249 5

0.914 5

0.092 4

0.055 7

0.102 5

0.000 2

0.048 6

0.392 5

0.302 7

0.000 3

0.697 5

0.056 5

Scratch_LR_SEM

0.251 8

0.000 2

0.457 6

0.238 7

0.205 3

0.528 8

0.123 7

0.419 2

0.195 5

0.246 6

0.905 6

0.086 7

0.048 8

0.103 4

0.000 2

0.132 2

0.331 8

0.308 6

0.000 3

0.675 8

0.015 8

CSC_LR_SEM

0.270 3

0.000 2

0.528 3

0.331 5

0.139 7

0.535 7

0.118 8

0.326 4

0.222 3

0.292 3

0.921 4

0.089 6

0.163 4

0.129 2

0.000 2

0.131 3

0.463 3

0.278 8

0.000 3

0.699 3

0.033 7

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

TWIST+CSC

0.186 2

0.000 2

0.545 2

0.153 2

0.148 2

0.667 3

0.050 1

0.420 2

0.007 4

0.210 3

0.175 2

0.016 5

0.218 2

0.000 2

0.000 5

0.511 2

0.227 2

0.000 2

0.000 5

Ruihang Chu: TWIST: Two-Way Inter-label Self-Training for Semi-supervised 3D Instance Segmentation. CVPR 2022

WS3D_LR_Ins

0.565 1

0.667 1

0.755 1

0.614 1

0.488 1

0.767 1

0.024 2

0.710 1

0.392 1

0.370 1

0.496 1

0.298 1

0.515 1

0.857 1

0.496 1

0.667 1

0.596 1

0.990 1

0.474 1

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

InstTeacher3D

0.162 3

0.000 2

0.500 4

0.000 5

0.120 3

0.738 2

0.000 3

0.404 3

0.005 5

0.291 2

0.155 3

0.153 2

0.008 6

0.000 2

0.059 2

0.289 3

0.192 4

0.000 2

0.000 4

Yizheng Wu, Zhiyu Pan, Kewei Wang, Xingyi Li, Jiahao Cui, Liwen Xiao, Guosheng Lin, Zhiguo Cao: Instance Consistency Regularization for Semi-Supervised 3D Instance Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence

Scratch_LR_INS

0.101 6

0.000 2

0.488 6

0.000 5

0.035 5

0.491 6

0.000 3

0.160 6

0.009 3

0.111 5

0.025 6

0.002 6

0.009 5

0.000 2

0.001 4

0.265 4

0.219 3

0.000 2

0.001 2

CSC_LR_INS

0.117 5

0.000 2

0.492 5

0.040 4

0.032 6

0.524 5

0.000 3

0.174 5

0.021 2

0.139 4

0.096 4

0.112 3

0.039 4

0.000 2

0.018 3

0.258 5

0.150 6

0.000 2

0.001 3

PointContrast_LR_INS

0.119 4

0.000 2

0.534 3

0.041 3

0.047 4

0.560 4

0.000 3

0.307 4

0.000 6

0.083 6

0.071 5

0.068 4

0.064 3

0.000 2

0.000 5

0.186 6

0.176 5

0.000 2

0.000 5

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WS3D_LR_OD

0.374 1

0.867 1

0.797 1

0.655 1

0.104 1

0.678 1

0.046 1

0.215 1

0.406 1

0.186 1

0.219 1

0.034 1

0.354 1

0.160 1

0.101 1

0.741 1

0.306 1

0.679 1

0.181 1

Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022

CSC_LR_DET

0.191 2

0.667 2

0.468 3

0.226 2

0.036 2

0.420 3

0.025 2

0.010 3

0.081 3

0.066 3

0.045 2

0.000 2

0.162 3

0.010 3

0.017 2

0.657 2

0.109 2

0.420 3

0.013 2

PointContrast_LR_DET

0.187 3

0.667 2

0.523 2

0.109 3

0.027 3

0.435 2

0.005 3

0.013 2

0.199 2

0.070 2

0.035 3

0.000 4

0.183 2

0.033 2

0.003 4

0.497 3

0.078 3

0.488 2

0.005 3

Scratch_LR_DET

0.076 4

0.667 2

0.099 4

0.015 4

0.005 4

0.190 4

0.000 4

0.033 4

0.007 4

0.001 4

0.000 3

0.000 4

0.010 4

0.004 3

0.094 4

0.014 4

0.237 4

0.000 4

3D Semantic Label with Limited Annotations Benchmark

3D Semantic Instance with Limited Annotations Benchmark

3D Object Detection with Limited Annotations Benchmark

3D Semantic Label with Limited Reconstructions Benchmark

3D Semantic Instance with Limited Reconstructions Benchmark

3D Object Detection with Limited Reconstructions Benchmark