Benchmark Results - ScanNet Data Efficient

This table lists the benchmark results for the 3D semantic label with limited annotations scenario.

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

ActiveST	0.703 1	0.977 1	0.776 2	0.657 4	0.707 1	0.874 1	0.541 1	0.744 1	0.605 1	0.610 1	0.968 1	0.442 3	0.126 5	0.705 1	0.785 1	0.742 1	0.791 1	0.586 1	0.940 1	0.839 1	0.645 1
Gengxin Liu, Oliver van Kaick, Hui Huang, Ruizhen Hu: Active Self-Training for Weakly Supervised 3D Scene Semantic Segmentation.
RM3D	0.662 2	0.812 3	0.762 3	0.742 1	0.635 2	0.828 5	0.474 2	0.736 2	0.588 2	0.546 2	0.947 4	0.450 2	0.174 4	0.536 4	0.752 2	0.668 2	0.735 4	0.583 2	0.902 5	0.797 5	0.573 4
Kangcheng Liu: RM3D: Robust Data-Efficient 3D Scene Parsing via Traditional and Learnt 3D Descriptors-Based Semantic Region Merging. International Journal of Computer Vision (IJCV), 2022
DE-3DLearner LA	0.639 3	0.839 2	0.723 5	0.681 2	0.629 3	0.839 4	0.424 3	0.728 3	0.538 4	0.526 3	0.945 5	0.427 5	0.120 6	0.511 6	0.643 5	0.547 5	0.781 2	0.566 4	0.905 3	0.809 4	0.607 3
Ping-Chung Yu, Cheng Sun, Min Sun: Data Efficient 3D Learner via Knowledge Transferred from 2D Model. ECCV 2022
GaIA	0.638 4	0.536 11	0.783 1	0.651 5	0.600 4	0.840 2	0.413 4	0.728 3	0.490 6	0.520 5	0.948 3	0.475 1	0.299 1	0.518 5	0.680 3	0.629 3	0.729 5	0.573 3	0.906 2	0.815 3	0.626 2
Min Seok Lee, Seok Woo Yang, and Sung Won Han: GaIA: Graphical Information gain based Attention Network for Weakly Supervised 3D Point Cloud Semantic Segmentation. WACV 2023
LE	0.608 5	0.791 4	0.726 4	0.651 5	0.589 5	0.779 8	0.346 8	0.662 7	0.493 5	0.524 4	0.923 12	0.430 4	0.234 3	0.572 2	0.638 6	0.411 9	0.708 6	0.533 7	0.855 6	0.782 6	0.508 6

One-Thing-One-Click	0.594 6	0.756 5	0.722 6	0.494 11	0.546 7	0.795 6	0.371 5	0.725 5	0.559 3	0.488 6	0.957 2	0.367 7	0.261 2	0.547 3	0.575 11	0.225 11	0.671 9	0.543 5	0.904 4	0.826 2	0.557 5
Zhengzhe Liu, Xiaojuan Qi, Chi-Wing Fu: One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation. CVPR 2021
VIBUS	0.586 7	0.736 7	0.623 10	0.664 3	0.559 6	0.840 2	0.358 6	0.666 6	0.447 7	0.429 9	0.944 6	0.421 6	0.000 12	0.411 8	0.629 7	0.614 4	0.745 3	0.541 6	0.848 8	0.758 7	0.493 7
Beiwen Tian,Liyi Luo,Hao Zhao,Guyue Zhou: VIBUS: Data-efficient 3D Scene Parsing with VIewpoint Bottleneck and Uncertainty-Spectrum Modeling. ISPRS Journal of Photogrammetry and Remote Sensing
PointContrast_LA_SEM	0.550 8	0.735 8	0.676 7	0.601 8	0.475 8	0.794 7	0.288 10	0.621 9	0.378 11	0.430 8	0.940 7	0.303 9	0.089 9	0.379 9	0.580 10	0.531 6	0.689 8	0.422 10	0.852 7	0.758 7	0.468 8

Viewpoint_BN_LA_AIR	0.548 9	0.747 6	0.574 12	0.631 7	0.456 9	0.762 10	0.355 7	0.639 8	0.412 8	0.404 10	0.940 7	0.335 8	0.107 7	0.277 11	0.645 4	0.495 7	0.666 10	0.517 8	0.818 9	0.740 10	0.431 10
Liyi Luo, Beiwen Tian, Hao Zhao, Guyue Zhou: Pointly-supervised 3D Scene Parsing with Viewpoint Bottleneck.
CSC_LA_SEM	0.531 10	0.659 9	0.638 9	0.578 9	0.417 10	0.775 9	0.254 11	0.537 10	0.396 9	0.439 7	0.939 9	0.284 11	0.083 10	0.414 7	0.599 9	0.488 8	0.698 7	0.444 9	0.785 10	0.747 9	0.440 9

SQN_LA	0.486 11	0.587 10	0.649 8	0.527 10	0.372 11	0.718 11	0.320 9	0.510 11	0.393 10	0.325 11	0.924 11	0.290 10	0.095 8	0.287 10	0.607 8	0.356 10	0.626 11	0.416 11	0.672 11	0.680 12	0.359 11

Scratch_LA_SEM	0.382 12	0.389 12	0.606 11	0.401 12	0.303 12	0.705 12	0.169 12	0.460 12	0.292 12	0.282 12	0.939 9	0.207 12	0.004 11	0.147 12	0.201 12	0.184 12	0.592 12	0.389 12	0.409 12	0.714 11	0.250 12

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WLab3D-Net_Ins(WS3D)

0.548 1

1.000 1

0.690 1

0.476 2

0.406 1

0.756 1

0.031 1

0.733 1

0.215 1

0.351 1

0.415 2

0.319 1

0.541 1

1.000 1

0.477 1

0.576 2

0.557 1

0.941 2

0.377 1

Box2Mask_LA

0.465 2

0.667 2

0.591 3

0.773 1

0.331 2

0.682 2

0.029 2

0.409 2

0.122 2

0.284 2

0.432 1

0.253 2

0.466 2

1.000 1

0.127 4

0.806 1

0.280 3

0.821 4

0.291 2

Julian Chibane, Francis Engelmann, Tuan Anh Tran, Gerard Pons-Moll: Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation Using Bounding Boxes. ECCV 2022

CSC_LA_INS

0.289 3

0.667 2

0.580 4

0.427 3

0.202 3

0.424 5

0.000 3

0.384 3

0.015 3

0.061 4

0.180 4

0.014 3

0.071 4

0.119 5

0.173 3

0.445 3

0.390 2

0.938 3

0.120 3

PointContrast_LA_INS

0.259 4

0.333 5

0.286 5

0.334 4

0.142 4

0.485 3

0.000 3

0.343 4

0.010 4

0.127 3

0.219 3

0.005 4

0.324 3

0.267 3

0.226 2

0.402 5

0.103 5

0.994 1

0.069 4

Scratch_LA_INS

0.200 5

0.667 2

0.673 2

0.145 5

0.100 5

0.430 4

0.000 3

0.314 5

0.004 5

0.025 5

0.099 5

0.000 5

0.143 4

0.076 5

0.424 4

0.198 4

0.297 5

0.006 5

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WS3D_LA

0.341 1

1.000 1

0.629 1

0.426 1

0.070 1

0.608 1

0.063 1

0.176 1

0.503 1

0.132 1

0.084 1

0.001 1

0.337 1

0.220 1

0.103 1

0.628 1

0.282 1

0.739 1

0.131 1

PointContrast_LA_DET

0.135 2

0.667 2

0.296 3

0.145 2

0.002 3

0.330 3

0.000 4

0.000 3

0.050 3

0.049 2

0.023 2

0.000 3

0.002 4

0.006 4

0.034 3

0.472 2

0.052 2

0.285 4

0.011 2

CSC_LA_DET

0.135 2

0.444 3

0.336 2

0.029 4

0.001 4

0.356 2

0.008 2

0.000 2

0.011 4

0.045 3

0.010 4

0.000 2

0.032 2

0.011 3

0.043 2

0.458 3

0.028 3

0.602 2

0.010 3

Scratch_LA_DET

0.098 4

0.167 4

0.253 4

0.074 3

0.002 2

0.257 4

0.004 3

0.000 4

0.080 2

0.038 4

0.013 3

0.000 3

0.006 3

0.143 2

0.002 4

0.243 4

0.017 4

0.473 3

0.001 4

Method

Info

avg iou

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

floor

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

wall

window

WeakLab3DNet (WS3D)

0.682 1

0.863 1

0.765 2

0.782 1

0.648 1

0.803 7

0.438 3

0.793 1

0.607 1

0.589 1

0.944 3

0.455 1

0.223 2

0.536 2

0.768 1

0.726 1

0.758 2

0.623 1

0.906 1

0.821 2

0.596 3

MFA-Net

0.678 2

0.779 4

0.782 1

0.774 2

0.637 2

0.827 4

0.491 1

0.736 2

0.597 2

0.561 2

0.947 2

0.438 2

0.206 3

0.610 1

0.758 2

0.667 2

0.773 1

0.594 3

0.880 2

0.824 1

0.673 1

DE-3DLearner LR

0.608 3

0.853 2

0.689 5

0.593 7

0.483 5

0.830 2

0.466 2

0.652 3

0.528 4

0.482 3

0.954 1

0.288 6

0.250 1

0.448 4

0.595 4

0.532 5

0.748 3

0.503 6

0.822 4

0.806 3

0.647 2

Ping-Chung Yu, Cheng Sun, Min Sun: Data Efficient 3D Learner via Knowledge Transferred from 2D Model. ECCV 2022

CSC_LR_SEM

0.575 4

0.671 8

0.740 3

0.727 3

0.445 6

0.847 1

0.380 7

0.602 5

0.512 5

0.447 5

0.942 4

0.291 5

0.184 4

0.353 8

0.468 8

0.508 6

0.745 4

0.602 2

0.855 3

0.765 5

0.420 8

CSG_3DSegNet

0.570 5

0.717 6

0.730 4

0.697 4

0.521 3

0.823 5

0.377 8

0.419 8

0.531 3

0.452 4

0.935 8

0.316 3

0.147 5

0.359 7

0.551 7

0.551 4

0.692 7

0.513 5

0.797 6

0.764 6

0.508 4

Viewpoint_BN_LR_AIR

0.566 6

0.780 3

0.659 8

0.677 5

0.484 4

0.799 8

0.419 5

0.636 4

0.480 6

0.432 7

0.940 5

0.238 8

0.124 6

0.396 5

0.609 3

0.432 8

0.735 5

0.527 4

0.787 7

0.752 8

0.423 7

PointContrast_LR_SEM

0.555 7

0.711 7

0.668 6

0.622 6

0.425 7

0.830 2

0.433 4

0.552 6

0.273 8

0.440 6

0.938 6

0.287 7

0.096 7

0.470 3

0.576 5

0.612 3

0.687 8

0.438 8

0.781 8

0.785 4

0.474 5

Scratch_LR_SEM

0.531 8

0.750 5

0.666 7

0.553 8

0.409 8

0.816 6

0.387 6

0.487 7

0.285 7

0.368 8

0.938 6

0.310 4

0.074 8

0.388 6

0.564 6

0.468 7

0.698 6

0.448 7

0.804 5

0.761 7

0.454 6

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WeakLabel3DNet(WS3D)

0.649 1

1.000 1

0.800 1

0.721 1

0.603 1

0.807 2

0.044 3

0.735 1

0.377 1

0.466 1

0.550 1

0.605 1

0.550 2

1.000 1

0.506 2

0.776 2

0.618 1

1.000 1

0.526 1

InstTeacher3D

0.598 2

1.000 1

0.727 5

0.205 6

0.420 2

0.833 1

0.405 1

0.470 3

0.247 2

0.463 2

0.536 2

0.559 2

0.533 3

1.000 1

0.552 1

0.782 1

0.587 2

1.000 1

0.444 2

TWIST+CSC

0.481 3

0.667 3

0.760 2

0.468 3

0.313 3

0.802 3

0.008 4

0.529 2

0.098 6

0.364 3

0.411 3

0.348 3

0.500 5

0.571 3

0.504 3

0.646 5

0.530 3

0.944 4

0.201 3

CSC_LR_INS

0.440 4

0.667 3

0.737 4

0.418 5

0.218 6

0.791 4

0.094 2

0.328 5

0.185 3

0.251 6

0.382 4

0.273 4

0.565 1

0.539 4

0.377 4

0.588 6

0.371 6

1.000 1

0.128 5

PointContrast_LR_INS

0.432 5

0.667 3

0.757 3

0.560 2

0.278 5

0.740 5

0.003 6

0.435 4

0.123 5

0.309 4

0.347 5

0.109 6

0.522 4

0.429 6

0.223 6

0.739 3

0.434 5

0.944 4

0.149 4

Scratch_LR_INS

0.413 6

0.667 3

0.720 6

0.442 4

0.288 4

0.735 6

0.005 5

0.326 6

0.138 4

0.302 5

0.329 6

0.204 5

0.445 6

0.498 5

0.229 5

0.657 4

0.452 4

0.889 6

0.115 6

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WLabel3DNet-LR(WS3D)

0.374 1

0.867 1

0.797 1

0.655 1

0.104 1

0.678 1

0.046 1

0.215 1

0.406 1

0.186 1

0.219 1

0.034 1

0.354 1

0.160 1

0.101 1

0.741 1

0.306 1

0.679 1

0.181 1

CSC_LR_DET

0.191 2

0.667 2

0.468 3

0.226 2

0.036 2

0.420 3

0.025 2

0.010 3

0.081 3

0.066 3

0.045 2

0.000 2

0.162 3

0.010 3

0.017 2

0.657 2

0.109 2

0.420 3

0.013 2

PointContrast_LR_DET

0.187 3

0.667 2

0.523 2

0.109 3

0.027 3

0.435 2

0.005 3

0.013 2

0.199 2

0.070 2

0.035 3

0.000 4

0.183 2

0.033 2

0.003 4

0.497 3

0.078 3

0.488 2

0.005 3

Scratch_LR_DET

0.076 4

0.667 2

0.099 4

0.015 4

0.005 4

0.190 4

0.000 4

0.033 4

0.007 4

0.001 4

0.000 3

0.000 4

0.010 4

0.004 3

0.094 4

0.014 4

0.237 4

0.000 4

3D Semantic Label with Limited Annotations Benchmark

3D Semantic Instance with Limited Annotations Benchmark

3D Object Detection with Limited Annotations Benchmark

3D Semantic Label with Limited Reconstructions Benchmark

3D Semantic Instance with Limited Reconstructions Benchmark

3D Object Detection with Limited Reconstructions Benchmark