Benchmark Results - ScanNet Data Efficient

This table lists the benchmark results for the 3D semantic label with limited annotations scenario.

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

WeakLab-3D-Net(WS3D)	0.662 1	0.812 1	0.762 1	0.742 1	0.635 1	0.828 2	0.474 1	0.736 1	0.588 1	0.546 1	0.947 2	0.450 1	0.174 3	0.536 3	0.752 1	0.668 1	0.735 2	0.583 1	0.902 2	0.797 2	0.573 1

LE	0.608 2	0.791 2	0.726 2	0.651 3	0.589 2	0.779 5	0.346 5	0.662 4	0.493 3	0.524 2	0.923 9	0.430 2	0.234 2	0.572 1	0.638 3	0.411 6	0.708 3	0.533 4	0.855 3	0.782 3	0.508 3

One-Thing-One-Click	0.594 3	0.756 3	0.722 3	0.494 8	0.546 4	0.795 3	0.371 2	0.725 2	0.559 2	0.488 3	0.957 1	0.367 4	0.261 1	0.547 2	0.575 8	0.225 8	0.671 6	0.543 2	0.904 1	0.826 1	0.557 2
Zhengzhe Liu, Xiaojuan Qi, Chi-Wing Fu: One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation. CVPR 2021
Spec-Unc Field	0.586 4	0.736 5	0.623 7	0.664 2	0.559 3	0.840 1	0.358 3	0.666 3	0.447 4	0.429 6	0.944 3	0.421 3	0.000 9	0.411 5	0.629 4	0.614 2	0.745 1	0.541 3	0.848 5	0.758 4	0.493 4

PointContrast_LA_SEM	0.550 5	0.735 6	0.676 4	0.601 5	0.475 5	0.794 4	0.288 7	0.621 6	0.378 8	0.430 5	0.940 4	0.303 6	0.089 6	0.379 6	0.580 7	0.531 3	0.689 5	0.422 7	0.852 4	0.758 4	0.468 5

Viewpoint_BN_LA_AIR	0.548 6	0.747 4	0.574 9	0.631 4	0.456 6	0.762 7	0.355 4	0.639 5	0.412 5	0.404 7	0.940 4	0.335 5	0.107 4	0.277 8	0.645 2	0.495 4	0.666 7	0.517 5	0.818 6	0.740 7	0.431 7
Liyi Luo, Beiwen Tian, Hao Zhao, Guyue Zhou: Pointly-supervised 3D Scene Parsing with Viewpoint Bottleneck.
CSC_LA_SEM	0.531 7	0.659 7	0.638 6	0.578 6	0.417 7	0.775 6	0.254 8	0.537 7	0.396 6	0.439 4	0.939 6	0.284 8	0.083 7	0.414 4	0.599 6	0.488 5	0.698 4	0.444 6	0.785 7	0.747 6	0.440 6

SQN_LA	0.486 8	0.587 8	0.649 5	0.527 7	0.372 8	0.718 8	0.320 6	0.510 8	0.393 7	0.325 8	0.924 8	0.290 7	0.095 5	0.287 7	0.607 5	0.356 7	0.626 8	0.416 8	0.672 8	0.680 9	0.359 8

Scratch_LA_SEM	0.382 9	0.389 9	0.606 8	0.401 9	0.303 9	0.705 9	0.169 9	0.460 9	0.292 9	0.282 9	0.939 6	0.207 9	0.004 8	0.147 9	0.201 9	0.184 9	0.592 9	0.389 9	0.409 9	0.714 8	0.250 9

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WLab3D-Net_Ins(WS3D)

0.548 1

1.000 1

0.690 1

0.476 2

0.406 1

0.756 1

0.031 1

0.733 1

0.215 1

0.351 1

0.415 2

0.327 1

0.541 1

1.000 1

0.477 1

0.576 2

0.557 1

0.941 2

0.377 1

MaskVoteNet_LA

0.465 2

0.667 2

0.591 3

0.773 1

0.331 2

0.682 2

0.029 2

0.409 2

0.122 2

0.284 2

0.432 1

0.253 2

0.466 2

1.000 1

0.127 4

0.806 1

0.280 3

0.821 4

0.291 2

CSC_LA_INS

0.289 3

0.667 2

0.580 4

0.427 3

0.202 3

0.424 5

0.000 3

0.384 3

0.015 3

0.061 4

0.180 4

0.014 3

0.071 4

0.119 5

0.173 3

0.445 3

0.390 2

0.938 3

0.120 3

PointContrast_LA_INS

0.259 4

0.333 5

0.286 5

0.334 4

0.142 4

0.485 3

0.000 3

0.343 4

0.010 4

0.127 3

0.219 3

0.005 4

0.324 3

0.267 3

0.226 2

0.402 5

0.103 5

0.994 1

0.069 4

Scratch_LA_INS

0.200 5

0.667 2

0.673 2

0.145 5

0.100 5

0.430 4

0.000 3

0.314 5

0.004 5

0.025 5

0.099 5

0.000 5

0.143 4

0.076 5

0.424 4

0.198 4

0.297 5

0.006 5

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WLabel3DNet-LA(WS3D)

0.341 1

1.000 1

0.629 1

0.426 1

0.070 1

0.608 1

0.063 1

0.176 1

0.503 1

0.132 1

0.084 1

0.001 1

0.337 1

0.220 1

0.103 1

0.628 1

0.282 1

0.739 1

0.131 1

PointContrast_LA_DET

0.135 2

0.667 2

0.296 3

0.145 2

0.002 3

0.330 3

0.000 4

0.000 3

0.050 3

0.049 2

0.023 2

0.000 3

0.002 4

0.006 4

0.034 3

0.472 2

0.052 2

0.285 4

0.011 2

CSC_LA_DET

0.135 2

0.444 3

0.336 2

0.029 4

0.001 4

0.356 2

0.008 2

0.000 2

0.011 4

0.045 3

0.010 4

0.000 2

0.032 2

0.011 3

0.043 2

0.458 3

0.028 3

0.602 2

0.010 3

Scratch_LA_DET

0.098 4

0.167 4

0.253 4

0.074 3

0.002 2

0.257 4

0.004 3

0.000 4

0.080 2

0.038 4

0.013 3

0.000 3

0.006 3

0.143 2

0.002 4

0.243 4

0.017 4

0.473 3

0.001 4

Method

Info

avg iou

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

floor

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

wall

window

WeakLab3DNet (WS3D)

0.682 1

0.863 1

0.765 1

0.782 1

0.648 1

0.803 4

0.438 1

0.793 1

0.607 1

0.589 1

0.944 1

0.455 1

0.223 1

0.536 1

0.768 1

0.726 1

0.758 1

0.623 1

0.906 1

0.821 1

0.596 1

CSC_LR_SEM

0.575 2

0.671 5

0.740 2

0.727 2

0.445 3

0.847 1

0.380 5

0.602 3

0.512 2

0.447 2

0.942 2

0.291 3

0.184 2

0.353 5

0.468 5

0.508 3

0.745 2

0.602 2

0.855 2

0.765 3

0.420 5

Viewpoint_BN_LR_AIR

0.566 3

0.780 2

0.659 5

0.677 3

0.484 2

0.799 5

0.419 3

0.636 2

0.480 3

0.432 4

0.940 3

0.238 5

0.124 3

0.396 3

0.609 2

0.432 5

0.735 3

0.527 3

0.787 4

0.752 5

0.423 4

PointContrast_LR_SEM

0.555 4

0.711 4

0.668 3

0.622 4

0.425 4

0.830 2

0.433 2

0.552 4

0.273 5

0.440 3

0.938 4

0.287 4

0.096 4

0.470 2

0.576 3

0.612 2

0.687 5

0.438 5

0.781 5

0.785 2

0.474 2

Scratch_LR_SEM

0.531 5

0.750 3

0.666 4

0.553 5

0.409 5

0.816 3

0.387 4

0.487 5

0.285 4

0.368 5

0.938 4

0.310 2

0.074 5

0.388 4

0.564 4

0.468 4

0.698 4

0.448 4

0.804 3

0.761 4

0.454 3

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WeakLabel3DNet(WS3D)

0.566 1

0.667 1

0.755 3

0.614 1

0.488 1

0.767 3

0.024 2

0.710 1

0.392 1

0.370 1

0.496 1

0.306 2

0.515 3

0.857 1

0.496 2

0.667 2

0.596 1

0.990 2

0.474 1

TWIST+CSC

0.481 2

0.667 1

0.760 1

0.468 3

0.313 2

0.802 1

0.008 3

0.529 2

0.098 5

0.364 2

0.411 2

0.348 1

0.500 4

0.571 2

0.504 1

0.646 4

0.530 2

0.944 3

0.201 2

CSC_LR_INS

0.440 3

0.667 1

0.737 4

0.418 5

0.218 5

0.791 2

0.094 1

0.328 4

0.185 2

0.251 5

0.382 3

0.273 3

0.565 1

0.539 3

0.377 3

0.588 5

0.371 5

1.000 1

0.128 4

PointContrast_LR_INS

0.432 4

0.667 1

0.757 2

0.560 2

0.278 4

0.740 4

0.003 5

0.435 3

0.123 4

0.309 3

0.347 4

0.109 5

0.522 2

0.429 5

0.223 5

0.739 1

0.434 4

0.944 3

0.149 3

Scratch_LR_INS

0.413 5

0.667 1

0.720 5

0.442 4

0.288 3

0.735 5

0.005 4

0.326 5

0.138 3

0.302 4

0.329 5

0.204 4

0.445 5

0.498 4

0.229 4

0.657 3

0.452 3

0.889 5

0.115 5

Method

Info

avg ap 50%

bathtub

bed

bookshelf

cabinet

chair

counter

curtain

desk

door

otherfurniture

picture

refrigerator

shower curtain

sink

sofa

table

toilet

window

WLabel3DNet-LR(WS3D)

0.374 1

0.867 1

0.797 1

0.655 1

0.104 1

0.678 1

0.046 1

0.215 1

0.406 1

0.186 1

0.219 1

0.034 1

0.354 1

0.160 1

0.101 1

0.741 1

0.306 1

0.679 1

0.181 1

CSC_LR_DET

0.191 2

0.667 2

0.468 3

0.226 2

0.036 2

0.420 3

0.025 2

0.010 3

0.081 3

0.066 3

0.045 2

0.000 2

0.162 3

0.010 3

0.017 2

0.657 2

0.109 2

0.420 3

0.013 2

PointContrast_LR_DET

0.187 3

0.667 2

0.523 2

0.109 3

0.027 3

0.435 2

0.005 3

0.013 2

0.199 2

0.070 2

0.035 3

0.000 4

0.183 2

0.033 2

0.003 4

0.497 3

0.078 3

0.488 2

0.005 3

Scratch_LR_DET

0.076 4

0.667 2

0.099 4

0.015 4

0.005 4

0.190 4

0.000 4

0.033 4

0.007 4

0.001 4

0.000 3

0.000 4

0.010 4

0.004 3

0.094 4

0.014 4

0.237 4

0.000 4

3D Semantic label with Limited Annotations benchmark

3D Semantic instance with Limited Annotations benchmark

3D Object detection with Limited Annotations benchmark

3D Semantic label with Limited Reconstructions benchmark

3D Semantic instance with Limited Reconstructions benchmark

3D Object detection with Limited Reconstructions benchmark