3D Object Detection with Limited Annotations Benchmark
The 3D semantic instance prediction task involves detecting and segmenting the object in an 3D scan mesh.
Evaluation and metricsOur evaluation ranks all methods according to the average precision for each class. We report the mean average precision AP at overlap 0.25 (AP 25%), overlap 0.5 (AP 50%), and over overlaps in the range [0.5:0.95:0.05] (AP). Note that multiple predictions of the same ground truth instance are penalized as false positives.
This table lists the benchmark results for the 3D object detection with limited annotations scenario.
Method | Info | avg ap 25% | bathtub | bed | bookshelf | cabinet | chair | counter | curtain | desk | door | otherfurniture | picture | refrigerator | shower curtain | sink | sofa | table | toilet | window |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
WS3D_LA_OD | 0.566 1 | 0.917 1 | 0.891 1 | 0.674 1 | 0.379 1 | 0.840 1 | 0.486 1 | 0.301 1 | 0.649 1 | 0.436 1 | 0.425 1 | 0.162 1 | 0.428 1 | 0.484 2 | 0.474 1 | 0.836 1 | 0.434 1 | 0.882 1 | 0.491 1 | |
Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022 | ||||||||||||||||||||
CSC_LA_DET | 0.336 2 | 0.667 2 | 0.498 4 | 0.494 2 | 0.061 2 | 0.653 3 | 0.038 4 | 0.138 3 | 0.268 3 | 0.220 2 | 0.211 2 | 0.052 2 | 0.338 2 | 0.333 3 | 0.241 2 | 0.798 2 | 0.174 2 | 0.709 2 | 0.148 2 | |
PointContrast_LA_DET | 0.315 3 | 0.278 4 | 0.589 2 | 0.492 3 | 0.034 4 | 0.701 2 | 0.142 2 | 0.158 2 | 0.173 4 | 0.219 3 | 0.199 3 | 0.003 3 | 0.306 3 | 0.550 1 | 0.163 3 | 0.695 3 | 0.165 3 | 0.662 3 | 0.139 3 | |
Scratch_LA_DET | 0.281 4 | 0.667 2 | 0.583 3 | 0.339 4 | 0.046 3 | 0.586 4 | 0.062 3 | 0.106 4 | 0.310 2 | 0.160 4 | 0.095 4 | 0.001 4 | 0.135 4 | 0.323 4 | 0.148 4 | 0.627 4 | 0.141 4 | 0.618 4 | 0.116 4 | |