3D Semantic Instance with Limited Annotations Benchmark

The 3D semantic instance prediction task involves detecting and segmenting the object in an 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the average precision for each class. We report the mean average precision AP at overlap 0.25 (AP 25%), overlap 0.5 (AP 50%), and over overlaps in the range [0.5:0.95:0.05] (AP). Note that multiple predictions of the same ground truth instance are penalized as false positives.


This table lists the benchmark results for the 3D semantic instance with limited annotations scenario.




Method Infoavg ap 25%bathtubbedbookshelfcabinetchaircountercurtaindeskdoorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwindow
sorted bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort by
WS3D_LA_Inspermissive0.759 11.000 10.945 20.851 10.694 10.821 20.519 20.838 10.556 20.598 20.624 10.506 10.668 11.000 10.853 10.810 40.716 11.000 10.663 1
Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022
Box2Mask_LA0.731 21.000 10.998 10.777 40.660 30.853 10.616 10.629 30.907 10.610 10.611 20.415 20.515 41.000 10.450 50.905 10.688 20.983 40.543 2
Julian Chibane, Francis Engelmann, Tuan Anh Tran, Gerard Pons-Moll: Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation Using Bounding Boxes. ECCV 2022
CSC_LA_INS0.654 31.000 10.864 40.844 20.672 20.661 30.480 30.533 50.385 30.473 40.543 30.239 30.539 30.714 50.853 10.866 20.675 31.000 10.425 5
PointContrast_LA_INS0.637 41.000 10.895 30.829 30.605 50.660 40.359 50.765 20.373 40.488 30.502 50.123 50.423 51.000 10.737 30.743 50.521 50.994 30.454 3
Scratch_LA_INS0.623 51.000 10.859 50.727 50.613 40.611 50.468 40.603 40.261 50.463 50.519 40.204 40.600 20.819 40.703 40.836 30.567 40.938 50.432 4