3D Semantic Instance with Limited Annotations Benchmark

The 3D semantic instance prediction task involves detecting and segmenting the object in an 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the average precision for each class. We report the mean average precision AP at overlap 0.25 (AP 25%), overlap 0.5 (AP 50%), and over overlaps in the range [0.5:0.95:0.05] (AP). Note that multiple predictions of the same ground truth instance are penalized as false positives.


This table lists the benchmark results for the 3D semantic instance with limited annotations scenario.




Method Infoavg ap 25%bathtubbedbookshelfcabinetchaircountercurtaindeskdoorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwindow
sorted bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort by
WS3D_LA_Inspermissive0.719 11.000 10.834 20.639 30.620 20.832 10.477 20.782 10.608 20.572 10.592 20.506 10.587 11.000 10.822 10.764 20.658 10.994 20.659 1
Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022
Box2Mask_LA0.685 21.000 10.994 10.847 10.635 10.787 20.518 10.544 30.741 10.561 20.598 10.396 20.541 21.000 10.258 50.912 10.531 30.869 40.591 2
Julian Chibane, Francis Engelmann, Tuan Anh Tran, Gerard Pons-Moll: Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation Using Bounding Boxes. ECCV 2022
CSC_LA_INS0.496 31.000 10.697 40.692 20.494 30.563 50.272 30.655 20.239 40.309 40.377 40.024 40.216 40.466 40.369 30.724 40.585 20.938 30.311 3
PointContrast_LA_INS0.474 40.655 50.452 50.612 40.382 40.616 30.098 40.434 50.340 30.422 30.445 30.095 30.523 30.493 30.760 20.701 50.261 50.994 10.244 4
Scratch_LA_INS0.357 50.667 40.711 30.598 50.319 50.593 40.000 50.452 40.229 50.190 50.222 50.000 50.037 50.160 50.349 40.751 30.344 40.686 50.114 5