3D Semantic Instance with Limited Annotations Benchmark

The 3D semantic instance prediction task involves detecting and segmenting the object in an 3D scan mesh.

Evaluation and metrics

Our evaluation ranks all methods according to the average precision for each class. We report the mean average precision AP at overlap 0.25 (AP 25%), overlap 0.5 (AP 50%), and over overlaps in the range [0.5:0.95:0.05] (AP). Note that multiple predictions of the same ground truth instance are penalized as false positives.


This table lists the benchmark results for the 3D semantic instance with limited annotations scenario.




Method Infoavg ap 50%bathtubbedbookshelfcabinetchaircountercurtaindeskdoorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwindow
sorted bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort bysort by
WS3D_LA_Inspermissive0.668 11.000 10.786 10.845 10.647 10.777 10.137 10.618 20.432 10.506 10.564 20.561 10.546 31.000 10.562 10.786 20.675 11.000 10.577 1
Kangcheng Liu: WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. European Conference on Computer Vision (ECCV), 2022
Box2Mask_LA0.592 21.000 10.619 50.820 20.471 20.773 20.104 20.618 10.377 20.409 20.591 10.364 20.515 40.857 20.443 30.782 30.524 41.000 10.382 2
Julian Chibane, Francis Engelmann, Tuan Anh Tran, Gerard Pons-Moll: Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation Using Bounding Boxes. ECCV 2022
CSC_LA_INS0.494 30.528 40.766 30.800 30.408 40.611 30.045 30.547 40.055 50.368 30.429 30.126 30.389 50.819 30.421 40.775 40.550 21.000 10.253 4
PointContrast_LA_INS0.471 40.667 30.773 20.646 50.330 50.490 40.032 40.470 50.122 30.368 40.349 50.048 50.592 10.614 50.338 50.789 10.536 30.997 40.316 3
Scratch_LA_INS0.464 50.222 50.763 40.714 40.464 30.469 50.027 50.591 30.079 40.303 50.395 40.075 40.550 20.777 40.458 20.712 50.512 50.997 40.243 5