Benchmark Results - ScanRefer Benchmark

This table lists the benchmark results for the ScanRefer Localization Benchmark scenario.

		Unique	Unique	Multiple	Multiple	Overall	Overall
Method	Info	acc@0.25IoU	acc@0.5IoU	acc@0.25IoU	acc@0.5IoU	acc@0.25IoU	acc@0.5IoU
Method	Info
ConcreteNet		0.8607 1	0.7923 1	0.4746 2	0.4091 1	0.5612 2	0.4950 1
Ozan Unal, Christos Sakaridis, Suman Saha, Fisher Yu, Luc Van Gool: Three Ways to Improve Verbo-visual Fusion for Dense 3D Visual Grounding.
M3DRef-CLIP		0.7980 5	0.7085 2	0.4692 3	0.3807 3	0.5433 3	0.4545 2
Yiming Zhang, ZeMing Gong, Angel X. Chang: Multi3DRefer: Grounding Text Description to Multiple 3D Objects. ICCV 2023
CORE-3DVG		0.8557 2	0.6867 4	0.5275 1	0.3850 2	0.6011 1	0.4527 3

3DInsVG		0.8170 3	0.6925 3	0.4582 4	0.3617 4	0.5386 4	0.4359 4

HAM		0.7799 10	0.6373 13	0.4148 12	0.3324 5	0.4967 12	0.4007 5
Jiaming Chen, Weixin Luo, Ran Song, Xiaolin Wei, Lin Ma, Wei Zhang: Learning Point-Language Hierarchical Alignment for 3D Visual Grounding.
CSA-M3LM		0.8137 4	0.6241 14	0.4544 5	0.3317 6	0.5349 5	0.3972 6

D3Net		0.7923 6	0.6843 5	0.3905 16	0.3074 10	0.4806 15	0.3919 7
Dave Zhenyu Chen, Qirui Wu, Matthias Niessner, Angel X. Chang: D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Grounding. 17th European Conference on Computer Vision (ECCV), 2022
ContraRefer		0.7832 9	0.6801 8	0.3850 17	0.2947 11	0.4743 16	0.3811 8

Clip		0.7733 15	0.6810 7	0.3619 25	0.2919 16	0.4542 21	0.3791 9

Clip-pre		0.7766 13	0.6843 5	0.3617 27	0.2904 17	0.4547 20	0.3787 10

3DJCG(Grounding)		0.7675 18	0.6059 16	0.4389 7	0.3117 8	0.5126 7	0.3776 11
Daigang Cai, Lichen Zhao, Jing Zhang†, Lu Sheng, Dong Xu: 3DJCG: A Unified Framework for Joint Dense Captioning and Visual Grounding on 3D Point Clouds. CVPR2022 Oral
3DVG-Trans +		0.7733 15	0.5787 21	0.4370 8	0.3102 9	0.5124 8	0.3704 12
Lichen Zhao∗, Daigang Cai∗, Lu Sheng†, Dong Xu: 3DVG-Transformer: Relation Modeling for Visual Grounding on Point Clouds. ICCV2021
bo3d-1		0.7469 24	0.5606 23	0.4539 6	0.3124 7	0.5196 6	0.3680 13

Se2d		0.7799 10	0.6628 10	0.3636 23	0.2823 19	0.4569 18	0.3677 14

secg		0.7288 26	0.6175 15	0.3696 22	0.2933 13	0.4501 24	0.3660 15

FE-3DGQA		0.7857 7	0.5862 20	0.4317 9	0.2935 12	0.5111 9	0.3592 16

D3Net - Pretrained		0.7659 19	0.6579 11	0.3619 25	0.2726 20	0.4525 23	0.3590 17
Dave Zhenyu Chen, Qirui Wu, Matthias Niessner, Angel X. Chang: D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Grounding. 17th European Conference on Computer Vision (ECCV), 2022
HGT		0.7692 17	0.5886 19	0.4141 13	0.2924 15	0.4937 13	0.3588 18

InstanceRefer		0.7782 12	0.6669 9	0.3457 30	0.2688 21	0.4427 26	0.3580 19
Zhihao Yuan, Xu Yan, Yinghong Liao, Ruimao Zhang, Zhen Li*, Shuguang Cui: InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referring. ICCV 2021
3DVG-Transformer		0.7576 20	0.5515 24	0.4224 11	0.2933 13	0.4976 11	0.3512 20
Lichen Zhao∗, Daigang Cai∗, Lu Sheng†, Dong Xu: 3DVG-Transformer: Relation Modeling for Visual Grounding on Point Clouds. ICCV2021
SAVG		0.7758 14	0.5664 22	0.4236 10	0.2826 18	0.5026 10	0.3462 21

PointGroup_MCAN		0.7510 21	0.6397 12	0.3271 32	0.2535 23	0.4222 28	0.3401 22

TransformerVG		0.7502 22	0.5977 17	0.3712 21	0.2628 22	0.4562 19	0.3379 23

TGNN		0.6834 30	0.5894 18	0.3312 31	0.2526 24	0.4102 31	0.3281 24
Pin-Hao Huang, Han-Hung Lee, Hwann-Tzong Chen, Tyng-Luh Liu: Text-Guided Graph Neural Network for Referring 3D Instance Segmentation. AAAI 2021
BEAUTY-DETR		0.7848 8	0.5499 25	0.3934 15	0.2480 25	0.4811 14	0.3157 25
Ayush Jain, Nikolaos Gkanatsios, Ishita Mediratta, Katerina Fragkiadaki: Looking Outside the Box to Ground Language in 3D Scenes.
grounding		0.7298 25	0.5458 26	0.3822 19	0.2421 27	0.4538 22	0.3046 26

henet		0.7110 27	0.5180 28	0.3936 14	0.2472 26	0.4590 17	0.3030 27

SRGA		0.7494 23	0.5128 29	0.3631 24	0.2218 28	0.4497 25	0.2871 28

SR-GAB		0.7016 28	0.5202 27	0.3233 34	0.1959 31	0.4081 32	0.2686 29

SPANet		0.5614 36	0.4641 31	0.2800 38	0.2071 30	0.3431 39	0.2647 30

ScanRefer		0.6859 29	0.4353 32	0.3488 29	0.2097 29	0.4244 27	0.2603 31
Dave Zhenyu Chen, Angel X. Chang, Matthias Nießner: ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language. 16th European Conference on Computer Vision (ECCV), 2020
scanrefer2		0.6340 33	0.4353 32	0.3193 35	0.1947 32	0.3898 34	0.2486 32

TransformerRefer		0.6010 34	0.4658 30	0.2540 40	0.1730 37	0.3318 40	0.2386 33

ScanRefer Baseline		0.6422 32	0.4196 34	0.3090 36	0.1832 33	0.3837 35	0.2362 34

ScanRefer_vanilla		0.6488 31	0.4056 35	0.3052 37	0.1782 35	0.3823 36	0.2292 35

pairwisemethod		0.5779 35	0.3603 36	0.2792 39	0.1746 36	0.3462 38	0.2163 36

bo3d		0.5400 37	0.1550 37	0.3817 20	0.1785 34	0.4172 30	0.1732 37

Co3d3		0.5326 38	0.1369 38	0.3848 18	0.1651 38	0.4179 29	0.1588 38

Co3d2		0.5070 39	0.1195 40	0.3569 28	0.1511 39	0.3906 33	0.1440 39

bo3d0		0.4823 40	0.1278 39	0.3271 32	0.1394 40	0.3619 37	0.1368 40

Co3d		0.0000 41	0.0000 41	0.0000 41	0.0000 41	0.0000 41	0.0000 41

		Captioning F1-Score				Dense Captioning	Object Detection
Method	Info	CIDEr@0.5IoU	BLEU-4@0.5IoU	Rouge-L@0.5IoU	METEOR@0.5IoU	DCmAP	mAP@0.5
Method	Info
vote2cap-detr		0.3128 1	0.1778 1	0.2842 1	0.1316 1	0.1825 1	0.4454 1
Sijin Chen, Hongyuan Zhu, Xin Chen, Yinjie Lei, Tao Chen, Gang YU: End-to-End 3D Dense Captioning with Vote2Cap-DETR. CVPR 2023
CFM		0.2360 2	0.1417 2	0.2253 2	0.1034 2	0.1379 5	0.3008 5

CM3D-Trans+		0.2348 3	0.1383 3	0.2250 4	0.1030 3	0.1398 4	0.2966 7
Yufeng Zhong, Long Xu, Jiebo Luo, Lin Ma: Contextual Modeling for 3D Dense Captioning on Point Clouds.
Forest-xyz		0.2266 4	0.1363 4	0.2250 3	0.1027 4	0.1161 10	0.2825 10

D3Net - Speaker		0.2088 5	0.1335 6	0.2237 5	0.1022 5	0.1481 3	0.4198 2
Dave Zhenyu Chen, Qirui Wu, Matthias Niessner, Angel X. Chang: D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Grounding. 17th European Conference on Computer Vision (ECCV), 2022
3DJCG(Captioning)		0.1918 6	0.1350 5	0.2207 6	0.1013 6	0.1506 2	0.3867 3
Daigang Cai, Lichen Zhao, Jing Zhang†, Lu Sheng, Dong Xu: 3DJCG: A Unified Framework for Joint Dense Captioning and Visual Grounding on 3D Point Clouds. CVPR2022 Oral
REMAN		0.1662 7	0.1070 7	0.1790 7	0.0815 7	0.1235 8	0.2927 9

NOAH		0.1382 8	0.0901 8	0.1598 8	0.0747 8	0.1359 6	0.2977 6

SpaCap3D		0.1359 9	0.0883 9	0.1591 9	0.0738 9	0.1182 9	0.3275 4
Heng Wang, Chaoyi Zhang, Jianhui Yu, Weidong Cai: Spatiality-guided Transformer for 3D Dense Captioning on Point Clouds. the 31st International Joint Conference on Artificial Intelligence (IJCAI), 2022
X-Trans2Cap		0.1274 10	0.0808 11	0.1392 11	0.0653 11	0.1244 7	0.2795 11
Yuan, Zhihao and Yan, Xu and Liao, Yinghong and Guo, Yao and Li, Guanbin and Cui, Shuguang and Li, Zhen: X-Trans2Cap: Cross-Modal Knowledge Transfer Using Transformer for 3D Dense Captioning. CVPR 2022
MORE-xyz		0.1239 11	0.0796 12	0.1362 12	0.0631 12	0.1116 12	0.2648 12
Yang Jiao, Shaoxiang Chen, Zequn Jie, Jingjing Chen, Lin Ma, Yu-Gang Jiang: MORE: Multi_ORder RElation Mining for Dense Captioning in 3D Scenes. ECCV 2022
SUN+		0.1148 12	0.0846 10	0.1564 10	0.0711 10	0.1143 11	0.2958 8

Scan2Cap		0.0849 13	0.0576 13	0.1073 13	0.0492 13	0.0970 13	0.2481 13
Dave Zhenyu Chen, Ali Gholami, Matthias Nießner and Angel X. Chang: Scan2Cap: Context-aware Dense Captioning in RGB-D Scans. CVPR 2021

ScanRefer Benchmark

Scan2Cap Benchmark