Benchmark Results - ScanRefer Benchmark

This table lists the benchmark results for the ScanRefer Localization Benchmark scenario.

		Unique	Unique	Multiple	Multiple	Overall	Overall
Method	Info	acc@0.25IoU	acc@0.5IoU	acc@0.25IoU	acc@0.5IoU	acc@0.25IoU	acc@0.5IoU
Method	Info
ConcreteNet		0.8607 1	0.7923 1	0.4746 4	0.4091 1	0.5612 3	0.4950 1
Ozan Unal, Christos Sakaridis, Suman Saha, Fisher Yu, Luc Van Gool: Three Ways to Improve Verbo-visual Fusion for Dense 3D Visual Grounding.
cus3d		0.8384 3	0.7073 5	0.4908 2	0.4000 2	0.5688 2	0.4689 2

pointclip		0.8211 4	0.7082 4	0.4803 3	0.3884 3	0.5567 4	0.4601 3

M3DRef-SCLIP		0.7997 7	0.7123 2	0.4708 5	0.3805 6	0.5445 5	0.4549 4

M3DRef-CLIP		0.7980 8	0.7085 3	0.4692 6	0.3807 5	0.5433 6	0.4545 5
Yiming Zhang, ZeMing Gong, Angel X. Chang: Multi3DRefer: Grounding Text Description to Multiple 3D Objects. ICCV 2023
CORE-3DVG		0.8557 2	0.6867 7	0.5275 1	0.3850 4	0.6011 1	0.4527 6

3DInsVG		0.8170 5	0.6925 6	0.4582 8	0.3617 7	0.5386 7	0.4359 7

RG-SAN		0.7964 9	0.6785 12	0.4591 7	0.3600 8	0.5348 9	0.4314 8

HAM		0.7799 14	0.6373 17	0.4148 16	0.3324 9	0.4967 16	0.4007 9
Jiaming Chen, Weixin Luo, Ran Song, Xiaolin Wei, Lin Ma, Wei Zhang: Learning Point-Language Hierarchical Alignment for 3D Visual Grounding.
CSA-M3LM		0.8137 6	0.6241 18	0.4544 9	0.3317 10	0.5349 8	0.3972 10

D3Net		0.7923 10	0.6843 8	0.3905 20	0.3074 14	0.4806 19	0.3919 11
Dave Zhenyu Chen, Qirui Wu, Matthias Niessner, Angel X. Chang: D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Grounding. 17th European Conference on Computer Vision (ECCV), 2022
ContraRefer		0.7832 13	0.6801 11	0.3850 21	0.2947 16	0.4743 20	0.3811 12

Clip		0.7733 19	0.6810 10	0.3619 30	0.2919 21	0.4542 25	0.3791 13

Clip-pre		0.7766 17	0.6843 8	0.3617 32	0.2904 22	0.4547 24	0.3787 14

3DJCG(Grounding)		0.7675 22	0.6059 20	0.4389 11	0.3117 12	0.5126 11	0.3776 15
Daigang Cai, Lichen Zhao, Jing Zhang†, Lu Sheng, Dong Xu: 3DJCG: A Unified Framework for Joint Dense Captioning and Visual Grounding on 3D Point Clouds. CVPR2022 Oral
3DVG-Trans +		0.7733 19	0.5787 25	0.4370 12	0.3102 13	0.5124 12	0.3704 16
Lichen Zhao∗, Daigang Cai∗, Lu Sheng†, Dong Xu: 3DVG-Transformer: Relation Modeling for Visual Grounding on Point Clouds. ICCV2021
bo3d-1		0.7469 28	0.5606 28	0.4539 10	0.3124 11	0.5196 10	0.3680 17

Se2d		0.7799 14	0.6628 14	0.3636 28	0.2823 24	0.4569 22	0.3677 18

secg		0.7288 30	0.6175 19	0.3696 27	0.2933 18	0.4501 28	0.3660 19

SAF		0.6348 37	0.5647 27	0.3726 25	0.3009 15	0.4314 31	0.3601 20

FE-3DGQA		0.7857 11	0.5862 24	0.4317 13	0.2935 17	0.5111 13	0.3592 21

D3Net - Pretrained		0.7659 23	0.6579 15	0.3619 30	0.2726 25	0.4525 27	0.3590 22
Dave Zhenyu Chen, Qirui Wu, Matthias Niessner, Angel X. Chang: D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Grounding. 17th European Conference on Computer Vision (ECCV), 2022
HGT		0.7692 21	0.5886 23	0.4141 17	0.2924 20	0.4937 17	0.3588 23

InstanceRefer		0.7782 16	0.6669 13	0.3457 35	0.2688 26	0.4427 30	0.3580 24
Zhihao Yuan, Xu Yan, Yinghong Liao, Ruimao Zhang, Zhen Li*, Shuguang Cui: InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referring. ICCV 2021
3DVG-Transformer		0.7576 24	0.5515 29	0.4224 15	0.2933 18	0.4976 15	0.3512 25
Lichen Zhao∗, Daigang Cai∗, Lu Sheng†, Dong Xu: 3DVG-Transformer: Relation Modeling for Visual Grounding on Point Clouds. ICCV2021
SAVG		0.7758 18	0.5664 26	0.4236 14	0.2826 23	0.5026 14	0.3462 26

PointGroup_MCAN		0.7510 25	0.6397 16	0.3271 37	0.2535 28	0.4222 33	0.3401 27

TransformerVG		0.7502 26	0.5977 21	0.3712 26	0.2628 27	0.4562 23	0.3379 28

TGNN		0.6834 34	0.5894 22	0.3312 36	0.2526 29	0.4102 36	0.3281 29
Pin-Hao Huang, Han-Hung Lee, Hwann-Tzong Chen, Tyng-Luh Liu: Text-Guided Graph Neural Network for Referring 3D Instance Segmentation. AAAI 2021
BEAUTY-DETR		0.7848 12	0.5499 30	0.3934 19	0.2480 30	0.4811 18	0.3157 30
Ayush Jain, Nikolaos Gkanatsios, Ishita Mediratta, Katerina Fragkiadaki: Looking Outside the Box to Ground Language in 3D Scenes.
grounding		0.7298 29	0.5458 31	0.3822 23	0.2421 32	0.4538 26	0.3046 31

henet		0.7110 31	0.5180 33	0.3936 18	0.2472 31	0.4590 21	0.3030 32

SRGA		0.7494 27	0.5128 34	0.3631 29	0.2218 33	0.4497 29	0.2871 33

SR-GAB		0.7016 32	0.5202 32	0.3233 39	0.1959 36	0.4081 37	0.2686 34

SPANet		0.5614 41	0.4641 36	0.2800 43	0.2071 35	0.3431 44	0.2647 35

ScanRefer		0.6859 33	0.4353 37	0.3488 34	0.2097 34	0.4244 32	0.2603 36
Dave Zhenyu Chen, Angel X. Chang, Matthias Nießner: ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language. 16th European Conference on Computer Vision (ECCV), 2020
scanrefer2		0.6340 38	0.4353 37	0.3193 40	0.1947 37	0.3898 39	0.2486 37

TransformerRefer		0.6010 39	0.4658 35	0.2540 45	0.1730 42	0.3318 45	0.2386 38

ScanRefer Baseline		0.6422 36	0.4196 39	0.3090 41	0.1832 38	0.3837 40	0.2362 39

ScanRefer_vanilla		0.6488 35	0.4056 40	0.3052 42	0.1782 40	0.3823 41	0.2292 40

pairwisemethod		0.5779 40	0.3603 41	0.2792 44	0.1746 41	0.3462 43	0.2163 41

bo3d		0.5400 42	0.1550 42	0.3817 24	0.1785 39	0.4172 35	0.1732 42

Co3d3		0.5326 43	0.1369 43	0.3848 22	0.1651 43	0.4179 34	0.1588 43

Co3d2		0.5070 44	0.1195 45	0.3569 33	0.1511 44	0.3906 38	0.1440 44

bo3d0		0.4823 45	0.1278 44	0.3271 37	0.1394 45	0.3619 42	0.1368 45

Co3d		0.0000 46	0.0000 46	0.0000 46	0.0000 46	0.0000 46	0.0000 46

		Captioning F1-Score				Dense Captioning	Object Detection
Method	Info	CIDEr@0.5IoU	BLEU-4@0.5IoU	Rouge-L@0.5IoU	METEOR@0.5IoU	DCmAP	mAP@0.5
Method	Info
Vote2Cap-DETR++		0.3360 1	0.1908 1	0.3012 1	0.1386 1	0.1864 1	0.5090 1
Sijin Chen, Hongyuan Zhu, Mingsheng Li, Xin Chen, Peng Guo, Yinjie Lei, Gang Yu, Taihao Li, Tao Chen: Vote2Cap-DETR++: Decoupling Localization and Describing for End-to-End 3D Dense Captioning.
vote2cap-detr		0.3128 2	0.1778 2	0.2842 3	0.1316 3	0.1825 2	0.4454 3
Sijin Chen, Hongyuan Zhu, Xin Chen, Yinjie Lei, Tao Chen, Gang YU, Taihao Li: End-to-End 3D Dense Captioning with Vote2Cap-DETR. CVPR 2023
TMP		0.3029 3	0.1728 3	0.2898 2	0.1332 2	0.1801 3	0.4605 2

CFM		0.2360 4	0.1417 4	0.2253 4	0.1034 4	0.1379 7	0.3008 7

CM3D-Trans+		0.2348 5	0.1383 5	0.2250 6	0.1030 5	0.1398 6	0.2966 9
Yufeng Zhong, Long Xu, Jiebo Luo, Lin Ma: Contextual Modeling for 3D Dense Captioning on Point Clouds.
Forest-xyz		0.2266 6	0.1363 6	0.2250 5	0.1027 6	0.1161 12	0.2825 12

D3Net - Speaker		0.2088 7	0.1335 8	0.2237 7	0.1022 7	0.1481 5	0.4198 4
Dave Zhenyu Chen, Qirui Wu, Matthias Niessner, Angel X. Chang: D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Grounding. 17th European Conference on Computer Vision (ECCV), 2022
3DJCG(Captioning)		0.1918 8	0.1350 7	0.2207 8	0.1013 8	0.1506 4	0.3867 5
Daigang Cai, Lichen Zhao, Jing Zhang†, Lu Sheng, Dong Xu: 3DJCG: A Unified Framework for Joint Dense Captioning and Visual Grounding on 3D Point Clouds. CVPR2022 Oral
REMAN		0.1662 9	0.1070 9	0.1790 9	0.0815 9	0.1235 10	0.2927 11

NOAH		0.1382 10	0.0901 10	0.1598 10	0.0747 10	0.1359 8	0.2977 8

SpaCap3D		0.1359 11	0.0883 11	0.1591 11	0.0738 11	0.1182 11	0.3275 6
Heng Wang, Chaoyi Zhang, Jianhui Yu, Weidong Cai: Spatiality-guided Transformer for 3D Dense Captioning on Point Clouds. the 31st International Joint Conference on Artificial Intelligence (IJCAI), 2022
X-Trans2Cap		0.1274 12	0.0808 13	0.1392 13	0.0653 13	0.1244 9	0.2795 13
Yuan, Zhihao and Yan, Xu and Liao, Yinghong and Guo, Yao and Li, Guanbin and Cui, Shuguang and Li, Zhen: X-Trans2Cap: Cross-Modal Knowledge Transfer Using Transformer for 3D Dense Captioning. CVPR 2022
MORE-xyz		0.1239 13	0.0796 14	0.1362 14	0.0631 14	0.1116 14	0.2648 14
Yang Jiao, Shaoxiang Chen, Zequn Jie, Jingjing Chen, Lin Ma, Yu-Gang Jiang: MORE: Multi_ORder RElation Mining for Dense Captioning in 3D Scenes. ECCV 2022
SUN+		0.1148 14	0.0846 12	0.1564 12	0.0711 12	0.1143 13	0.2958 10

Scan2Cap		0.0849 15	0.0576 15	0.1073 15	0.0492 15	0.0970 15	0.2481 15
Dave Zhenyu Chen, Ali Gholami, Matthias Nießner and Angel X. Chang: Scan2Cap: Context-aware Dense Captioning in RGB-D Scans. CVPR 2021

ScanRefer Benchmark

Scan2Cap Benchmark