Benchmark Results - ScanRefer Benchmark

This table lists the benchmark results for the ScanRefer Localization Benchmark scenario.

		Unique	Unique	Multiple	Multiple	Overall	Overall
Method	Info	acc@0.25IoU	acc@0.5IoU	acc@0.25IoU	acc@0.5IoU	acc@0.25IoU	acc@0.5IoU
Method	Info
DMO-3DG		0.8195 5	0.6900 7	0.4975 2	0.3967 4	0.5697 2	0.4625 3

TransformerRefer		0.6010 41	0.4658 37	0.2540 47	0.1730 44	0.3318 47	0.2386 40

HGT		0.7692 23	0.5886 25	0.4141 19	0.2924 22	0.4937 19	0.3588 25

BEAUTY-DETR		0.7848 14	0.5499 32	0.3934 21	0.2480 32	0.4811 20	0.3157 32
Ayush Jain, Nikolaos Gkanatsios, Ishita Mediratta, Katerina Fragkiadaki: Looking Outside the Box to Ground Language in 3D Scenes.
Clip-pre		0.7766 19	0.6843 9	0.3617 34	0.2904 24	0.4547 26	0.3787 16

Clip		0.7733 21	0.6810 11	0.3619 32	0.2919 23	0.4542 27	0.3791 15

TransformerVG		0.7502 28	0.5977 23	0.3712 28	0.2628 29	0.4562 25	0.3379 30

FE-3DGQA		0.7857 13	0.5862 26	0.4317 15	0.2935 19	0.5111 15	0.3592 23

ContraRefer		0.7832 15	0.6801 12	0.3850 23	0.2947 18	0.4743 22	0.3811 14

D3Net		0.7923 11	0.6843 9	0.3905 22	0.3074 16	0.4806 21	0.3919 13
Dave Zhenyu Chen, Qirui Wu, Matthias Niessner, Angel X. Chang: D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Grounding. 17th European Conference on Computer Vision (ECCV), 2022
D3Net - Pretrained		0.7659 25	0.6579 17	0.3619 32	0.2726 27	0.4525 29	0.3590 24
Dave Zhenyu Chen, Qirui Wu, Matthias Niessner, Angel X. Chang: D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Grounding. 17th European Conference on Computer Vision (ECCV), 2022
SR-GAB		0.7016 34	0.5202 34	0.3233 41	0.1959 38	0.4081 39	0.2686 36

grounding		0.7298 31	0.5458 33	0.3822 25	0.2421 34	0.4538 28	0.3046 33

pairwisemethod		0.5779 42	0.3603 43	0.2792 46	0.1746 43	0.3462 45	0.2163 43

PointGroup_MCAN		0.7510 27	0.6397 18	0.3271 39	0.2535 30	0.4222 35	0.3401 29

3DJCG(Grounding)		0.7675 24	0.6059 22	0.4389 13	0.3117 14	0.5126 13	0.3776 17
Daigang Cai, Lichen Zhao, Jing Zhang†, Lu Sheng, Dong Xu: 3DJCG: A Unified Framework for Joint Dense Captioning and Visual Grounding on 3D Point Clouds. CVPR2022 Oral
3DVG-Transformer		0.7576 26	0.5515 31	0.4224 17	0.2933 20	0.4976 17	0.3512 27
Lichen Zhao∗, Daigang Cai∗, Lu Sheng†, Dong Xu: 3DVG-Transformer: Relation Modeling for Visual Grounding on Point Clouds. ICCV2021
3DVG-Trans +		0.7733 21	0.5787 27	0.4370 14	0.3102 15	0.5124 14	0.3704 18
Lichen Zhao∗, Daigang Cai∗, Lu Sheng†, Dong Xu: 3DVG-Transformer: Relation Modeling for Visual Grounding on Point Clouds. ICCV2021
SRGA		0.7494 29	0.5128 36	0.3631 31	0.2218 35	0.4497 31	0.2871 35

InstanceRefer		0.7782 18	0.6669 15	0.3457 37	0.2688 28	0.4427 32	0.3580 26
Zhihao Yuan, Xu Yan, Yinghong Liao, Ruimao Zhang, Zhen Li*, Shuguang Cui: InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referring. ICCV 2021
ScanRefer Baseline		0.6422 38	0.4196 41	0.3090 43	0.1832 40	0.3837 42	0.2362 41

TGNN		0.6834 36	0.5894 24	0.3312 38	0.2526 31	0.4102 38	0.3281 31
Pin-Hao Huang, Han-Hung Lee, Hwann-Tzong Chen, Tyng-Luh Liu: Text-Guided Graph Neural Network for Referring 3D Instance Segmentation. AAAI 2021
SAVG		0.7758 20	0.5664 28	0.4236 16	0.2826 25	0.5026 16	0.3462 28

henet		0.7110 33	0.5180 35	0.3936 20	0.2472 33	0.4590 23	0.3030 34

M3DRef-test		0.7865 12	0.6793 13	0.4963 3	0.3977 3	0.5614 4	0.4608 4

Co3d3		0.5326 45	0.1369 45	0.3848 24	0.1651 45	0.4179 36	0.1588 45

RG-SAN		0.7964 10	0.6785 14	0.4591 9	0.3600 10	0.5348 11	0.4314 10

SAF		0.6348 39	0.5647 29	0.3726 27	0.3009 17	0.4314 33	0.3601 22

M3DRef-SCLIP		0.7997 8	0.7123 2	0.4708 7	0.3805 8	0.5445 7	0.4549 6

cus3d		0.8384 3	0.7073 5	0.4908 4	0.4000 2	0.5688 3	0.4689 2

pointclip		0.8211 4	0.7082 4	0.4803 5	0.3884 5	0.5567 6	0.4601 5

Se2d		0.7799 16	0.6628 16	0.3636 30	0.2823 26	0.4569 24	0.3677 20

secg		0.7288 32	0.6175 21	0.3696 29	0.2933 20	0.4501 30	0.3660 21

CORE-3DVG		0.8557 2	0.6867 8	0.5275 1	0.3850 6	0.6011 1	0.4527 8

bo3d-1		0.7469 30	0.5606 30	0.4539 12	0.3124 13	0.5196 12	0.3680 19

bo3d0		0.4823 47	0.1278 46	0.3271 39	0.1394 47	0.3619 44	0.1368 47

SPANet		0.5614 43	0.4641 38	0.2800 45	0.2071 37	0.3431 46	0.2647 37

bo3d		0.5400 44	0.1550 44	0.3817 26	0.1785 41	0.4172 37	0.1732 44

Co3d2		0.5070 46	0.1195 47	0.3569 35	0.1511 46	0.3906 40	0.1440 46

Co3d		0.0000 48	0.0000 48	0.0000 48	0.0000 48	0.0000 48	0.0000 48

3DInsVG		0.8170 6	0.6925 6	0.4582 10	0.3617 9	0.5386 9	0.4359 9

M3DRef-CLIP		0.7980 9	0.7085 3	0.4692 8	0.3807 7	0.5433 8	0.4545 7
Yiming Zhang, ZeMing Gong, Angel X. Chang: Multi3DRefer: Grounding Text Description to Multiple 3D Objects. ICCV 2023
ConcreteNet		0.8607 1	0.7923 1	0.4746 6	0.4091 1	0.5612 5	0.4950 1
Ozan Unal, Christos Sakaridis, Suman Saha, Fisher Yu, Luc Van Gool: Three Ways to Improve Verbo-visual Fusion for Dense 3D Visual Grounding.
scanrefer2		0.6340 40	0.4353 39	0.3193 42	0.1947 39	0.3898 41	0.2486 39

CSA-M3LM		0.8137 7	0.6241 20	0.4544 11	0.3317 12	0.5349 10	0.3972 12

ScanRefer_vanilla		0.6488 37	0.4056 42	0.3052 44	0.1782 42	0.3823 43	0.2292 42

HAM		0.7799 16	0.6373 19	0.4148 18	0.3324 11	0.4967 18	0.4007 11
Jiaming Chen, Weixin Luo, Ran Song, Xiaolin Wei, Lin Ma, Wei Zhang: Learning Point-Language Hierarchical Alignment for 3D Visual Grounding.
ScanRefer		0.6859 35	0.4353 39	0.3488 36	0.2097 36	0.4244 34	0.2603 38
Dave Zhenyu Chen, Angel X. Chang, Matthias Nießner: ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language. 16th European Conference on Computer Vision (ECCV), 2020

		Captioning F1-Score				Dense Captioning	Object Detection
Method	Info	CIDEr@0.5IoU	BLEU-4@0.5IoU	Rouge-L@0.5IoU	METEOR@0.5IoU	DCmAP	mAP@0.5
Method	Info
Vote2Cap-DETR++		0.3360 1	0.1908 1	0.3012 1	0.1386 1	0.1864 1	0.5090 1
Sijin Chen, Hongyuan Zhu, Mingsheng Li, Xin Chen, Peng Guo, Yinjie Lei, Gang Yu, Taihao Li, Tao Chen: Vote2Cap-DETR++: Decoupling Localization and Describing for End-to-End 3D Dense Captioning.
TMP		0.3029 3	0.1728 3	0.2898 2	0.1332 2	0.1801 3	0.4605 2

vote2cap-detr		0.3128 2	0.1778 2	0.2842 3	0.1316 3	0.1825 2	0.4454 3
Sijin Chen, Hongyuan Zhu, Xin Chen, Yinjie Lei, Tao Chen, Gang YU, Taihao Li: End-to-End 3D Dense Captioning with Vote2Cap-DETR. CVPR 2023
CFM		0.2360 4	0.1417 4	0.2253 4	0.1034 4	0.1379 7	0.3008 7

Forest-xyz		0.2266 6	0.1363 6	0.2250 5	0.1027 6	0.1161 12	0.2825 12

CM3D-Trans+		0.2348 5	0.1383 5	0.2250 6	0.1030 5	0.1398 6	0.2966 9
Yufeng Zhong, Long Xu, Jiebo Luo, Lin Ma: Contextual Modeling for 3D Dense Captioning on Point Clouds.
D3Net - Speaker		0.2088 7	0.1335 8	0.2237 7	0.1022 7	0.1481 5	0.4198 4
Dave Zhenyu Chen, Qirui Wu, Matthias Niessner, Angel X. Chang: D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Grounding. 17th European Conference on Computer Vision (ECCV), 2022
3DJCG(Captioning)		0.1918 8	0.1350 7	0.2207 8	0.1013 8	0.1506 4	0.3867 5
Daigang Cai, Lichen Zhao, Jing Zhang†, Lu Sheng, Dong Xu: 3DJCG: A Unified Framework for Joint Dense Captioning and Visual Grounding on 3D Point Clouds. CVPR2022 Oral
REMAN		0.1662 9	0.1070 9	0.1790 9	0.0815 9	0.1235 10	0.2927 11

NOAH		0.1382 10	0.0901 10	0.1598 10	0.0747 10	0.1359 8	0.2977 8

SpaCap3D		0.1359 11	0.0883 11	0.1591 11	0.0738 11	0.1182 11	0.3275 6
Heng Wang, Chaoyi Zhang, Jianhui Yu, Weidong Cai: Spatiality-guided Transformer for 3D Dense Captioning on Point Clouds. the 31st International Joint Conference on Artificial Intelligence (IJCAI), 2022
SUN+		0.1148 14	0.0846 12	0.1564 12	0.0711 12	0.1143 13	0.2958 10

X-Trans2Cap		0.1274 12	0.0808 13	0.1392 13	0.0653 13	0.1244 9	0.2795 13
Yuan, Zhihao and Yan, Xu and Liao, Yinghong and Guo, Yao and Li, Guanbin and Cui, Shuguang and Li, Zhen: X-Trans2Cap: Cross-Modal Knowledge Transfer Using Transformer for 3D Dense Captioning. CVPR 2022
MORE-xyz		0.1239 13	0.0796 14	0.1362 14	0.0631 14	0.1116 14	0.2648 14
Yang Jiao, Shaoxiang Chen, Zequn Jie, Jingjing Chen, Lin Ma, Yu-Gang Jiang: MORE: Multi_ORder RElation Mining for Dense Captioning in 3D Scenes. ECCV 2022
Scan2Cap		0.0849 15	0.0576 15	0.1073 15	0.0492 15	0.0970 15	0.2481 15
Dave Zhenyu Chen, Ali Gholami, Matthias Nießner and Angel X. Chang: Scan2Cap: Context-aware Dense Captioning in RGB-D Scans. CVPR 2021

ScanRefer Benchmark

Scan2Cap Benchmark