On the token distance modeling ability of higher RoPE attention dimension

On the token distance modeling ability of higher RoPE attention dimension Xiangyu Hong author Che Jiang author Biqing Qi author Fandong Meng author Mo Yu author Bowen Zhou author Jie Zhou author 2024-11 text Findings of the Association for Computational Linguistics: EMNLP 2024 Yaser Al-Onaizan editor Mohit Bansal editor Yun-Nung Chen editor Association for Computational Linguistics Miami, Florida, USA conference publication hong-etal-2024-token 10.18653/v1/2024.findings-emnlp.338 https://aclanthology.org/2024.findings-emnlp.338/ 2024-11 5877 5888