We’re trying to understand and improve the relation of language abilities across different languages in LLMs. It is important to improve the language ability for Chinese and relatively low resource languages. And one idea way is to align different languages altogether, which may happen or be performed during the pretraining, continue pretraining, or post training phase. In the end, we aim at building LLMs that are equally effective for all languages.
(mostly in Chinese, hosted on bilibili.com)
Multilingual LLM Research from an MT perspective (20 mins, video). Invited talk at CCMT-2024. 2024-11.
Application of the Multilingual Ability of Large Language Models (50 mins, video). Invited talk at HIT-SZ (later at Huawei, CLLM2024), 2024-05.
Exploring the Multilingual Ability of Large Language Models (60 mins, video). Invited talk at Bytedance. 2023-08.
Research and Challenges of Multilingual Large Language Models (50 mins, video, slides). Tutorial Talk at NLPCC2024. 2024-10.
Research Development of Multilingualism in the Era of LLM (15 mins, video). Invited talk at the Frontier Research Overview session, CCL2024. 2024-07.
A More Complete List on Google Scholar
* marks corresponding author(s).
2025
Could Thinking Multilingually Empower LLM Reasoning?.
Changjiang Gao, Xu Huang, Wenhao Zhu, Shujian Huang*, Lei Li, Fei Yuan*.
preprint. (arxiv:2504.11833)
Understanding LLMs’ Cross-Lingual Context Retrieval: How Good It Is And Where It Comes From.
Changjiang Gao, Hankun Lin, Shujian Huang*, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Jiajun Chen.
preprint. (arxiv:2504.10906, code)
Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Training.
Zhijun Wang, Jiahuan Li, Hao Zhou, Rongxiang Weng, Jingang Wang, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Shujian Huang*.
Findings of ACL 2025. (arxiv:2504.01801, code)
BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models.
Xu Huang, Wenhao Zhu, Hanxu Hu, Conghui He, Lei Li, Shujian Huang*, Fei Yuan*.
preprint. (arxiv:2502.07346)
Large Language Models Are Cross-Lingual Knowledge-Free Reasoners.
Peng Hu, Sizhe Liu, Changjiang Gao, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Shujian Huang*.
NAACL 2025. (arxiv:2406.16655, video, code)
MoE-LPR: Multilingual Extension of Large Language Models through Mixture-of-Experts with Language Priors Routing.
Hao Zhou, Zhijun Wang, Shujian Huang*, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Weihua Luo, Jiajun Chen.
AAAI 2025. (arxiv:2408.11396, code)
2024
PreAlign: Boosting Cross-Lingual Transfer by Early Establishment of Multilingual Alignment.
Jiahuan Li, Shujian Huang*, Xinyu Dai, Jiajun Chen.
EMNLP 2024. (arxiv:2407.16222, video, code)
Multilingual Contrastive Decoding via Language-Agnostic Layers Skipping.
Wenhao Zhu, Sizhe Liu, Shujian Huang*, Shuaijie She, Chris Wendler, Jiajun Chen.
Findings of EMNLP 2024. (arxiv:2407.10795, code)
Why Not Transform Chat Large Language Models to Non-English?.
Xiang Geng, Ming Zhu, Jiahuan Li, Zhejian Lai, Wei Zou, Shuaijie She, Jiaxin Guo, Xiaofeng Zhao, Yinglu Li, Yuang Li, Chang Su, Yanqing Zhao, Xinglin Lyu, Min Zhang, Jiajun Chen, Hao Yang, Shujian Huang*.
preprint. (arxiv:2405.13923, code)
Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners.
Shimao Zhang, Changjiang Gao, Wenhao Zhu, Jiajun Chen, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Shujian Huang*.
EMNLP 2024. (arxiv:2405.13816)
The Power of Question Translation Training in Multilingual Reasoning: Broadened Scope and Deepened Insights.
Wenhao Zhu, Shujian Huang*, Fei Yuan, Jiajun Chen, Alexandra Birch.
preprint. (arxiv:2405.01345, code)
Question Translation Training for Better Multilingual Reasoning.
Wenhao Zhu, Shujian Huang*, Fei Yuan, Shuaijie She, Jiajun Chen, Alexandra Birch.
Findings of ACL 2024. (arxiv:2401.07817, code)
MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization.
Shuaijie She, Wei Zou, Shujian Huang*, Wenhao Zhu, Xiang Liu, Xiang Geng, Jiajun Chen.
ACL 2024. (arxiv:2401.06838, code)
Multilingual Pretraining and Instruction Tuning Improve Cross-Lingual Knowledge Alignment, But Only Shallowly.
Changjiang Gao, Hongda Hu, Peng Hu, Jiajun Chen, Jixing Li, Shujian Huang*.
NAACL 2024. (arxiv:2403.04325)
2023
Extrapolating Large Language Models to Non-English by Aligning Languages.
Wenhao Zhu, Yunzhe Lv, Qingxiu Dong, Fei Yuan, Jingjing Xu, Shujian Huang*, Lingpeng Kong, Jiajun Chen, Lei Li.
preprint. (arxiv:2308.04948, video, code)