We're trying to understand the abilities of LLMs, including the abilities for language comprehension, in-context learning, knowledge learning, logic reasoning, long-cot reasoning, safety and alignment, as well as the ability of working as intelligent agents for a given task. It is interesting and also important to know how these abilities are obtained, how they are applied to specific tasks and how to improve them if needed.
(mostly in Chinese, hosted on bilibili.com)
A More Complete List on Google Scholar
* marks corresponding author(s).
2025
Internal Bias in Reasoning Models leads to Overthinking.
Renfei Dang, Shujian Huang*, Jiajun Chen.
preprint. (arxiv:2505.16448, code)
Why Not Act on What You Know? Unleashing Safety Potential of LLMs via Self-Aware Guard Enhancement.
Peng Ding, Jun Kuang, Zongyu Wang, Xuezhi Cao, Xunliang Cai, Jiajun Chen, Shujian Huang*.
Findings of ACL 2025. (arxiv:2505.12060, code)
R-PRM: Reasoning-Driven Process Reward Modeling.
Shuaijie She, Junxiao Liu, Yifeng Liu, Jiajun Chen, Xin Huang, Shujian Huang*.
preprint. (arxiv:2503.21295, code)
Process-based Self-Rewarding Language Models.
Shimao Zhang, Xiao Liu*, Xin Zhang, Junxiao Liu, Zheheng Luo, Shujian Huang*, Yeyun Gong.
Findings of ACL 2025. (arxiv:2503.03746)
Generalizing From Short to Long: Effective Data Synthesis for Long-Context Instruction Tuning.
Wenhao Zhu, Pinzhen Chen, Hanxu Hu, Shujian Huang*, Fei Yuan, Jiajun Chen, Alexandra Birch.
preprint. (arxiv:2502.15592, code)
Multi-Candidate Speculative Decoding.
Sen Yang, Shujian Huang*, Xinyu Dai, Jiajun Chen.
NLPCC 2025. (arxiv:2401.06706, code)
2024
Formality is Favored: Unraveling the Learning Preferences of Large Language Models on Data with Conflicting Knowledge.
Jiahuan Li, Yiqing Cao, Shujian Huang*, Jiajun Chen.
EMNLP 2024. Outstanding Paper Award (5+20/1271/6105). (arxiv:2410.04784, video, code)
Hallu-PI: Evaluating Hallucination in Multi-modal Large Language Models within Perturbed Inputs.
Peng Ding, Jingyu Wu, Jun Kuang, Dan Ma, Xuezhi Cao, Xunliang Cai, Shi Chen, Jiajun Chen, Shujian Huang*.
ACM MM 2024. (arxiv:2408.01355, code)
Large Language Models are Limited in Out-of-Context Knowledge Reasoning.
Peng Hu, Changjiang Gao, Ruiqi Gao, Jiajun Chen, Shujian Huang*.
Findings of EMNLP 2024. (arxiv:2406.07393, code)
Measuring Meaning Composition in the Human Brain with Composition Scores from Large Language Models.
Changjiang Gao, Jixing Li*, Jiajun Chen, Shujian Huang*.
ACL 2024. (arxiv:2403.04325, code)
A Wolf in Sheep’s Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily.
Peng Ding, Jun Kuang, Dan Ma, Xuezhi Cao, Yunsen Xian, Jiajun Chen, Shujian Huang*.
NAACL 2024. (arxiv:2311.08268, code)
Exploring the Factual Consistency in Dialogue Comprehension of Large Language Models.
Shuaijie She, Shujian Huang*, Xingyun Wang, Yanke Zhou, Jiajun Chen.
NAACL 2024. (arxiv:2311.07194, code)
2023
Roles of Scaling and Instruction Tuning in Language Perception: Model vs. Human Attention.
Changjiang Gao, Shujian Huang*, Jixing Li, Jiajun Chen.
Findings of EMNLP 2023.
Food-500 Cap: A Fine-Grained Food Caption Benchmark for Evaluating Vision-Language Models.
Zheng Ma, Mianzhi Pan, Wenhan Wu, Kanzhi Cheng, Jianbing Zhang*, Shujian Huang*, and Jiajun Chen.
ACM MM 2023.
Local Interpretation of Transformer Based on Linear Decomposition.
Sen Yang, Shujian Huang*, Wei Zou, Jianbing Zhang, Xinyu Dai and Jiajun CHEN.
ACL 2023. (video, code)
CoP: Factual Inconsistency Detection by Controlling the Preference.
Shuaijie She, Xiang Geng, Shujian Huang*, Jiajun Chen.
AAAI 2023. (video, code)