本页保存的是公开项目资料快照，阅读过程不需要连接 GitHub。

Awesome Reasoning Generalization

Beyond the Training Distribution: A Survey of Reasoning Generalization in Large Language Models

图片：Awesome 图片：Core papers 图片：2024 papers 图片：2025 papers 图片：2026 papers 图片：GitHub last commit

Status: The literature search was updated on 10 September 2026. The corpus contains 145 core studies: 86 from 2026, 39 from 2025, 14 from 2024, and 6 earlier foundations.

Taxonomy
Paper list
- Training for Generalization
  - Pre-training and mid-training
  - Post-training: supervised adaptation
  - Post-training: RL and reward design
  - Post-training: hybrid and self-improvement
- Inference for Generalization
  - Prompt elicitation and decomposition
  - Sampling and search
  - Verification, repair, and stopping
  - Tools and memory
- Architecture for Generalization
  - Recurrent depth and adaptive computation
  - Position, attention, and locality
  - Modules, symbols, and structured state
  - Latent recurrence and equilibrium computation
- Analysis of Generalization
  - Behavioral observations
  - Empirical mechanisms and explanations
  - Theoretical analysis
Citation

Taxonomy

Pillar	Central question
Training for Generalization	How can learning produce reasoning skills that transfer to unfamiliar problems?
Inference for Generalization	How can a fixed model solve unfamiliar problems through changes at inference?
Architecture for Generalization	Which structural properties support transferable reasoning?
Analysis of Generalization	When does reasoning generalize, and what explains its successes and failures?

The first three pillars concern interventions. The fourth separates behavioral observations from empirical mechanisms and theoretical results. Each paper receives one primary manifest label, even when it informs several sections.

Paper list

Training for Generalization

Pre-training and mid-training

What You Can’t See Is What You Learn: Slot-Selective Evidence Masking Favors Compositional Generalization in Shared-Genome Language-Model Societies. Narcis Marincat. [Paper] [PDF]
Selective Left-Shift: Turning Test-Time Compute and Difficulty-based Curation into Training Data for Low-Resource Code Generation. Didula Samaraweera, Anjana Supun, Srinath Perera. [Paper] [PDF]
Randomized YaRN Improves Length Generalization for Long-Context Reasoning. Manas Mehta, Fangcong Yin, Greg Durrett. [Paper] [PDF]
Rethinking Easy-to-Hard: Limits of Curriculum Learning in Post-Training for Deductive Reasoning. Maximilian Mordig, Andreas Opedal, Weiyang Liu, Bernhard Schölkopf. [Paper] [PDF]
Learning from Synthetic Data Improves Multi-hop Reasoning. Anmol Kabra, Yilun Yin, Albert Gong, Kamilė Stankevičiūtė, Dongyoung Go, Johann Lee, Katie Z. Luo, Carla P. Gomes, Kilian Q. Weinberger. [Paper] [PDF]
Fundamental Reasoning Paradigms Induce Out-of-Domain Generalization in Language Models. Mingzi Cao, Xingwei Tan, Mahmud Elahi Akhter, Marco Valentino, Maria Liakata, Xi Wang, Nikolaos Aletras. [Paper] [PDF]
The Imitation Game: Turing Machine Imitator is Length Generalizable Reasoner. Zhouqi Hua, Wenwei Zhang, Chengqi Lyu, Yuzhe Gu, Songyang Gao, Kuikun Liu, Dahua Lin, Kai Chen. [Paper] [PDF]
Beyond Single-Task: Robust Multi-Task Length Generalization for LLMs. Yi Hu, Shijia Kang, Haotong Yang, Haotian Xu, Muhan Zhang. [Paper] [PDF]
Complexity Control Facilitates Reasoning-Based Compositional Generalization in Transformers. Zhongwang Zhang, Pengxiao Lin, Zhiwei Wang, Yaoyu Zhang, Zhi-Qin John Xu. [Paper] [PDF]

Post-training: supervised adaptation

Every Coin Has Two Sides: On the Dual Nature of Generalization in On-Policy Distillation of Large Language Models. Zhaoyi Li, Deyang Kong, Yuan Wei, Evan Yang, Ranran Shen, Mahardika Krisna Ihsani, Ming Yang, Wei Zhang, Chuan Hao, Jian Yang, Ran Tao, Bryan Dai, Shikun Zhang, Wei Ye, Ying Wei, Defu Lian. [Paper] [PDF]
RP-OPSD: Reasoning-Pivot-Guided On-Policy Self-Distillation for Multilingual Reasoning Transfer. Xinye Wang, Junxiao Liu, Shujian Huang. [Paper] [PDF]
Geometric Self-Distillation for Reasoning Generalization. Josip Jukić, Ivan Titov. [Paper] [PDF]
Invariant Gradient Alignment for Robust Reasoning Distillation. Zehua Cheng, Wei Dai, Jiahao Sun. [Paper] [PDF]
Learning to Adapt SFT Data for Better Reasoning Generalization. Lisong Sun, Li Wang, Chen Zhang, Jinyang Wu, Kui Zhang, Tianhao Peng, Wenjun Wu. [Paper] [PDF]
Memorize Theorems, Not Instances: Probing SFT Generalization through Mathematical Reasoning. Ruiying Peng, Mengyu Yang, Jing Lei, Xiaohui Li, Xueyu Wu, Xinlei Chen. [Paper] [PDF]
Making Expert Reasoning Learnable with Self-Distillation. Ethan Mendes, Jungsoo Park, Alan Ritter. [Paper] [PDF]
Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalization. Xueyun Tian, Minghua Ma, Bingbing Xu, Nuoyan Lyu, Wei Li, Heng Dong, Zheng Chu, Yuanzhuo Wang, Huawei Shen. [Paper] [PDF]
Mitigating Spurious Correlations in LLMs via Causality-Aware Post-Training. Shurui Gui, Shuiwang Ji. [Paper] [PDF]
Learning Composable Chains-of-Thought. Fangcong Yin, Zeyu Leo Liu, Liu Leqi, Xi Ye, Greg Durrett. [Paper] [PDF]

Post-training: RL and reward design

GRAIN: Bridging Name and Narrative Shifts in Real-World Graph Reasoning through Invariance-Rewarded Agentic RL. Zike Yuan, Han Zhang, Jianzhi Yan, Le Liu, Cai Ke, Huozhi Zhou, Jian Xie, Jiran Yin, Yukun Cao, Yue Yu, Hui Wang, Ming Liu, Bing Qin. [Paper] [PDF]
CEDAR-GRPO: Process-Aware Reinforcement Learning for General Abductive Reasoning in LLMs. Moein Salimi, Danial Parnian, Shaygan Adim, Amirmohammad Ebrahiminasab, Nima Alighardashi, Parsa Gholami, Sahand Akramipour, Mahdi Jafari Siavoshani, Mohammad Hossein Rohban. [Paper] [PDF]
Transferability for General Reasoning: An Automated Curriculum for Multi-Domain RLVR. Yongjin Yang, Jiarui Liu, Yinghui He, Lechen Zhang, Bernhard Schölkopf, Zhijing Jin. [Paper] [PDF]
Verifiable Environments Are LEGO Bricks: Recursive Composition for Reasoning Generalization. Hao Xiang, Qiaoyu Tang, Le Yu, Yaojie Lu, Xianpei Han, Ben He, Le Sun, Bowen Yu, Peng Wang, Hongyu Lin, Dayiheng Liu. [Paper] [PDF]
Exploration-Driven Optimization for Test-Time Large Language Model Reasoning. Changhao Li, Yuchen Zhuang, Chenxiao Gao, Haotian Sun, Rushi Qiang, Chao Zhang, Bo Dai. [Paper] [PDF]
Rubric-Grounded RL: Structured Judge Rewards for Generalizable Reasoning. Manish Bhattarai, Ismael Boureima, Nishath Rajiv Ranasinghe, Scott Pakin, Dan O’Malley. [Paper] [PDF]
SUPERNOVA: Eliciting General Reasoning in LLMs with Reinforcement Learning on Natural Instructions. Ashima Suvarna, Kendrick Phan, Mehrab Beikzadeh, Hritik Bansal, Saadia Gabriel. [Paper] [PDF]
Can LLMs Learn to Reason Robustly under Noisy Supervision?. Shenzhi Yang, Guangcheng Zhu, Bowen Song, Sharon Li, Haobo Wang, Xing Zheng, Yingfan Ma, Zhongqi Chen, Weiqiang Wang, Gang Chen. [Paper] [PDF]
Towards Generalizable Reasoning: Group Causal Counterfactual Policy Optimization for LLM Reasoning. Jingyao Wang, Peizheng Guo, Wenwen Qiang, Jiahuan Zhou, Huijie Guo, Changwen Zheng, Hui Xiong. [Paper] [PDF]
GraphDancer: Training LLMs to Explore and Reason over Graphs via Two-Stage Curriculum Post-Training. Yuyang Bai, Zhuofeng Li, Ping Nie, Jianwen Xie, Yu Zhang. [Paper] [PDF]
Sharpness-Guided Group Relative Policy Optimization via Probability Shaping. Tue Le, Linh Ngo Van, Trung Le. [Paper] [PDF]
Can GRPO Help LLMs Transcend Their Pretraining Origin?. Kangqi Ni, Zhen Tan, Zijie Liu, Pingzhi Li, Tianlong Chen. [Paper] [PDF]
From $f(x)$ and $g(x)$ to $f(g(x))$: LLMs Learn New Skills in RL by Composing Old Ones. Lifan Yuan, Weize Chen, Yuchen Zhang, Ganqu Cui, Hanbin Wang, Ziming You, Ning Ding, Zhiyuan Liu, Maosong Sun, Hao Peng. [Paper] [PDF]
Can One Domain Help Others? A Data-Centric Study on Multi-Domain Reasoning via Reinforcement Learning. Yu Li, Zhuoshi Pan, Honglin Lin, Mengyuan Sun, Conghui He, Lijun Wu. [Paper] [PDF]
LogicPuzzleRL: Cultivating Robust Mathematical Reasoning in LLMs via Reinforcement Learning. Zhen Hao Wong, Jingwen Deng, Runming He, Zirong Chen, Qijie You, Hejun Dong, Hao Liang, Chengyu Shen, Bin Cui, Wentao Zhang. [Paper] [PDF]
X-Reasoner: Towards Generalizable Reasoning Across Modalities and Domains. Qianchu Liu, Sheng Zhang, Guanghui Qin, Timothy Ossowski, Yu Gu, Ying Jin, Sid Kiblawi, Sam Preston, Mu Wei, Paul Vozila, Tristan Naumann, Hoifung Poon. [Paper] [PDF]
Rewarding Graph Reasoning Process makes LLMs more Generalized Reasoners. Miao Peng, Nuo Chen, Zongrui Suo, Jia Li. [Paper] [PDF]

Post-training: hybrid and self-improvement

Training Language Models to Cooperate with Inference-Time Controllers. Moumita Choudhury, Vanshaj Khattar, Jing Liu, Toshiaki Koike-Akino, Ankush Chakrabarty, Shlomo Zilberstein, Ye Wang. [Paper] [PDF]
When RL Fails after SFT: Rejuvenating Model Plasticity for Robust SFT-to-RL Handoff. Runze Liu, Jiashun Liu, Xu Wan, Yuqian Fu, Ling Pan. [Paper] [PDF]
Stratagem: Learning Transferable Reasoning via Trajectory-Modulated Game Self-Play. Xiachong Feng, Deyi Yin, Xiaocheng Feng, Yi Jiang, Libo Qin, Yangfan Ye, Lei Huang, Weitao Ma, Qiming Li, Yuxuan Gu, Bing Qin, Lingpeng Kong. [Paper] [PDF]
Bridging SFT and RL: Dynamic Policy Optimization for Robust Reasoning. Taojie Zhu, Dongyang Xu, Ding Zou, Sen Zhao, Qiaobo Hao, Zhiguo Yang, Yonghong He. [Paper] [PDF]
Why Does RL Generalize Better Than SFT? A Data-Centric Perspective on VLM Post-Training. Aojun Lu, Tao Feng, Hangjie Yuan, Wei Li, Yanan Sun. [Paper] [PDF]
From Meta-Thought to Execution: Cognitively Aligned Post-Training for Generalizable and Reliable LLM Reasoning. Shaojie Wang, Liang Zhang. [Paper] [PDF]
Towards Compositional Generalization of LLMs via Skill Taxonomy Guided Data Synthesis. Yifan Wei, Li Du, Xiaoyan Yu, Yang Feng, Angsheng Li. [Paper] [PDF]
Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges. Nayoung Lee, Ziyang Cai, Avi Schwarzschild, Kangwook Lee, Dimitris Papailiopoulos. [Paper] [PDF]

Inference for Generalization

Prompt elicitation and decomposition

Constraint-First Reasoning: A Training-Free Protocol for Exploiting Answer-Space Constraints in Mathematical Problem Solving. Hongbo Ma, Bangji Yang, Yunqian Selina Cheng, Jiajun Fan, Hanwen Zhang, Ge Liu. [Paper] [PDF]
Test-Time Hinting for Black-Box Vision-Language Models. Kaihua Hou, Abhijith Varma Mudunuri, Jiaxing Qiu, Roxana Daneshjou, Thomas Hartvigsen, Ahmed Alaa. [Paper] [PDF]
Least-to-Most Prompting Enables Complex Reasoning in Large Language Models. Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, Ed Chi. [Paper] [PDF]

Sampling and search

Test-Time Scaling via Error Localization. Rajiv Shailesh Chitale, Rahul Madhavan, Taneesh Gupta, Deepanway Ghosal, Aravindan Raghuveer. [Paper] [PDF]
Improving Latent Generalization Using Test-time Compute. Arslan Chaudhry, Sridhar Thiagarajan, Andrew Lampinen. [Paper] [PDF]

Verification, repair, and stopping

Reasoning Errors Have a Region and a Direction in the Residual-Stream Trajectory of LLMs. Hamed Damirchi, Ignacio Meza De la Jara, Damith Ranasinghe, Yuhang Liu, Javen Shi. [Paper] [PDF]
UPAIR: Diagnosing Reasoning States via Uncertainty-Progress Alignment for Selective Intervention. Cheng Yan, Zhijun Fan, Guangyang Ye, Fan Xu, Xiang Xia, Yawei Wang, Wuyang Zhang. [Paper] [PDF]

Tools and memory

MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning. Ruilin Tong, Dong Gong. [Paper] [PDF]
To Infinity and Beyond: Tool-Use Unlocks Length Generalization in State Space Models. Eran Malach, Omid Saremi, Sinead Williamson, Arwen Bradley, Aryo Lotfi, Emmanuel Abbe, Josh Susskind, Etai Littwin. [Paper] [PDF]
ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory. Siru Ouyang, Jun Yan, I-Hung Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T. Le, Samira Daruki, Xiangru Tang, Vishy Tirumalashetty, George Lee, Mahsan Rofouei, Hangfei Lin, Jiawei Han, Chen-Yu Lee, Tomas Pfister. [Paper] [PDF]

Architecture for Generalization

Recurrent depth and adaptive computation

Universal Transformers for Circuit Computations: Perfect Length Generalization in Tiny Transformers. Takuya Ito, Ruchir Puri, Murray Campbell, Parikshit Ram. [Paper] [PDF]
Think Shallow, Solve Deep: Controlling Recurrent Dynamics for Reliable Test-Time Depth. Ivan Viakhirev, Kirill Borodin, Amirah Almutairi, Serguei Barannikov, Maxim Abramov, Grach Mkrtchian. [Paper] [PDF]
Stabilizing Extrapolation in Looped Transformers via Learned Stochastic Stopping. Hsun-Yu Kuo, El Mahdi Chayti, Patrik Reizinger, Wieland Brendel, Martin Jaggi. [Paper] [PDF]
Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers. Harsh Kohli, Srinivasan Parthasarathy, Huan Sun, Yuekun Yao. [Paper] [PDF]
Thinking Deeper, Not Longer: Depth-Recurrent Transformers for Compositional Generalization. Hung-Hsuan Chen. [Paper] [PDF]
Enhancing Auto-regressive Chain-of-Thought through Loop-Aligned Reasoning. Qifan Yu, Zhenyu He, Sijie Li, Xun Zhou, Jun Zhang, Jingjing Xu, Di He. [Paper] [PDF]
Looped Transformers for Length Generalization. Ying Fan, Yilun Du, Kannan Ramchandran, Kangwook Lee. [Paper] [PDF]
Universal Transformers. Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, Łukasz Kaiser. [Paper] [PDF]

Position, attention, and locality

On Locality and Length Generalization in Visual Reasoning. Pulkit Madan, Sanjay Haresh, Reza Ebrahimi, Sunny Panchal, Apratim Bhattacharyya, Roland Memisevic. [Paper] [PDF]
How Data Shapes RoPE Frequency Usage: From Positional Scale Matching to Length Generalization. Xinyi Wu, Siyuan Liu, Ali Jadbabaie. [Paper] [PDF]
Position Encoding with Random Float Sampling Enhances Length Generalization of Transformers. Atsushi Shimizu, Shohei Taniguchi, Yutaka Matsuo. [Paper] [PDF]
Explicitly Encoding Structural Symmetry is Key to Length Generalization in Arithmetic Tasks. Mahdi Sabbaghi, George Pappas, Hamed Hassani, Surbhi Goel. [Paper] [PDF]
Position Coupling: Improving Length Generalization of Arithmetic Transformers Using Task Structure. Hanseul Cho, Jaeyoung Cha, Pranjal Awasthi, Srinadh Bhojanapalli, Anupam Gupta, Chulhee Yun. [Paper] [PDF]

Modules, symbols, and structured state

Looped Language Models Improve Compositional Tool Calling. Andrei Cristian Popescu, Haitz Sáez de Ocáriz Borde, Pietro Liò. [Paper] [PDF]
AGEL-Comp: A Neuro-Symbolic Framework for Compositional Generalization in Interactive Agents. Mahnoor Shahid, Hannes Rothe. [Paper] [PDF]
Barriers to Universal Reasoning With Transformers (And How to Overcome Them). Oliver Kraus, Yash Sarrof, Yuekun Yao, Alexander Koller, Michael Hahn. [Paper] [PDF]
Rational Transductors. Mehryar Mohri. [Paper] [PDF]
Arithmetic Transformers Can Length-Generalize in Both Operand Length and Count. Hanseul Cho, Jaeyoung Cha, Srinadh Bhojanapalli, Chulhee Yun. [Paper] [PDF]

Latent recurrence and equilibrium computation

Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning. Benhao Huang, Zhengyang Geng, Zico Kolter. [Paper] [PDF]
Generalizable Reasoning through Compositional Energy Minimization. Alexandru Oarga, Yilun Du. [Paper] [PDF]
Unlocking Out-of-Distribution Generalization in Transformers via Recursive Latent Space Reasoning. Awni Altabaa, Siyu Chen, John Lafferty, Zhuoran Yang. [Paper] [PDF]

Analysis of Generalization

Behavioral observations

LLMs Can See the Smoke but not the Fire: Evaluating Abductive Reasoning with Elenchos. Julius Steiglechner, Lucas Mahler, Gabriele Lohmann. [Paper] [PDF]
How Post-Training Shapes Biological Reasoning Models. Lukas Fesser, Hanlin Zhang, Michelle M. Li, Eric Wang, Bryan Perozzi, Shekoofeh Azizi, Sham M. Kakade, Marinka Zitnik. [Paper] [PDF]
Testing LLM Arithmetic Reasoning Generalization with Automatic Numeric-Remapping Attacks. Malia Barker, Bishal Lakha, Edoardo Serra, Francesco Gullo. [Paper] [PDF]
Reasoners or Translators? Contamination-aware Evaluation and Neuro-Symbolic Robustness in Tax Law. Parisa Kordjamshidi, Samer Aslan, Madhavan Seshadri, Leslie Barrett, Enrico Santus. [Paper] [PDF]
XDomainBench: Diagnosing Reasoning Collapse in High-Dimensional Scientific Knowledge Composition. Gong Zhiren, Tiantong Wu, Jiaming Zhang, Fuyao Zhang, Che Wang, Yurong Hao, Yikun Hou, Foo Ping, Yilei Zhao, Fei Huang, Chau Yuen, Wei Yang Bryan Lim. [Paper] [PDF]
Generalization in LLM Problem Solving: The Case of the Shortest Path. Yao Tong, Jiayuan Ye, Anastasia Borovykh, Reza Shokri. [Paper] [PDF]
General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Tasks. Junlin Liu, Shengnan An, Shuang Zhou, Dan Ma, Shixiong Luo, Ying Xie, Yuan Zhang, Wenling Yuan, Yifan Zhou, Xiaoyu Li, Ziwen Wang, Xuezhi Cao, Xunliang Cai. [Paper] [PDF]
Rethinking Generalization in Reasoning SFT: A Conditional Analysis on Optimization, Data, and Model Capability. Qihan Ren, Peng Wang, Ruikun Cai, Shuai Shao, Dadi Guo, Yuejin Xie, Yafu Li, Quanshi Zhang, Xia Hu, Jing Shao, Dongrui Liu. [Paper] [PDF]
EsoLang-Bench: Evaluating Genuine Reasoning in Large Language Models via Esoteric Programming Languages. Aman Sharma, Paras Chopra. [Paper] [PDF]
On the Out-of-Distribution Generalization of Reasoning in Multimodal LLMs for Simple Visual Planning Tasks. Yannic Neuhaus, Nicolas Flammarion, Matthias Hein, Francesco Croce. [Paper] [PDF]
When Domains Interact: Asymmetric and Order-Sensitive Cross-Domain Effects in Reinforcement Learning for Reasoning. Wang Yang, Shouren Wang, Chaoda Song, Chuang Ma, Xinpeng Li, Nengbo Wang, Kaixiong Zhou, Vipin Chaudhary, Xiaotian Han. [Paper] [PDF]
Paying Less Generalization Tax: A Cross-Domain Generalization Study of RL Training for LLM Agents. Zhihan Liu, Lin Guan, Yixin Nie, Kai Zhang, Zhuoqun Hao, Lin Chen, Asli Celikyilmaz, Zhaoran Wang, Na Zhang. [Paper] [PDF]
On the Emergence and Test-Time Use of Structural Information in Large Language Models. Michelle Chao Chen, Moritz Miller, Bernhard Schölkopf, Siyuan Guo. [Paper] [PDF]
Disentangling generalization and memorization in large language models using chess. Leonard S. Pleiss, Maximilian Schiffer, Robert K. von Weizsaecker. [Paper] [PDF]
Beyond Memorization: Testing LLM Reasoning on Unseen Theory of Computation Tasks. Shlok Shelat, Jay Raval, Souvik Roy, Manas Gaur. [Paper] [PDF]
Generalization of RLVR Using Causal Reasoning as a Testbed. Brian Lu, Hongyu Zhao, Shuo Sun, Hao Peng, Rui Ding, Hongyuan Mei. [Paper] [PDF]
On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models. Charlie Zhang, Graham Neubig, Xiang Yue. [Paper] [PDF]
Exploring Depth Generalization in Large Language Models for Solving Recursive Logic Tasks. Zhiyuan He. [Paper] [PDF]
Revisiting Generalization Across Difficulty Levels: It’s Not So Easy. Yeganeh Kordi, Nihal V. Nayak, Max Zuo, Ilana Nguyen, Stephen H. Bach. [Paper] [PDF]
RL Grokking Recipe: How Does RL Unlock and Transfer New Algorithms in LLMs?. Yiyou Sun, Yuhan Cao, Pohao Huang, Haoyue Bai, Hannaneh Hajishirzi, Nouha Dziri, Dawn Song. [Paper] [PDF]
Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens. Chengshuai Zhao, Zhen Tan, Pingchuan Ma, Dawei Li, Bohan Jiang, Yancheng Wang, Yingzhen Yang, Huan Liu. [Paper] [PDF]
Breaking Barriers: Do Reinforcement Post Training Gains Transfer To Unseen Domains?. Chuxuan Hu, Yuxuan Zhu, Antony Kellermann, Caleb Biddulph, Suppakit Waiwitlikhit, Jason Benn, Daniel Kang. [Paper] [PDF]
OMEGA: Can LLMs Reason Outside the Box in Math? Evaluating Exploratory, Compositional, and Transformative Generalization. Yiyou Sun, Shawn Hu, Georgia Zhou, Ken Zheng, Hannaneh Hajishirzi, Nouha Dziri, Dawn Song. [Paper] [PDF]
Extrapolation by Association: Length Generalization Transfer in Transformers. Ziyang Cai, Nayoung Lee, Avi Schwarzschild, Samet Oymak, Dimitris Papailiopoulos. [Paper] [PDF]
Reinforcement Learning for Reasoning in Large Language Models with One Training Example. Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Liyuan Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, Simon Shaolei Du, Yelong Shen. [Paper] [PDF]
Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?. Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, Gao Huang. [Paper] [PDF]
Compositional-ARC: Assessing Systematic Generalization in Abstract Spatial Reasoning. Philipp Mondorf, Shijia Zhou, Monica Riedler, Barbara Plank. [Paper] [PDF]
SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training. Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V. Le, Sergey Levine, Yi Ma. [Paper] [PDF]
MathGAP: Out-of-Distribution Evaluation on Problems with Arbitrarily Complex Proofs. Andreas Opedal, Haruki Shirakami, Bernhard Schölkopf, Abulhair Saparov, Mrinmaya Sachan. [Paper] [PDF]
The Mystery of Compositional Generalization in Graph-based Generative Commonsense Reasoning. Xiyan Fu, Anette Frank. [Paper] [PDF]
GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models. Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, Mehrdad Farajtabar. [Paper] [PDF]
Rule Extrapolation in Language Models: A Study of Compositional Generalization on OOD Prompts. Anna Mészáros, Szilvia Ujváry, Wieland Brendel, Patrik Reizinger, Ferenc Huszár. [Paper] [PDF]
modeLing: A Novel Dataset for Testing Linguistic Reasoning in Language Models. Nathan A. Chi, Teodor Malchev, Riley Kong, Ryan A. Chi, Lucas Huang, Ethan A. Chi, R. Thomas McCoy, Dragomir Radev. [Paper] [PDF]
GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers. Qintong Li, Leyang Cui, Xueliang Zhao, Lingpeng Kong, Wei Bi. [Paper] [PDF]
Transformers Can Achieve Length Generalization But Not Robustly. Yongchao Zhou, Uri Alon, Xinyun Chen, Xuezhi Wang, Rishabh Agarwal, Denny Zhou. [Paper] [PDF]
Grokking of Hierarchical Structure in Vanilla Transformers. Shikhar Murty, Pratyusha Sharma, Jacob Andreas, Christopher D. Manning. [Paper] [PDF]
Exploring Length Generalization in Large Language Models. Cem Anil, Yuhuai Wu, Anders Andreassen, Aitor Lewkowycz, Vedant Misra, Vinay Ramasesh, Ambrose Slone, Guy Gur-Ari, Ethan Dyer, Behnam Neyshabur. [Paper] [PDF]
Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. Brenden M. Lake, Marco Baroni. [Paper] [PDF]

Empirical mechanisms and explanations

Shared circuits predict whether LLMs generalize across formats in arithmetic reasoning. Andrea Gregor de Varda, Sana Pandey, Pengrui Han, Jacob Andreas, Evelina Fedorenko. [Paper] [PDF]
Why Knowing Both Hops Is Not Enough: Understanding Two-Hop Generalization in Language Models. Zili Zhang, Yilin Wang, Heng Wang, Herun Wan, Minnan Luo. [Paper] [PDF]
Protoreasoning in Tiny Transformers. Eduardo Valle, Fergal Reid. [Paper] [PDF]
Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning. Lu Dai, Ziyang Rao, Yili Wang, Hanqing Wang, Hao Liu, Hui Xiong. [Paper] [PDF]
RL Post-Training Builds Compositional Reasoning Strategies. Azwar Abdulsalam, Nishil Patel, Andrew Saxe. [Paper] [PDF]
From Reasoning Traces to Reusable Modules: Understanding Compositional Generalization in Language Model Reasoning. Lingjing Kong, Xin Liu, Guangyi Chen, Martin Q. Ma, Xiangchen Song, Yuekai Sun, Mikhail Yurochkin, Taylor W. Killian, Ruslan Salakhutdinov, Kun Zhang, Eric P. Xing, Zhengzhong Liu. [Paper] [PDF]
When RL Suppresses Its Own Vocabulary: Recovering Reasoning Diversity in Puzzle-to-Math Transfer. Mayug Maniparambil, Arjun Karuvally, Terrence Sejnowski, Fergal Reid. [Paper] [PDF]
Critical Windows of Complexity Control: When Transformers Decide to Reason or Memorize. Sarwan Ali. [Paper] [PDF]
Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?. Jeonghye Kim, Xufang Luo, Minbeom Kim, Sangmook Lee, Dohyung Kim, Jiwon Jeon, Dongsheng Li, Yuqing Yang. [Paper] [PDF]
Early-Warning Signals of Grokking via Loss-Landscape Geometry. Yongzhong Xu. [Paper] [PDF]
Discovering Interpretable Algorithms by Decompiling Transformers to RASP. Xinting Huang, Aleksandra Bakalova, Satwik Bhattamishra, William Merrill, Michael Hahn. [Paper] [PDF]
Representational Homomorphism Predicts and Improves Compositional Generalization In Transformer Language Model. Zhiyu An, Wan Du. [Paper] [PDF]
Is Grokking Worthwhile? Functional Analysis and Transferability of Generalization Circuits in Transformers. Kaiyu He, Zhang Mian, Peilin Wu, Xinya Du, Zhiyu Chen. [Paper] [PDF]
How Does RL Post-training Induce Skill Composition? A Case Study on Countdown. Simon Park, Simran Kaur, Sanjeev Arora. [Paper] [PDF]
RL Fine-Tuning Heals OOD Forgetting in SFT. Hangzhan Jin, Sitao Luan, Tianwei Ni, Sicheng Lyu, Guillaume Rabusseau, Reihaneh Rabbany, Doina Precup, Mohammad Hamdaqa. [Paper] [PDF]
Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoning. Maggie Huan, Yuetai Li, Tuney Zheng, Xiaoyu Xu, Seungone Kim, Minxin Du, Radha Poovendran, Graham Neubig, Xiang Yue. [Paper] [PDF]
Decomposing Elements of Problem Solving: What “Math” Does RL Teach?. Tian Qin, Core Francisco Park, Mujin Kwun, Aaron Walsman, Eran Malach, Nikhil Anand, Hidenori Tanaka, David Alvarez-Melis. [Paper] [PDF]
Characterizing Pattern Matching and Its Limits on Compositional Task Structures. Hoyeon Chang, Jinho Park, Hanseul Cho, Sohee Yang, Miyoung Ko, Hyeonbin Hwang, Seungpil Won, Dohaeng Lee, Youbin Ahn, Minjoon Seo. [Paper] [PDF]
Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Tracking. Yifan Zhang, Wenyu Du, Dongming Jin, Jie Fu, Zhi Jin. [Paper] [PDF]
Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoning. Xinhao Yao, Ruifeng Ren, Yun Liao, Lizhong Ding, Yong Liu. [Paper] [PDF]
Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization. Boshi Wang, Xiang Yue, Yu Su, Huan Sun. [Paper] [PDF]

Theoretical analysis

Algebraic Decomposition Theory for Transformer Length Generalization. Andy Yang, Blerta Veseli, Corentin Barloy, Michaël Cadilhac, Andreas Krebs, Charles Paperman, Howard Straubing, Michael Hahn. [Paper] [PDF]
Relative Positions Generalize, Absolute Positions Memorize: An Implicit-Bias Account of Length Generalization in Attention. Subham Singh, Ashutosh Mishra, Subha Raut. [Paper] [PDF]
Learning to Reason with Curriculum II: Compositional Generalization. Nived Rajaraman, Audrey Huang, Miroslav Dudik, Robert Schapire, Dylan Foster, Akshay Krishnamurthy. [Paper] [PDF]
A Measure-Theoretic Analysis of Reasoning: Structural Generalization and Approximation Limits. Yuyang Zhang, Yifu Zhang, Xuehai Zhou, Xiaoyin Chen. [Paper] [PDF]
When Symbol Names Should Not Matter: A Logistic Theory of Fresh-Symbol Classification. Wenjie Guan, Jelena Bradic. [Paper] [PDF]
On the Ability of Transformers to Verify Plans. Yash Sarrof, Yupei Du, Katharina Stein, Alexander Koller, Sylvie Thiébaux, Michael Hahn. [Paper] [PDF]
Length Generalization Bounds for Transformers. Andy Yang, Pascal Bergsträßer, Georg Zetzsche, David Chiang, Anthony W. Lin. [Paper] [PDF]
Transformers Provably Learn Chain-of-Thought Reasoning with Length Generalization. Yu Huang, Zixin Wen, Aarti Singh, Yuejie Chi, Yuxin Chen. [Paper] [PDF]
How Far Can Transformers Reason? The Globality Barrier and Inductive Scratchpad. Emmanuel Abbe, Samy Bengio, Aryo Lotfi, Colin Sandon, Omid Saremi. [Paper] [PDF]
On Provable Length and Compositional Generalization. Kartik Ahuja, Amin Mansouri. [Paper] [PDF]
What Algorithms can Transformers Learn? A Study in Length Generalization. Hattie Zhou, Arwen Bradley, Etai Littwin, Noam Razin, Omid Saremi, Josh Susskind, Samy Bengio, Preetum Nakkiran. [Paper] [PDF]

Citation

The paper citation will be added when the survey is released. To cite the living repository in the meantime:

@misc{awesome_reasoning_generalization_2026,
  title        = {Awesome Reasoning Generalization},
  author       = {{Awesome Reasoning Generalization Contributors}},
  year         = {2026},
  howpublished = {\url{https://github.com/tue09/awesome-reasoning-generalization}},
  note         = {Accessed: YYYY-MM-DD}
}

awesome-reasoning-generalization

README 快照