Yuekun Yao

I am Yuekun Yao, a postdoctoral researcher at The Ohio State University, where I work with Prof. Huan Sun. I completed my Ph.D. in the Department of Language Science and Technology at Saarland University, advised by Prof. Alexander Koller.

My research asks how neural networks learn and generalize tasks. Lately, I study the limitations of transformers, and how data, training recipes, and model architectures shape these behaviors. Areas I am interested in include interpretability, LLM knowledge, and reasoning. Before that, I worked on semantic parsing, with a focus on out-of-distribution and compositional generalization.

I am happy to collaborate on research projects. Please don’t hesitate to reach out if you are interested!

news

Nov 2026	Our new paper, ARLoop: Fully-Open Pre-trained Looped Transformers with Attention Residuals, is now on arXiv. Code, models, and training recipes are available.
Sep 2026	One paper accepted to NeurIPS 2026.
Aug 2026	One paper accepted to EMNLP 2026.
Jul 2026	Two papers accepted to COLM 2026.

selected publications

Preprint

ARLoop: Fully-Open Pre-trained Looped Transformers with Attention Residuals

Yuekun Yao, Yupei Du, Dawei Zhu, and Huan Sun

arXiv preprint, 2026

arXiv PDF Bib

@misc{yao2026arloop,
  title = {ARLoop: Fully-Open Pre-trained Looped Transformers with Attention Residuals},
  author = {Yao, Yuekun and Du, Yupei and Zhu, Dawei and Sun, Huan},
  year = {2026},
  eprint = {2611.05234},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.05234}
}

COLM

Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers

Harsh Kohli, Srinivasan Parthasarathy, Huan Sun, and 1 more author

Third Conference on Language Modeling, 2026

arXiv Bib PDF

@inproceedings{kohli2026loopthinkgeneralize,
  title = {Loop, Think, \& Generalize: Implicit Reasoning in Recurrent-Depth Transformers},
  author = {Kohli, Harsh and Parthasarathy, Srinivasan and Sun, Huan and Yao, Yuekun},
  year = {2026},
  booktitle = {Third Conference on Language Modeling},
  url = {https://arxiv.org/abs/2604.07822},
}

EMNLP

Language models can learn implicit multi-hop reasoning, but only if they have lots of training data

Yuekun Yao, Yupei Du, Dawei Zhu, and 2 more authors

In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025

arXiv Bib PDF

@inproceedings{yao2025implicit,
  title = {Language models can learn implicit multi-hop reasoning, but only if they have lots of training data},
  author = {Yao, Yuekun and Du, Yupei and Zhu, Dawei and Hahn, Michael and Koller, Alexander},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}

EMNLP

Structural generalization is hard for sequence-to-sequence models

Yuekun Yao and Alexander Koller

In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022

Bib PDF Code Website

@inproceedings{yao2022structural,
  title = {Structural generalization is hard for sequence-to-sequence models},
  author = {Yao, Yuekun and Koller, Alexander},
  booktitle = {Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
}