Chenyu Zheng (郑晨宇)   Ph.D. Student Gaoling School of Artificial Intelligence Renmin University of China Email: chenyu.zheng666[at]gmail[dot]com Links: [Google Scholar] [Github] [Zhihu] [CV]

About me

I am a second-year Ph.D. student in the GSAI-ML Group, Renmin University of China, advised by Prof. Chongxuan Li fortunately. Before that, I received my B.E. degree from the School of Computer Science, Wuhan University in 2023.

Research interests

My long-term research interests lie in deep learning/foundation models:

• leveraging mathematical tools from optimization and statistics to
• understand the mystery behind deep learning/foundation models (e.g., training dynamics and generalization),
• design efficient and robust algorithms for pre-training, post-training and inference.

Currently, I am mainly working on the bold parts.

Selected Papers

[* indicates the equal contribution, # indicates the corresponding author]

Foundation Models Theory (2023~)

• On Mesa-Optimization in Autoregressively Trained Transformers: Emergence and Capability
  Chenyu Zheng, Wei Huang, Rongzhen Wang, Guoqiang Wu, Jun Zhu, Chongxuan Li
  Advances in Neural Information Processing Systems (NeurIPS), 2024
  [Paper] [Code]

Machine Learning Theory (2022~)

• Toward Understanding Generative Data Augmentation
  Chenyu Zheng, Guoqiang Wu, Chongxuan Li
  Advances in Neural Information Processing Systems (NeurIPS), 2023
  [Paper] [Code] [Slides] [Blog] [Poster]
• Revisiting Discriminative vs. Generative Classifiers: Theory and Implications
  Chenyu Zheng, Guoqiang Wu, Fan Bao, Yue Cao, Chongxuan Li, Jun Zhu
  International Conference on Machine Learning (ICML), 2023
  [Paper] [Code] [Slides] [Blog] [Poster]

Experience

• Research Intern, ByteDance Research, Beijing, China (2023.11 - Present)
  

• Research Intern, Ant Group, Beijing, China (2023.10 - 2024.10)
  

• Research Intern, RSIDEA Group, LIESMARS, Wuhan, China (2021.01 - 2022.08)
  Advisors: Ph.D. candidate Junjue Wang and Prof. Ailong Ma

Honors

• Outstanding Graduate & Bachelor Thesis Award, Wuhan University, 2023
• Finalist, International Mathematical Contest in Modeling (MCM), 2022
• China National Scholarship, Ministry of Education of China, 2020

Academic Services

Journal Reviewer

• IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)

Conference Reviewer

• Advances in Neural Information Processing Systems (NeurIPS): 2024
• International Conference on Learning Representations (ICLR): 2025
• International Conference on Artificial Intelligence and Statistics (AISTATS): 2025

Tutorials/Talks

• Tutorial: A Brief Review of In-Context Learning Theory, 2024.05 [Slides]
• Tutorial: Classical Machine Learning Theory, 2023.06 [Slides]

Miscellaneous

• Platform Sutra and meditation teach me a lot. [Platform Sutra viedo] [meditation viedo]