Chulhee (Charlie) Yun

Hi!

My name is Charlie, and I am a fourth-year Ph.D. student in the Laboratory for Information and Decision Systems, Department of Electrical Engineering and Computer Science, at Massachusetts Institute of Technology. I am also a member of Machine Learning Group at MIT. I study optimization and machine learning, under joint supervision of Prof. Suvrit Sra and Prof. Ali Jadbabaie. Specifically, I am focusing on the theoretical aspects of deep learning from optimization and expressive power points of view. Before joining MIT, I was a master’s student in Electrical Engineering at Stanford University, where I had a great fortune to work with Prof. John Duchi. I finished my undergraduate program in Electrical Engineering at KAIST, South Korea.

[Google Scholar]

Contact

News

[12/2019] Our paper on universal approximation power of Transformers was accepted to ICLR 2020!

[10/2019] I will be presenting a paper titled “Are Transformers universal approximators of sequence-to-sequence functions?” at the NeurIPS 2019 Workshop on Machine Learning with Guarantees.

[09/2019] Two papers were accepted to NeurIPS 2019: “Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity” (Spotlight) and “Are deep ResNets provably better than linear predictors?” (Poster).

[07/2019] Spending a wonderful summer as a research intern at Google Research New York!

[02/2019] Two talks given in Feb: one at The 24th LIDS Student conference and the other at an invited seminar held by KAIST ISysE.

[12/2018] Two theoretical papers were accepted at The Seventh International Conference on Learning Representations (ICLR 2019)!

[09/2018] I’ll be giving a presentation based on “Small nonlinearities in activation functions create bad local minima in neural networks” at The 2018 INFORMS Annual Meeting.

[05/2018] A new paper “Minimax Bounds on Stochastic Batched Convex Optimization” was accepted for presentation at The 31st Annual Conference on Learning Theory (COLT 2018). The project during my master’s is finally to be published!

[01/2018] Our work “Global optimality conditions for deep neural networks” was accepted as a poster presentation at The Sixth International Conference on Learning Representations (ICLR 2018). Very excited!

[01/2018] I will be giving a presentation based on our work “Global optimality conditions for deep neural networks” at The 23rd Annual LIDS Student Conference on Feb 1, 2018.

[12/2017] I will be presenting a short paper “Global optimality conditions for deep neural networks” at Deep Learning: Bridging Theory and Practice Workshop at NIPS 2017.

Publications

Conference/Workshop Papers

Are Transformers universal approximators of sequence-to-sequence functions? [arXiv] [paper]
Chulhee Yun, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar
International Conference on Learning Representations (ICLR) 2020
NeurIPS 2019 Workshop on Machine Learning with Guarantees [short paper] [poster]

Are deep ResNets provably better than linear predictors? [arXiv] [paper] [slides] [poster]
Chulhee Yun, Suvrit Sra, Ali Jadbabaie
Neural Information Processing Systems (NeurIPS) 2019

Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity [arXiv] [paper] [slides] [poster]
Chulhee Yun, Suvrit Sra, Ali Jadbabaie
Neural Information Processing Systems (NeurIPS) 2019 (Spotlight)

Efficiently testing local optimality and escaping saddles for ReLU networks [arXiv] [paper]
Chulhee Yun, Suvrit Sra, Ali Jadbabaie
International Conference on Learning Representations (ICLR) 2019

Small nonlinearities in activation functions create bad local minima in neural networks [arXiv] [paper]
Chulhee Yun, Suvrit Sra, Ali Jadbabaie
International Conference on Learning Representations (ICLR) 2019

Minimax Bounds on Stochastic Batched Convex Optimization [paper]
John Duchi*, Feng Ruan*, Chulhee Yun*
Conference on Learning Theory (COLT) 2018

Global optimality conditions for deep neural networks [arXiv] [paper]
Chulhee Yun, Suvrit Sra, Ali Jadbabaie
International Conference on Learning Representations (ICLR) 2018
NIPS 2017 Workshop on Deep Learning: Bridging Theory and Practice [short paper]

Face detection using local hybrid patterns [paper]
Chulhee Yun, Donghoon Lee, Chang D. Yoo
International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2015

* indicates alphabetical order.

Talks

  • Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity. NeurIPS 2019 Spotlight Talk. Dec 2019. [slides]
  • On the Global and Local Optimality of Deep Learning. KAIST ISysE Seminar. Feb 2019.
  • Small nonlinearities in activation functions create bad local minima in neural networks. The 24th LIDS Student Conference. Feb 2019.
  • Spurious Local Minima in Neural Networks: A Critical View. The 2018 INFORMS Annual Meeting. Nov 2018.
  • Global optimality conditions for deep linear neural networks. The 23rd LIDS Student Conference. Feb 2018. [slides]

Awards

Gratefully, my study has been supported by numerous scholarships and awards. To name a few:

Services

  • Co-Chair: The 24th Annual LIDS Student Conference 2019
  • Member: The LIDS & Stats Tea Talk Organizing Committee 2019-2020
  • Reviewer: ICLR 2019–20, NIPS 2018–19, ICML 2019, AISTATS 2019, CDC 2018

Last update: 12/20/2019