Chulhee (Charlie) Yun

Hi!

My name is Charlie, and I am a fourth-year Ph.D. student in the Laboratory for Information and Decision Systems, Department of Electrical Engineering and Computer Science, at Massachusetts Institute of Technology. I am also a member of the Machine Learning Group at MIT. I study optimization and machine learning, under joint supervision of Prof. Suvrit Sra and Prof. Ali Jadbabaie. Specifically, I am focusing on the theoretical aspects of deep learning from optimization and expressive power points of view. Before joining MIT, I was a master’s student in Electrical Engineering at Stanford University, where I had a great fortune to work with Prof. John Duchi. I finished my undergraduate program in Electrical Engineering at KAIST, South Korea.

[Google Scholar]

Contact

News

[06/2020] Started my second summer internship at Google Research, hosted by Hossein Mobahi and Shankar Krishnan.

[05/2020] Our paper on the expressive power of multi-head attention models was accepted for presentation at ICML 2020!

[12/2019] Our paper on universal approximation power of Transformers was accepted to ICLR 2020!

[10/2019] I will be presenting a paper titled “Are Transformers universal approximators of sequence-to-sequence functions?” at the NeurIPS 2019 Workshop on Machine Learning with Guarantees.

[09/2019] Two papers were accepted to NeurIPS 2019: “Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity” (Spotlight) and “Are deep ResNets provably better than linear predictors?” (Poster).

[07/2019] Spending a wonderful summer as a research intern at Google Research New York!

[02/2019] Two talks given in Feb: one at The 24th LIDS Student conference and the other at an invited seminar held by KAIST ISysE.

[Older News]

Publications

* indicates alphabetical order or equal contribution.

Preprints

Minimum Width for Universal Approximation [arXiv]
Sejun Park, Chulhee Yun, Jaeho Lee, Jinwoo Shin

SGD with shuffling: optimal rates without component convexity and large epoch requirements [arXiv]
Kwangjun Ahn*, Chulhee Yun*, Suvrit Sra

O(n) Connections are Expressive Enough: Universal Approximability of Sparse Transformers [arXiv]
Chulhee Yun, Yin-Wen Chang, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar

Conference/Workshop Papers

Low-Rank Bottleneck in Multi-head Attention Models [arXiv]
Srinadh Bhojanapalli, Chulhee Yun, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar
International Conference on Machine Learning (ICML) 2020

Are Transformers universal approximators of sequence-to-sequence functions? [arXiv] [paper] [slides]
Chulhee Yun, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar
International Conference on Learning Representations (ICLR) 2020
NeurIPS 2019 Workshop on Machine Learning with Guarantees [short paper] [poster]
NYAS Machine Learning Symposium 2020 Poster Awards – Honorable Mention

Are deep ResNets provably better than linear predictors? [arXiv] [paper] [slides] [poster]
Chulhee Yun, Suvrit Sra, Ali Jadbabaie
Neural Information Processing Systems (NeurIPS) 2019

Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity [arXiv] [paper] [slides] [poster]
Chulhee Yun, Suvrit Sra, Ali Jadbabaie
Neural Information Processing Systems (NeurIPS) 2019 (Spotlight)

Efficiently testing local optimality and escaping saddles for ReLU networks [arXiv] [paper]
Chulhee Yun, Suvrit Sra, Ali Jadbabaie
International Conference on Learning Representations (ICLR) 2019

Small nonlinearities in activation functions create bad local minima in neural networks [arXiv] [paper]
Chulhee Yun, Suvrit Sra, Ali Jadbabaie
International Conference on Learning Representations (ICLR) 2019

Minimax Bounds on Stochastic Batched Convex Optimization [paper]
John Duchi*, Feng Ruan*, Chulhee Yun*
Conference on Learning Theory (COLT) 2018

Global optimality conditions for deep neural networks [arXiv] [paper]
Chulhee Yun, Suvrit Sra, Ali Jadbabaie
International Conference on Learning Representations (ICLR) 2018
NIPS 2017 Workshop on Deep Learning: Bridging Theory and Practice [short paper]

Face detection using local hybrid patterns [paper]
Chulhee Yun, Donghoon Lee, Chang D. Yoo
International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2015

Talks

  • On the optimality and memorization in deep learning. Invited Talk at Harvard CRISP Group Meeting. Mar 2020.
  • Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity. The 25th LIDS Student Conference. Jan 2020.
  • Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity. NeurIPS 2019 Spotlight Talk. Dec 2019. [slides]
  • On the Global and Local Optimality of Deep Learning. KAIST ISysE Seminar. Feb 2019.
  • Small nonlinearities in activation functions create bad local minima in neural networks. The 24th LIDS Student Conference. Feb 2019.
  • Spurious Local Minima in Neural Networks: A Critical View. The 2018 INFORMS Annual Meeting. Nov 2018.
  • Global optimality conditions for deep linear neural networks. The 23rd LIDS Student Conference. Feb 2018. [slides]

Awards

Gratefully, my study has been supported by numerous scholarships and awards. To name a few:

Services

  • Co-Chair: The 24th Annual LIDS Student Conference 2019
  • Member: The LIDS & Stats Tea Talk Organizing Committee 2019–2020
  • Reviewer: NeurIPS 2018–20, COLT 2020, ICML 2019–20, ICLR 2019–20, AISTATS 2019, CDC 2018

Last update: 06/21/2020