Hi!
My name is Charlie, and I am a fifth-year Ph.D. student in the Laboratory for Information and Decision Systems, Department of Electrical Engineering and Computer Science, at Massachusetts Institute of Technology. I am also a member of the Machine Learning Group at MIT. I study optimization and machine learning, under joint supervision of Prof. Suvrit Sra and Prof. Ali Jadbabaie. Specifically, I am focusing on the theoretical aspects of deep learning from optimization and expressive power points of view. Before joining MIT, I was a master’s student in Electrical Engineering at Stanford University, where I had a great fortune to work with Prof. John Duchi. I finished my undergraduate program in Electrical Engineering at KAIST, South Korea.
Contact
News
[09/2020] Two papers got accepted to NeurIPS 2020: “SGD with shuffling: optimal rates without component convexity and large epoch requirements” (Spotlight) and “O(n) Connections are Expressive Enough: Universal Approximability of Sparse Transformers” (Poster).
[09/2020] Our results on the expressive power of deep and narrow networks (partly based on a recent preprint) are to be presented as a contributed talk at the DeepMath 2020 conference!
[06/2020] Started my second summer internship at Google Research, hosted by Hossein Mobahi and Shankar Krishnan.
[05/2020] Our paper on the expressive power of multi-head attention models was accepted for presentation at ICML 2020!
[12/2019] Our paper on the universal approximation power of Transformers was accepted to ICLR 2020!
[10/2019] I will be presenting a paper titled “Are Transformers universal approximators of sequence-to-sequence functions?” at the NeurIPS 2019 Workshop on Machine Learning with Guarantees.
[09/2019] Two papers were accepted to NeurIPS 2019: “Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity” (Spotlight) and “Are deep ResNets provably better than linear predictors?” (Poster).
Publications
* indicates alphabetical order or equal contribution.
Preprints
Provable Memorization via Deep Neural Networks using Sub-linear Parameters [arXiv]
Sejun Park, Jaeho Lee, Chulhee Yun, Jinwoo Shin
To be presented as a contributed talk at DeepMath 2020
A Unifying View on Implicit Bias in Training Linear Neural Networks [arXiv]
Chulhee Yun, Shankar Krishnan, Hossein Mobahi
Minimum Width for Universal Approximation [arXiv]
Sejun Park, Chulhee Yun, Jaeho Lee, Jinwoo Shin
To be presented as a contributed talk at DeepMath 2020
Conference/Workshop Papers
SGD with shuffling: optimal rates without component convexity and large epoch requirements [arXiv]
Kwangjun Ahn*, Chulhee Yun*, Suvrit Sra
Neural Information Processing Systems (NeurIPS) 2020 (Spotlight)
O(n) Connections are Expressive Enough: Universal Approximability of Sparse Transformers [arXiv]
Chulhee Yun, Yin-Wen Chang, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar
Neural Information Processing Systems (NeurIPS) 2020
Low-Rank Bottleneck in Multi-head Attention Models [arXiv]
Srinadh Bhojanapalli, Chulhee Yun, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar
International Conference on Machine Learning (ICML) 2020
Are Transformers universal approximators of sequence-to-sequence functions? [arXiv] [paper] [slides]
Chulhee Yun, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar
International Conference on Learning Representations (ICLR) 2020
NeurIPS 2019 Workshop on Machine Learning with Guarantees [short paper] [poster]
NYAS Machine Learning Symposium 2020 Poster Awards – Honorable Mention
Are deep ResNets provably better than linear predictors? [arXiv] [paper] [slides] [poster]
Chulhee Yun, Suvrit Sra, Ali Jadbabaie
Neural Information Processing Systems (NeurIPS) 2019
Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity [arXiv] [paper] [slides] [poster]
Chulhee Yun, Suvrit Sra, Ali Jadbabaie
Neural Information Processing Systems (NeurIPS) 2019 (Spotlight)
Efficiently testing local optimality and escaping saddles for ReLU networks [arXiv] [paper]
Chulhee Yun, Suvrit Sra, Ali Jadbabaie
International Conference on Learning Representations (ICLR) 2019
Small nonlinearities in activation functions create bad local minima in neural networks [arXiv] [paper]
Chulhee Yun, Suvrit Sra, Ali Jadbabaie
International Conference on Learning Representations (ICLR) 2019
Minimax Bounds on Stochastic Batched Convex Optimization [paper]
John Duchi*, Feng Ruan*, Chulhee Yun*
Conference on Learning Theory (COLT) 2018
Global optimality conditions for deep neural networks [arXiv] [paper]
Chulhee Yun, Suvrit Sra, Ali Jadbabaie
International Conference on Learning Representations (ICLR) 2018
NIPS 2017 Workshop on Deep Learning: Bridging Theory and Practice [short paper]
Face detection using local hybrid patterns [paper]
Chulhee Yun, Donghoon Lee, Chang D. Yoo
International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2015
Talks
- Theory of optimization in deep learning. KAIST Graduate School of AI Fall 2020 Colloquium. Oct 2020.
- On the optimality and memorization in deep learning. Invited Talk at Harvard CRISP Group Meeting. Mar 2020.
- Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity. The 25th LIDS Student Conference. Jan 2020.
- Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity. NeurIPS 2019 Spotlight Talk. Dec 2019. [slides]
- On the Global and Local Optimality of Deep Learning. KAIST ISysE Seminar. Feb 2019.
- Small nonlinearities in activation functions create bad local minima in neural networks. The 24th LIDS Student Conference. Feb 2019.
- Spurious Local Minima in Neural Networks: A Critical View. The 2018 INFORMS Annual Meeting. Nov 2018.
- Global optimality conditions for deep linear neural networks. The 23rd LIDS Student Conference. Feb 2018. [slides]
Awards
Gratefully, my study has been supported by numerous scholarships and awards. To name a few:
- Conference Travel Awards: ICLR 2018–20, NeurIPS 2019, COLT 2018
- Doctoral Study Abroad Program by Korea Foundation for Advanced Studies. Sep 2016–Current.
- Samsung Scholarship. Sep 2014–June 2016.
Services
- Co-Chair: The 24th Annual LIDS Student Conference 2019
- Member: The LIDS & Stats Tea Talk Organizing Committee 2019–2020
- Reviewer: COLT 2020–21, ICLR 2019–21, NeurIPS 2018–20, ICML 2019–20, AISTATS 2019, CDC 2018, JMLR, Annals of Statistics, IEEE TNNLR
Last update: 10/27/2020