Over the last few years, deep neural networks (DNN) have fundamentally transformed the way people think of machine learning and approach practical problems. Successes around DNN have ranged from traditional AI fields such as computer vision, natural language processing, interactive games, to health care, physical sciences—touching each and every corner of theoretical and applied domains. On the other hand, DNN still largely operate as black-boxes and we only have very limited understanding as for when and why they work. This course introduces basic ingredients of DNN, samples important applications, and throws around open problems. Emphasis is put on thinking from first principles, as the field is still evolving rapidly and there is nothing there that cannot be changed.

Target: Graduate and advanced undergrad students. Registration is based on permission from the instructor. If you’re interested, please email Prof. Sun (jusun AT umn.edu) and describe your academic standing, relevant course experience, and research experience if any.

No Panic: The enrollment has reached the cap. While we’re maintaining a waiting list, and may decide to increase the cap later, there’s no guarantee. We’re likely to re-run the course in fall 2020 and to make the course regular in the near future, and so please consider next iterations if you’re not in.

Prerequisite: Introduction to machine learning or equivalent. Maturity in linear algebra, calculus, and basic probability is assumed. Familiarity with Python (esp. numpy, scipy) is necessary to complete the homework assignments and final projects.

When/Where: T/Th 2:30PM–3:45PM, Akerman Hall 225

TA’s: Yuan Yao Email: yaoxx340 AT umn.edu   Taihui Li Email: lixx5027 AT umn.edu

Tentative topics to cover:

  • Course overview
  • Neural networks: old and new
  • Fundamental belief: universal approximation theorem
  • Numerical optimization with math: optimization with gradient descent and beyond
  • Numerical optimization without math: auto-differentiation and differential programming
  • Working with images: convolutional neural networks
  • Working with images: recognition, detection, segmentation
  • To train or not? scattering transforms
  • Working with sequences: recurrent neural networks
  • Learning probability distributions: generative adversarial networks
  • Learning representation without labels: dictionary learning and autoencoders
  • Gaming time: deep reinforcement learning

Tentative discussion sessions:

  • Python, Numpy, and Google Cloud/Colab
  • Project ideas
  • Tensorflow 2.0 and Pytorch
  • Backpropagation and computational tricks
  • Research ideas