Deep Learning---Models, Computation, and Applications
Over the last few years, deep neural networks (DNNs) have fundamentally transformed the way people think of machine learning and approach practical problems. Successes around DNNs have ranged from traditional AI fields such as computer vision, natural language processing, interactive games, to healthcare, physical sciences—touching each and every corner of theoretical and applied domains. On the other hand, DNNs still largely operate as black-boxes and we only have very limited understanding as for when and why they work. This course introduces basic ingredients of DNNs, samples important applications, and throws around open problems. Emphasis is put on thinking from first principles and basic building blocks, as the field is still evolving rapidly and there is nothing there that cannot be changed.
Full syllabus: Syllabus.pdf
Instructor: Professor Ju Sun Email: jusun AT umn.edu (Office Hours: 4–6pm, Thur)
When/Where: Tue/Thur 2:30–3:45pm @ Bruininks Hall 220
TA’s:
Hengkang Wang (email: wang9881 AT umn.edu, office Hours: 10am–12pm Fri @ Keller 2-209)
Wenjie Zhang (email: zhan7867 AT umn.edu, office Hours: 4–6pm Tue @ Keller 1-213)
Lecture Schedule
Disclaimer: The schedule is tentative and subject to change
| Date | Topics |
|---|---|
| Jan 21/23 | Deep learning: overview [slides] Neural networks: old and new [slides] Supplementary notes on high-dimensional calculus |
| Jan 28/30 | Fundamental belief: universal approximation theorems [slides] Basics of numerical optimization: optimality conditions [slides] |
| Feb 04/06 | Basics of numerical optimization: iterative methods [slides] |
| Feb 11/13 | Basics of numerical optimization: computing derivatives [slides] Introduction to Google Colab and PyTorch [slides] |
| Feb 18/20 | |
| Feb 25/27 | |
| Mar 04/06 | |
| Mar 11/13 | SPRING BREAK; NO LECTURES |
| Mar 18/20 | |
| Mar 25/27 | |
| Apr 01/03 | |
| Apr 08/10 | |
| Apr 15/17 | |
| Apr 22/24 | |
| Apr 29 May 01 |
|
Homework