What are state-of-the-art machine learning methods, and why they work? This graduate-level introductory course to machine learning focuses on the foundations of modern machine learning. We will cover selected topics from supervised learning, unsupervised learning, and interactive learning. For each topic, key algorithmic ideas/intuitions and basic theoretical insights will be highlighted. The end goal is that the students will be able to develop and deploy novel learning methods for their applications, and potentially derive basic theoretical understanding

Full syllabus: AML.pdf

Instructor: Professor Ju Sun Email: jusun AT umn.edu (Office Hours: 5–6pm Tue/Thur)

When/Where: Mon/Wed 1:00 – 2:15pm/Keller 3-210

TA’s: Le Peng Email: peng0347 AT umn.edu (Office Hours: 3–4pm Mon/Wed)

Tiancong Chen Email: chen6271 AT umn.edu (Office Hours: 3–4pm Fri)

Lecture Schedule

Date Topics Notes
Jan 20 Overview [Slides]
Calculus Review
Jan 25 Review of high-dimensional calculus - I [Notes]
Jan 27 Review of high-dimensional calculus - II
Linear Predictions
Feb 01 Linear regression and least-squares problem [Notes]
Feb 03 Gradient descent for unconstrained optimization
Feb 15 Linear classification: Perceptron, SVM, and logistic regression
Support Vector Machines and Kernel Methods
Feb 17 Subspaces, hyperplanes, and margins [Notes]
Feb 22 Hard-margin SVMs and properties
Feb 24 Review of convex analysis & optimization; analysis of hard-margin SVMs
Mar 01 Soft-margin SVMs; optimizing SVMs
Mar 03 Kernel methods
Elements of Statistical Learning Theory
Mar 08 PAC learning with finite hypothesis classes [Notes]
Mar 10 Agnostic PAC learning with finite hypothesis classes; Uniform convergence
Mar 15 Agnostic PCA learning with infinite hypothesis classes; Rademacher complexity
Mar 17 VC dimensions; Bias-complexity tradeoff and model selection
Ensemble Methods: from Simple to Powerful
Mar 22 Decision stumps and their linear combinations [Notes]
Mar 24 Adaboost and its training error
Mar 29 Generalizations of Adaboost: greedy methods and gradient methods
Mar 31 Computing with decision trees
Apr 12 CART; Generalization gap of Adaboost
Apr 14 Bagging and random forests
Linear and Nonlinear Dimension Reduction
Apr 19 PCA as subspace fitting/autoencoder; random projection [Notes]
Apr 21 Compressive sensing; nonlinear dimension reduction
Clustering
Apr 26 K-means, hierarchical clustering, spectral clustering [Notes]
Apr 28 More on spectral clustering; mode seeking methods
Generative Models
May 03 Mixture modeling, MLE, and EM principle [Notes]
May 05 MAP, normalization flows, GANs
Neural Networks: Taking the Universal Power
Apr 05 [Notes]
Apr 07

Homework Assignments
HW1 (Due: Feb 10 2021)
HW2 (Due: Mar 19 2021)
HW3 (Due: Mar 31 2021)
HW4 (Due: Apr 28 2021)
Mid Term HW5 (Due: May 14 2021)
[HW6] (optional)