# Advanced Machine Learning

What are state-of-the-art machine learning methods, and why they work? This graduate-level introductory course to machine learning focuses on the foundations of modern machine learning. We will cover selected topics from supervised learning, unsupervised learning, and interactive learning. For each topic, key algorithmic ideas/intuitions and basic theoretical insights will be highlighted. The end goal is that the students will be able to develop and deploy novel learning methods for their applications, and potentially derive basic theoretical understanding

Full syllabus: AML.pdf

**Instructor**: Professor Ju Sun Email: jusun AT umn.edu (Office Hours: 5–6pm Tue/Thur)

**When/Where**: Mon/Wed 1:00 – 2:15pm/Keller 3-210

**TA’s**:
Le Peng Email: peng0347 AT umn.edu (Office Hours: 3–4pm Mon/Wed)

Tiancong Chen Email: chen6271 AT umn.edu (Office Hours: 3–4pm Fri)

**Lecture Schedule**

Date | Topics | Notes |
---|---|---|

Jan 20 | Overview | [Slides] |

Calculus Review |
||

Jan 25 | Review of high-dimensional calculus - I | [Notes] |

Jan 27 | Review of high-dimensional calculus - II | |

Linear Predictions |
||

Feb 01 | Linear regression and least-squares problem | [Notes] |

Feb 03 | Gradient descent for unconstrained optimization | |

Feb 15 | Linear classification: Perceptron, SVM, and logistic regression | |

Support Vector Machines and Kernel Methods |
||

Feb 17 | Subspaces, hyperplanes, and margins | [Notes] |

Feb 22 | Hard-margin SVMs and properties | |

Feb 24 | Review of convex analysis & optimization; analysis of hard-margin SVMs | |

Mar 01 | Soft-margin SVMs; optimizing SVMs | |

Mar 03 | Kernel methods | |

Elements of Statistical Learning Theory |
||

Mar 08 | PAC learning with finite hypothesis classes | [Notes] |

Mar 10 | Agnostic PAC learning with finite hypothesis classes; Uniform convergence | |

Mar 15 | Agnostic PCA learning with infinite hypothesis classes; Rademacher complexity | |

Mar 17 | VC dimensions; Bias-complexity tradeoff and model selection | |

Ensemble Methods: from Simple to Powerful |
||

Mar 22 | Decision stumps and their linear combinations | [Notes] |

Mar 24 | Adaboost and its training error | |

Mar 29 | Generalizations of Adaboost: greedy methods and gradient methods | |

Mar 31 | Computing with decision trees | |

Apr 12 | CART; Generalization gap of Adaboost | |

Apr 14 | Bagging and random forests | |

Linear and Nonlinear Dimension Reduction |
||

Apr 19 | PCA as subspace fitting/autoencoder; random projection | [Notes] |

Apr 21 | Compressive sensing; nonlinear dimension reduction | |

Clustering |
||

Apr 26 | K-means, hierarchical clustering, spectral clustering | [Notes] |

Apr 28 | More on spectral clustering; mode seeking methods | |

Generative Models |
||

May 03 | Mixture modeling, MLE, and EM principle | [Notes] |

May 05 | MAP, normalization flows, GANs | |

Neural Networks: Taking the Universal Power |
||

Apr 05 | [Notes] | |

Apr 07 |

**Homework Assignments**

HW1 (Due: Feb 10 2021)

HW2 (Due: Mar 19 2021)

HW3 (Due: Mar 31 2021)

HW4 (Due: Apr 28 2021)

Mid Term
HW5 (Due: May 14 2021)

[HW6] (optional)