Lectures

Stat 154/254: Statistical Machine Learning

---

Lectures marked with \(\star\) are still in progress.

Introduction

Class introduction

Unit 1: Regression in a machine learning setting

• Linear regression in matrix notation (review)
• Linear regression in an ML setting
• Basis expansions and splines for linear regression
• Asymptotics and bias–variance tradeoff in linear regression
• L1 and L2 penalization

Unit 2: Risk and complexity

• Generalization error and uniform laws
• Uniform laws for smooth function classes
• VC dimension and zero–one loss
• Held-out data and cross-validation

Unit 3: Classification

• Empirical loss minimization for classification
• Trading off false positives and negatives
• Perceptron algorithm
• Support vector machines

Floating unit: Optimization

• Optimization notes

Unit 4: Trees and weak learners

• Classification and regression trees
• Bagging weak learners
• Boosting weak learners

Unit 5: Kernel methods

• The kernel trick
• Valid kernels and inner products
• Reproducing kernel Hilbert spaces

Extra content

• Linear algebra review materials