Statistics 154/254: Modern Statistical Prediction and Machine Learning

UC Berkeley, Fall 2024

No matching items

Description

  • Instructor: Song Mei (songmei [at] berkeley.edu)
  • Lectures: Tuesday/Thursday 9:30 am - 11:00 am. Etcheverry 3108.
  • Office Hours: Tuesday 1 pm - 3 pm. Evans 387.
  • GSI: Ruiqi Zhang (rqzhang [at] berkeley.edu)
  • Lab sessions Friday 11:00 am - 12:59 pm, Evans 334; 3:00 pm - 4:59 pm, Evans 342.
  • Office Hours: Thursday 4 pm - 6 pm; Friday 1 pm - 3 pm. Evans 428.

This course will focus on statistical/machine learning methods, data analysis/programming skills. Upon completing this course, the students are expected to be able to 1) build baseline models for real world data analysis problems; 2) implement models using programming languages; 3) draw insights/conclusions from models.

Announcement

  1. First class: Aug 29, 2024 (Thursday).
  2. I will process the CE students during the week of Sep 9. I will prioritize CE students performing well in HW1.
  3. We will use Ed (https://edstem.org/us/courses/61329) for discussions and questions.
  4. Please submit HW using Gradescope (https://www.gradescope.com/courses/811006, code: YRD5EG).
  5. Please find homework and lecture notes on bCourse under “Files”.
  6. HW policy: There are in total 3 late days that you can use without penalty towards grade throughout the semester. After that, there will be a 10% deduction on grades of a HW for each late day. The least grade can be dropped counting towards total grades.
  7. The lectures will be recorded through Course Capture. The recordings can be found on bCourse under “Media Gallery”.

Grading

  • Class attendance is required.
  • Homework per two weeks. There will be 6-7 HWs.
  • In class mid-term: Oct 24. Logistics: https://docs.google.com/document/d/1VfW1isSq68IwW8TQnZF_3o_SxJ0Yy0sDzI8QVeaSSSw/edit?usp=sharing
  • Final exam date: Dec 17. Logistics: https://docs.google.com/document/d/1VfW1isSq68IwW8TQnZF_3o_SxJ0Yy0sDzI8QVeaSSSw/edit?usp=sharing
  • Final grade will be Homework × 40 % + mid-term × 25 % + final × 35 %.
  • HW policy: There are in total three late days that you can use without penalty towards grade throughout the semester. After that, there will be a 10% deduction on grades of a HW for each late day. The least grade can be dropped counting towards total grades.

Topics

Basic topics:
  • Tasks: Regression. Classification. Dimension reduction. Clustering.
  • Algorithms: Solving linear systems. Gradient descent. Newton’s method. Power iteration for eigenvalue problems. EM algorithms.
  • Others: Kernel methods. Regularization. Sample splitting. Resampling methods. Cross validation.
Advanced topics:
  • Statistical learning theory and optimization theory.
  • Bagging and Boosting. Tree based models. Neural networks. Bayesian models.
  • Online learning. Bandit problems.

Schedule

Week 1

Aug 29: Lecture 1 Introduction

Week 2

Sep 3: Lecture 2 Linear model and linear regression
Sep 5: Lecture 3 Hypothesis testing in linear models (I)

Week 3

Sep 10: Lecture 4 Hypothesis testing in linear models (II)
Sep 12: Lecture 5 Computational aspects of linear regression; Logistic regression

Week 4

Sep 17: Lecture 6 Logistic regression; Generative modeling approach for classification
Sep 19: Lecture 7 Generative modeling approach for classification; Support vector machine

Week 5

Sep 24: Lecture 8 Estimation of test error and model selection
Sep 26: Lecture 9 Zoom Lecture; Regularization

Week 5

Oct 1: No Lecture
Sep 26: Lecture 10 Zoom lecture; Bayes expansion and feature map

Week 6

Oct 8: Lecture 11 Zoom lecture; Kernel ridge regression
Oct 10: Lecture 12 Kernel machine

Week 7

Oct 15: Lecture 13 Bootstrap method
Oct 17: Lecture 14 PCA I

Week 8

Oct 22: Lecture 15 PCA II
Oct 24: In class midterm

Week 9

Oct 29: Lecture 16 Clustering I
Oct 31: Lecture 17 Clustering II

Week 10

Nov 5: No lecture
Nov 7: Lecture 18 Decision tree

Week 11

Nov 12: Lecture 19 Emsembling method
Nov 14: Lecture 20 Basics of statistical learning theory

Week 12

Nov 19: Lecture 21 Neural networks, deep learning
Nov 21: Lecture 22 Backpropagation

Week 13

Nov 26: Lecture 23 Deep learning theory I
Nov 28: no lecture

Week 14

Dec 3: Lecture 24 Deep learning theory II
Dec 5: Lecture 25 Deep learning theory III
No matching items