Hello! This site contains materials for CPSC 340 (Machine Learning and Data Mining) taught at the University of British Columbia in January-April 2018 by Mike Gelbart.

The lecture videos are available here.

# Date Topic, Slides Related readings and links
1 Jan 3 Syllabus Machine Learning, Rise of the Machines, Talking Machine Episode 1
2 Jan 5 Exploratory data analysis PDF version of lecture, Bonus slides, Gotta Catch’em all, Why Not to Trust Statistics, Visualization Types, Google Chart Gallery, Other tools
3 Jan 8 Decision trees Notes on big-O, A Visual Introduction to Machine Learning, Decision Trees, Entropy, What makes Dr. Seuss so silly?, AI:AMA 18.2-3, ESL: 9.2, ML:APP 16.2
4 Jan 10 Fundamentals of learning in-class demo, IID, Cross-validation, Bias-variance, No Free Lunch, AI: AMA 18.4-5, ESL 7.1-7.4, 7.10, ML:APP 1.4, 6.5
5 Jan 12 Non-parametric models: KNN in-class demo, K-nearest neighbours, Decision Theory for Darts, AI: AMA 18.8, ESL 13.3, ML:APP 1.4
6 Jan 15 Naive Bayes Notes on probability, Extra slides on probability, Conditional probability (demo), Naive Bayes, Notes on Naive Bayes, Probabilities and Battleship, ESL 4.3, ML: APP 2.2, 3.5, 4.1-4.2
7 Jan 17 Ensemble methods in-class demo, Ensemble Methods, Random Forests, Empirical Study, Kinect, AI: AMA 18.10, ESL: 7.11, 8.2, 15, 16.3, ML: APP 6.2.1, 16.2.5, 16.6
8 Jan 19 Clustering in-class demo, Clustering, K-means clustering (demo), K-Means++ (demo), DBSCAN (video, demo), IDM 8.1-8.2, ESL: 14.3
9 Jan 22 More clustering, outlier detection Hierarchical Clustering, Phylogenetic Trees, A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data, IDM 8.3-8.4, ESL 14.3.12, ML:APP 25.5, IDM 10.1-10.5
10 Jan 24 What is optimization? PDF version of lecture, Notes on convexity
11 Jan 26 Linear regression: predict Linear Regression (demo, 2D data, 2D video), Least Squares, Partial Derivatives, Gradient, ESL 3.1-2, ML:APP 7.1-3, AI:AMA 18.6
12 Jan 29 Linear regression: fit Norms, Notes on Linear Algebra, Linear/Quadratic Gradients, Matrix Differentiation, The Matrix Cookbook (probably overkill)
13 Jan 31 Gradient descent PDF version of lecture, bonus slides, Gradient Descent, Dimensional analysis of gradient descent, ML:APP 7.4
14 Feb 2 Nonlinear regression in-class demo, Fluid simulation paper, Fluid simulation video, ESL 5.1, 6.3, and 6.7
15 Feb 5 Feature selection and L0-regularization ESL 3.3
16 Feb 7 L2-Regularization in-class demo, Stein’s Paradox visualization, ESL 3.4, ML:APP 7.5, AI:AMA 18.4
17 Feb 9 L1-Regularization  
17.75 Feb 12 Bonus lecture Family day (no class)
       
18 Feb 26 Linear classifiers: predict in-class demo, Support Vector Machines, ESL 4.4, ML:APP 8.1-3, AI:AMA 18.9
19 Feb 28 Linear classifiers: fit in-class demo, ESL 4.5 and 12.1-2, ML:APP 14.5
20 Mar 2 Linear classifiers: multi-class Gmail Priority Inbox, ML:APP 8.3.7 and 9.3-5, ESL 4.4
21 Mar 5 Kernel methods in-class demo, ESL 12.3, ML:APP 14.1-4
22 Mar 7 Stochastic Gradient in-class demo, Stochastic Gradient, ML:APP 8.5
23 Mar 9 Maximum likelihood Maximum Likelihood Estimation, max and argmax notes, ESL 3.4, ML:APP 13.3-4 (TODO: verify these book chapters)
24 Mar 12 PCA: predict in-class demo, Principal Component Analysis, ESL 14.5, IDM B.1, ML:APP 12.2
25 Mar 14 PCA: fit PCA Explained Visually, SVD, Eigenfaces
26 Mar 16 Sparse Matrix Factorization in-class demo, sklearn topic modeling demo with NMF, Non-Negative Matrix Factorization, original NMF paper (you should have access to the PDF when on the UBC network), ESL 14.6, ML: APP 13.8
27 Mar 19 Nonlinear dimensionality reduction Nonlinear Dimensionality Reduction, t-SNE video, t-SNE caveats, ESL 14.8-9, IDM B.2
28 Mar 21 Recommender systems Recommender Systems, Netflix Prize, fast.ai video segment on collaborative filtering, Association Rule Learning, Apriori, Amazon Product Recommendation, IDM 6.1-6.3, ESL 14.2
29 Mar 23 Neural Networks: predict in-class demo, But what is a Neural Network? (video, 19min at 1x speed, highly recommended), Google Video, Fortune Article, great list of resources, ML:APP 16.5, ESL 11.1-4, AI: AMA 18.7
30 Mar 26 Neural Networks: fit & Convolutions What is backpropagation really doing? (video, 14min at 1x speed), Ali Rahimi @ NIPS 2017 (video, 18min at 1x speed), ML:APP 28.3, ESL 11.5
31 Mar 28 Convolutional Neural Networks in-class demo, Convolutional Neural Networks, AlexNet, ML:APP 28.4, ESL 11.7
32 Apr 4 More CNNs, deep learning software in-class demo, Artistic Style Transfer video, Deep Photo Style Transfer, The Building Blocks of Interpretability (for CNNs)
33 Apr 6 Conclusion  

The acroynms in the table above refer to the following textbooks:

  • AI:AMA: Artificial Intelligence: A Modern Approach by Russell and Norvig
  • ESL: The Elements of Statistical Learning by Hastie et al.
  • ML:APP: Machine Learning: A Probabilistic Perspective by Kevin Murphy
  • PRML: Pattern Recognition and Machine Learning by Christopher Bishop
  • IDM: Introduction to Data Mining by Steinbach et al.