(CS/CNS/EE 155) Machine Learning & Data Mining


2018/2019 Winter Term (previous year)

Course Description

Prerequisite: background in algorithms, linear algebra, calculus, probability, and statistics (CS/CNS/EE/NB 154 or CS/CNS/EE 156a or instructor’s permission)

This course will cover popular methods in machine learning and data mining, with an emphasis on developing a working understanding of how to apply these methods in practice. This course will also cover core foundational concepts underpinning and motivating modern machine learning and data mining approaches. This course will also cover some recent research developments.

Course Details

Late Homework Policy

Assignments will be due at 9pm on Wednesday via Moodle. Students are allowed to use up to 48 late hours. Late hours must be used in units of hours. Specify the number of hours used when turning in the assignment. Late hours cannot be used on the final exam. There will be no TA support over the weekends.

Collaboration Policy

Detailed policy available here
TLDR;

Instructor

Yisong Yue               yyue@caltech.edu

Teaching Assistants

Natalie Bernat      nbernat@caltech.edu
Nishanth Bhaskara nbhaskar@caltech.edu
Julia Deacon jcdeacon@caltech.edu
Marcus Dominguez-Kuhne mddoming@caltech.edu
Rupesh Jeyaram rjeyaram@caltech.edu
Cherie Jia cjia@caltech.edu
Karthik Karnik kkarnik@caltech.edu
Meera Krishnamoorthy mkrishna@caltech.edu
Karthik Nair knair@caltech.edu
Kapil Sinha ksinha@caltech.edu
Vaishnavi Shrivastavavshrivas@caltech.edu
Albet Tseng atseng@caltech.edu
Michelle Zhao mzhao@caltech.edu

Office Hours

Additional Textbooks and Resources

  • Machine Learning: a Probabilistic Perspective, by Kevin Murphy
  • Convex Optimization: Algorithms and Complexity (Free Version), by Sebastien Bubeck
  • Deep Learning, by Ian Goodfellow, Yoshua Bengio, and Aaron Courville
  • A Course in Machine Learning, by Hal Daume III
  • Matrix Cookbook
  • Probability Review
  • Maximum Entropy and Logistic Regression
  • A Beginner's Guide to Recurrent Networks and LSTMs [link]
  • Stochastic Gradient Descent Tricks [link]
  • Practical Bayesian Optimization for Efficient Grid Search of Tuning Parameters. [paper][software]
  • Overview of Topic Models. [paper]
  • Tutorial on Learning Reductions. [pdf]
  • Learning Reductions Overview. [paper]

    • Assignments

    Lectures & Recitation Schedule

    Note: schedule is subject to change.

                                    Further Reading:                                                
    1/08/2019 Lecture: Administrivia, Basics, Bias/Variance, Overfitting [slides]
    1/10/2019 Lecture: Perceptron, Gradient Descent [slides] Daume Chapter 3
    Mistake Bounds for Perceptron [link]
    AdaGrad [link]
    Stochastic Gradient Descent Tricks [link]
    Bubeck Chaper 3
    1/10/2019 Recitation: Introduction to Python for Machine Learning
    1/15/2019 Lecture: SVMs, Logistic Regression, Neural Nets, Loss Functions, Evaluation Metrics [slides] Bounds on Error Expectation for SVMs [link]
    1/17/2019 Lecture: Regularization, Lasso [slides] Murphy 13.3
    1/17/2019 Recitation: Linear Algebra The Matrix Cookbook [link]
    1/22/2019 Lecture: Decision Trees, Bagging, Random Forests [slides] Overview of Decision Trees [pdf]
    Overview of Bagging [pdf]
    Overview of Random Forests [pdf]
    1/24/2019 Lecture: Boosting, Ensemble Selection [slides] Schapire's Overview of Boosting [pdf]
    Papers on Ensemble Selection. [paper1][paper2]
    1/29/2019 Lecture: Deep Learning [slides] Deep Learning Book [html]
    1/31/2019 Lecture: Deep Learning Part 2 [slides]
    1/31/2019 Recitation: Keras Tutorial [link]
    2/5/2019 Lecture: Unsupervised Learning, Clustering, Dimensionality Reduction [slides]
    2/7/2019 Lecture: Latent Factor Models, Non-Negative Matrix Factorization [slides] Original Netflix Paper [link]
    2/12/2019 Lecture: Embeddings [slides] Locally Linear Embedding [link]
    Playlist Embedding [link]
    word2vec [link]
    2/14/2019 NO LECTURE
    2/19/2019 Lecture: Recent Applications [slides]
    2/21/2019 Lecture: Probabilistic Models, Naive Bayes [slides] Murphy 3.5
    2/21/2019 Recitation: Probability & Sampling
    2/26/2019 Lecture: Hidden Markov Models [slides][notes] Murphy 17.3--17.5
    2/28/2019 Lecture: Hidden Markov Models Part 2 (same material as above)
    2/28/2019 Recitation: Dynamic Programming
    3/5/2019 Lecture: Recent Applications: Deep Generative Models [slides]
    3/7/2019 Lecture: Survey of Advanced Topics [slides]
    3/12/2019 Lecture: Review & Q/A