Prerequisite: background in algorithms and statistics (CS/CNS/EE/NB 154 or CS/CNS/EE 156a or instructorâ€™s permission)

This course will cover popular methods in machine learning and data mining, with an emphasis on developing a working understanding of how to apply these methods in practice. This course will also cover core foundational concepts underpinning and motivating modern machine learning and data mining approaches. This course will be research-oriented, and will cover recent research developments.

- Lectures on Tu/Th at 2:30pm-4pm in Annenberg 105
- Recitations on Wed at 5pm-7pm in Annenberg 105 (usually lasting 1 hour)
- We will be using Moodle (enroll now!) [link]
- 4 Homeworks, worth 30-40% of final grade
- 2 Miniprojects, worth 20-30% of final grade
- Final Exam, worth 30-40% of final grade

Students are allowed 2 free late days for submitting homeworks and miniprojects. After using the two free late days, a 50% penalty will to submissions that are one day late, and submissions beyond one day late will not be accepted. You can use fractions of late days. For example, you can turn in the homework 12 hours late and use up half of a late day. Please specify how many late days you are using at the top when you submit your homework.

Yisong Yue yyue@caltech.edu

Bryan He | bryanhe@caltech.edu |

Masoud Farivar | mfarivar@caltech.edu |

Shenghan Yao | syao@caltech.edu |

Vighnesh Shiv | vshiv@caltech.edu |

Minfa Wang | mwang5@caltech.edu |

Vineet Augustine | vaaugust@caltech.edu |

Machine Learning: a Probabilistic Perspective,
by Kevin Murphy.

Since this is an advanced level course, all relevant course materials can be learned via research papers and supplementary lecture notes. However, this book is an excellent reference and I will refer to various chapters of it throughout the course.

- Homework 0 -- this homework will not be graded, and tests your knowledge of the prerequisite concepts.
- Homework 1 -- this homework is due at 5pm on January 20th, 2015 via Moodle. [Supplementary dataset]
- Homework 2 -- this homework is due at 2pm February 3rd, 2015 via Moodle. [Supplementary dataset]
- Miniproject 1 - This is a Kaggle Competition, closes at 2pm February 24th, 2015. Report is due at 2pm February 26th, 2015 via Moodle.
- Homework 3 -- this homework is due at 2pm February 17th, 2015 via Moodle.
- Homework 4 -- this homework is due at 2pm March 10th, 2015 via Moodle.
- Miniproject 2 -- This is a visualization project, due at 2pm on March 13th, 2015 via Moodle. [Supplementary dataset] [step-by-step instructions]
- Practice Questions -- this is a set of practice questions, and will be useful for the final exam.

Note: schedule is subject to change.

Further Reading: |
||||

1/06/2015 | Lecture: | Administrivia, Review | [slides][pdf] | |

1/07/2015 | Recitation: | Linear Algebra & Optimization | [pdf] | |

1/08/2015 | Lecture: | Review Part 2 | [slides][pdf] | |

1/13/2015 | Lecture: | Regularization, Sparsity & Lasso | [slides][pdf] | Murphy 13.3 |

1/14/2015 | Recitation: | Probability & Statistics | [pdf] | |

1/15/2015 | Lecture: | Recent Applications of Lasso | [slides][pdf] | |

1/20/2015 | Lecture: | Sequence Prediction & HMMs | [slides][pdf] | Murphy 17.3--17.5 |

1/21/2015 | Recitation: | Viterbi Algorithm | (no slides, sorry) | |

1/22/2015 | Lecture: | Conditional Random Fields | [slides][pdf][notes] | Wallach's intro to CRFs [pdf] |

1/27/2015 | Lecture: | Recap of CRFs & Structural SVMs | [slides][pdf][notes] | |

1/28/2015 | Recitation: | Gradient Descent for non-Differentiable Functions | [pdf] | |

1/29/2015 | Lecture: | Structural SVMs Part 2 & General Structured Prediction | [slides][pdf] | |

2/03/2015 | Lecture: | Decision Trees, Bagging & Random Forests | [slides][pdf] | Overview of Decision Trees [pdf] Overview of Bagging [pdf] Overview of Random Forests [pdf] |

2/04/2015 | Recitation: | Brief Tutorial on Kaggle & Decision Tree Packages | [pdf] | |

2/05/2015 | Lecture: | Boosting & Ensemble Selection | [slides][pdf] | Shapire's Overview of Boosting [pdf] |

2/10/2015 | Lecture: | Learning Reductions & Recent Applications of DTs | [slides][pdf] | |

2/12/2015 | NO LECTURE |
|||

2/17/2015 | Lecture: | Clustering & Dimensionality Reduction | [slides][pdf] | Murphy 12.2 |

2/19/2015 | Lecture: | Latent Factor Models & Non-Negative Matrix Factorization | [slides][pdf] | Original Netflix Paper [link] |

2/24/2015 | Lecture: | Embeddings | [slides][pdf] | Locally Linear Embedding [link] Playlist Embedding [link] |

2/26/2015 | Lecture: | Recent Applications of Latent Factor Models | [slides][pdf] | |

3/03/2015 | Lecture: | Deep Learning | [slides][pdf] | |

3/05/2015 | Lecture: | The Multi-Armed Bandit Problem | [slides][pdf] | |

3/10/2015 | Lecture: | Course Review |

- Papers on Ensemble Selection. [paper1][paper2][KDD Cup Report]
- Practical Bayesian Optimization for Efficient Grid Search of Tuning Parameters. [paper][software]
- Reasonably Accessible Paper on Regularized Multi-Task Learning. [paper]
- Overview of Topic Models. [paper]
- Overview of Structural SVMs. [paper]
- A Brief Overview of Deep Learning. [link]
- Tutorial on Learning Reductions. [pdf]
- The Matrix Cookbook (a lot of useful properties of matrices). [link]
- Learning Reductions Overview. [paper]