Welcome to the Fall 2024 edition of the Machine Learning course!

This course provides a rigorous introduction to the field of machine learning (ML). The aim of the course is not just to teach how to use ML algorithms but also to explain why, how, and when these algorithms work. The course introduces fundamental algorithms in supervised learning from the first principles. The course, while covering several problems in machine learning like regression, classification, representation learning, and dimensionality reduction, will introduce the core theory, which unifies all the algorithms.

This course will be offered in English. However, the students in this course can submit in English or French any written work that is to be graded.

Quebec university students from outside Polytechnique Montreal can register for the course via Inter-University Transfer Authorization.

Please note that I will also be teaching Reinforcement Learning (RL) (INF8250AE) this Fall. You can take both courses (ML and RL) in parallel.

If you are a student at Poly, UdeM, HEC, McGill, or Mila, then you can request to audit this course by filling out this Google Form.

General Information

When?
Wednesdays 9:30 am to 12:30 pm (starting from 28 Aug 2024)

Where?
C-631
C-631 is in the main building of Poly. You can find the main building on the map here.

About Labs
The lab slot for this course is on Thursdays from 4:45 pm to 7:45 pm. The lab slot will be mainly used for online tutorials, in-person/online recitations, and TA office hours. You can use the rest of the lab time to work on the practical assignments by yourself. Any in-person lab activities will happen in A-416.

People

Instructor

  • Sarath Chandar

TAs

  • Albert Orozco Camacho
  • Hadi Hojjati
  • Jeremy Qin
  • Maryam Hashemzadeh
  • Megh Thakkar

Office Hours

Name Day Time Location
Sarath Wednesday 12:30 PM to 1:30 PM M-3406

Please note that TAs will hold their office hours during the lab slot.

Logistics

Prerequisites

Basic knowledge of Probability Theory/statistics (MTH2302 or equivalent), calculus, and linear algebra (MTH1007 or equivalent) is required.

You should be already familiar with the following sections in this book: Mathematics for Machine Learning.

  • Section 2: Subsections 2.1, to 2.6 (inclusive)
  • Section 3: All subsections
  • Section 4: Subsections 4.1 to 4.5.1 (inclusive)
  • Section 5: Subsections 5.1, 5.2, 5.3, 5.4, 5.5, 5.7
  • Section 6: Subsections 6.1 to 6.5 (inclusive)

The course is intended for hard-working, technically skilled, highly motivated students. Participants will be expected to display initiative, creativity, scientific rigour, critical thinking, and good communication skills.

If you do not have the necessary prerequisites, then you have to spend a lot of time in this course (more than what is required for a 4-credit course).

Useful Online Courses covering the Prerequisites

While I do not expect you to know everything from the following courses, I recommend you to do these video courses at some point in the future if you are serious about doing Machine Learning.

Video Recordings

The lectures and tutorials might be recorded and released to the public. By registering for the course, you agree to record and release videos.

Programming Language

We will use Python 3 in all the assignments.

Evaluation Criteria

The class grade will be based on the following components:

  • 4 Theory/Programming assignments (individual) - 30%
  • Mid-term examination - 20%
  • Kaggle competition (team of 3) - 20%
  • End-term examination - 30%

To obtain a passing grade on the course (D or better), a necessary but not sufficient condition is to obtain at least 50% in both the mid-term and end-term exams combined.

We will use Gradescope for all assignments and projects. At the beginning of the course, more detailed instructions on how to use Gradescope will be released.

Late Submissions

If you submit your assignments and competition reports after the deadline, we will follow the following penalty scheme:

  • You will be penalized 5% if your submission is within 24 hours (1 day) from the deadline.
  • You will be penalized 10% if your submission is after 24 hours from the deadline and within 48 hours (2 days) from the deadline.
  • You will be penalized 20% if your submission is after 48 hours from the deadline and within 72 hours (3 days) from the deadline.
  • You cannot submit your assignments/reports after 72 hours from the deadline.

Syllabus

Tentative Course Content

Introduction - Prediction - Statistical Decision Theory - Linear Regression - Non-linear Regression - Bias-variance tradeoff - Linear Classification - Indicator Regression - PCA - LDA - QDA - GDA - Naive Bayes - Logistic Regression - Perceptron - Separating Hyperplanes - SVM - Decision Trees - ensemble learning - bagging - boosting - stacking - Neural Networks - Backpropagation - Training Deep Neural Nets - Optimization Methods - Convnets - RNNs - Estimation Theory - Maximum Likelihood Estimation - Maximum A Posteriori Estimation - Bayesian Learning - Bayesian Linear Regression - Kernel Methods - Gaussian Process - Clustering - K-means - GMM - EM Algorithm - Computational Learning Theory - Frontiers in ML.

Reference Materials

The course is based on the following references.

  1. [LN] INF8245E Course Lecture Notes.
  2. [HTF] Trevor Hastie, Robert Tibshirani and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. free online.
  3. [Bishop] Christopher Bishop. Pattern Recognition and Machine Learning. free online
  4. [Mitchell] Tom Mitchell. Machine Learning.
  5. [TSKK] Pang-Ning Tan, Michael Steinbach, Anuj Karpatne, Vipin Kumar. Introduction to Data Mining.
  6. [Rojas] Raul Rojas. Neural Networks.
  7. [GBC] Ian Goodfellow, Yoshua Bengio and Aaron Courville. Deep Learning. Available free online
  8. [Murphy] Kevin P. Murphy. Probabilistic Machine Learning: An Introduction. free online
  9. [Prince] Simon J.D. Prince. Understanding Deep Learning. free online