Logistics

Prerequisites

Basic knowledge of Probability Theory/statistics (MTH2302 or equivalent), calculus, and linear algebra (MTH1007 or equivalent) is required.

You should be already familiar with the following sections in this book: Mathematics for Machine Learning.

• Section 2: Subsections 2.1, to 2.6 (inclusive)
• Section 3: All subsections
• Section 4: Subsections 4.1 to 4.5.1 (inclusive)
• Section 5: Subsections 5.1, 5.2, 5.3, 5.4, 5.5, 5.7
• Section 6: Subsections 6.1 to 6.5 (inclusive)

The course is intended for hard-working, technically skilled, highly motivated students. Participants will be expected to display initiative, creativity, scientific rigour, critical thinking, and good communication skills.

If you do not have the necessary pre-requisites, then you have to spend a lot of time in this course (more than what is required for a 3-credit course).

Useful Online Courses covering the Prerequisites

While I do not expect you to know everything from the following courses, I recommend you to do these video courses at some point in the future if you are serious about doing Reinforcement Learning.

• Prof. Gilbert Strang’s video lectures on linear algebra.
• Prof. John Tsitsiklis’s video lectures on Applied Probability.
• Prof. Krishna Jagannathan’s video lectures on Probability Theory.
• Prof. Deepak Khemani’s video lectures on Artificial Intelligence.
• My video lectures on Machine Learning.

Video Recordings

The lectures and tutorials will be recorded and released to the public. By registering for the course, you agree for the recording and release of videos.

If you do not want your video to be visible while asking questions, you can turn off your video. If you do not want your audio to be part of the recording, you can ask your questions in chat or during office hours.

Programming Language

We will use Python 3 in all the assignments. All the programming tutorials will be only in Python 3.

Evaluation Criteria

The class grade will be based on the following components:

• 3 Theory/Programming assignments (individual) - 45%
• Course Project (team of 3) - 30%
• One endterm examination - 25%

We will use gradescope for all the assignments and projects. More detailed instructions on how to use gradescope will be released in the beginning of the course.

Late Submissions

If you submit your assignments and project reports after the deadline, we will follow the following penalty scheme:

• You will be penalized 5% if your submission is within 24 hours (1 day) from the deadline.
• You will be penalized 10% if your submission is after 24 hours from the deadline and within 48 hours (2 days) from the deadline.
• You will be penalized 20% if your submission is after 48 hours from the deadline and within 72 hours (3 days) from the deadline.
• You cannot submit your assignments/reports after 72 hours from the deadline.