Schedule
Tentative Class Schedule
All the lecture scribbles can be found here.
| Lec | Date | Instructor | Topic | Mandatory Readings | Optional Readings |
|---|---|---|---|---|---|
| 1 | 26 Aug 2024 | Sarath Chandar | Introduction to Reinforcement Learning, Sequential Decision Problems | Logistics and Intro Slides, Sutton and Barto Chapter 01, Reward is Enough, Scalar Reward is Not Enough, Faulty Reward Functions | Computing Machinery and Intelligence by Alan Turing, Probability Review, Linear Algebra Review |
| 2 | 09 Sep 2024 | Sarath Chandar | Immediate Reinforcement Learning and Multi-armed Bandits | Sutton and Barto Chapter 02, Proof of UCB1’s regret bound by Ann He and Jeremy Kun | 3 lectures on UCB1 by Prof. Balaraman Ravindran: UCB1, Concentration Bounds, UCB Regret Bound |
| 3 | 16 Sep 2024 | Nishanth Anand | Markov Decision Process | Sutton and Barto Chapter 03 | |
| 4 | 23 Sep 2024 | Nishanth Anand | Dynamic Programming, Monte-Carlo Methods | Sutton and Barto Chapter 04 and Chapter 05 | |
| 5 | 01 Oct 2024 | Nishanth Anand | Monte-Carlo Methods, Temporal Difference (TD) Learning - I | Sutton and Barto Chapter 05 and Chapter 06 | |
| 6 | 07 Oct 2024 | Nishanth Anand | Temporal Difference (TD) Learning - II | Sutton and Barto Chapter 06 and Chapter 07 | |
| 7 | 21 Oct 2024 | Sarath Chandar | Function Approximation - I | Sutton and Barto Chapter 09 | |
| 25 Oct 2024 | Mid-Term Exam [12:45 pm to 2:15 pm] | Last year Mid-Term Questions | |||
| 8 | 28 Oct 2024 | Sarath Chandar | Function Approximation - II | Sutton and Barto Chapter 10, and Chapter 11, DQN, DQN Nature, Double DQN, Rainbow, Prioritized Experience Replay | Dueling DQN, Distributional RL |
| 9 | 04 Nov 2024 | Sarath Chandar | Policy Gradients | Sutton and Barto Chapter 13, Off-Policy Actor-Critic, A3C paper | REINFORCE paper, Soft Q-Learning |
| 10 | 11 Nov 2024 | Sarath Chandar | Determistic Policy Gradients, Natural Policy Gradient | DPG, DDPG, TD3, NPG Explained | cleanrl, Deep RL that Matters |
| 11 | 18 Nov 2024 | Sarath Chandar | Natural Policy Gradients Continued, Eligibility Traces, Multi-Agent RL | NPG Explained, TRPO, PPO, Sutton and Barto Chapter 12, Multi-Agent RL | |
| 12 | 25 Nov 2024 | Sarath Chandar | Model-based RL | SB Chapter 8, AlphaGo, Dreamer | Dreamer v2, Dreamer v3, MuZero |
| 13 | 02 Dec 2024 | Sarath Chandar | Offline RL, Hierarchical RL, Frontiers in RL | CQL, HRL | Continual RL |
| 05 Dec 2024 | Final Exam [1:30 pm to 4 pm] | Last year final exam questions | The exam will be based on the first 12 weeks of lectures. |
Tutorials
| Lec | Date | Time | Topic | Lecture Videos | Lecture Materials |
|---|---|---|---|---|---|
| 1 | 30 Aug 2024 | No lab. Use the lab time for reading. | |||
| 2 | 06 Sep 2024 | No lab. Use the lab time for reading. | |||
| 3 | 13 Sep 2024 | 11:30 pm - 02:45 pm | Online Office Hours by Ali Rahimi-Kalahroudi (link in Piazza) | ||
| 4 | 20 Sep 2024 | 11:30 pm - 02:45 pm | Online Office Hours by Ali Rahimi-Kalahroudi (link in Piazza) | ||
| 5 | 27 Sep 2024 | 11:30 pm - 02:45 pm | Online Office Hours by Esther Derman (link in Piazza) | ||
| 6 | 04 Oct 2024 | No lab. Use the lab time for doing the assignment. | |||
| 07 Oct 2024 | 4:00 pm - 7:15 pm | Online Office Hours by Esther Derman (link in Piazza) | |||
| 7 | 11 Oct 2024 | 1 pm to 2:30 pm | Jax Tutorial by Artem Zholus (in person) | Recording | Colab Notebook |
| 11 Oct 2024 | 2:30 pm to 4:30 pm | Online Office Hours by Esther Derman (link in Piazza) | |||
| 16 Oct 2024 | 10:00 am to 12:00 pm | Online Office Hours by Esther Derman (link in Piazza) | |||
| 16 Oct 2024 | 10:00 am to 11:00 am | Online Office Hours by Sarath Chandar and Nishanth Anand (link in Piazza) | |||
| 21 Oct 2024 | 04:00 pm to 06:00 pm | Online Office Hours by Esther Derman (link in Piazza) | |||
| 8 | 25 Oct 2024 | 12:45 pm to 2:15 pm | Mid-term Exam | ||
| 9 | 01 Nov 2024 | No lab. Use the lab time for doing the assignment. | |||
| 04 Nov 2024 | 9 am to 11 am | Online Office Hours by Artem Zholus (link in Piazza) | |||
| 10 | 08 Nov 2024 | 11:30 pm to 2:30 pm | Online Office Hours by Artem Zholus (link in Piazza) | ||
| 11 | 15 Nov 2024 | 11:30 pm - 02:45 pm | Online Office Hours by Antoine Clavaud (link in Piazza) | ||
| 12 | 22 Nov 2024 | 11:30 pm - 02:45 pm | Online Office Hours by Antoine Clavaud (link in Piazza) | ||
| 13 | 29 Nov 2024 | 11:30 pm - 02:45 pm | Online Office Hours by Antoine Clavaud (link in Piazza) |