Schedule
Tentative Class Schedule
All the lecture scribbles can be found here.
| Lec | Date | Topic | Lecture Notes | Reference | Readings |
|---|---|---|---|---|---|
| 1 | 29 Aug 2022 | Introduction to RL, Immediate RL | Logistics Slides, Chapter 01, Chapter 02 |
SB Chapter 1 and 2 | Computing Machinery and Intelligence by Alan Turing, Probability Review, Linear Algebra Review |
| 2 | 12 Sep 2022 | Immediate RL, MDP | Chapter 02, Chapter-3 | SB Chapter 2 and 3 | Reward is enough |
| 3 | 19 Sep 2022 | DP - value iteration, policy iteration, Monte-Carlo Methods | Chapter-04, Chapter-05 | SB Chapter 4 and 5 | |
| 4 | 26 Sep 2022 | Off-policy Monte-Carlo, TD Learning, n-step Bootstrapping | Chapter-05 Chapter-6 |
SB Chapter 5, 6, and 7 | |
| 5 | 04 Oct 2022 | n-step methods, Function Approximation with on-policy methods | Chapter-6 Chapter-7 |
SB Chapter 7, 9, and 10 | |
| 6 | 17 Oct 2022 | Average reward setting, Function approximation with off-policy methods, Gradient Bandits | Chapter-7, Chapter-8 Chapter-2 |
SB Chapter 10, 11, and 2 | DQN, Double DQN, Prioritized Experience Replay |
| 7 | 24 Oct 2022 | Policy Gradient Methods, REINFORCE, Actor-Critic, A3C | Check scribbles | SB Book chapter-13 and A3C Paper | Policy Gradient with Function Approximation |
| 8 | 31 Oct 2022 | Deterministic Policy Gradient, DDPG, Model-based RL, Dyna | Check scribbles | DPG, DDPG, SB Chapter 08 | |
| 9 | 07 Nov 2022 | MCTS, Random Shooting, CEM, back to policy gradients: TRPO, PPO | Check scribbles | SB Chapter 08, TRPO, PPO | TRPO, PPO |
| 10 | 14 Nov 2022 | Eligibility Traces, Partially Observable MDPs (POMDPs) | Check scribbles | SB Chapter 12, POMDP | |
| 11 | 21 Nov 2022 | Hierarchical RL | Check scribbles | Hierarchical Approaches, HRL Survey | Option-Critic |
| 12 | 28 Nov 2022 | Imitation Learning, Offline RL, Multi-agent RL | Check scribbles | Chapter-18 in Algorithms for Decision Making | Offline RL - Tutorial |
| 13 | 05 Dec 2022 | Course Summary, Frontiers in RL, What is Next? | Check scribbles | Multi-objective RL - Review |
Tutorials
| Lec | Date | Time | Topic | Lecture Videos | Lecture Materials |
|---|---|---|---|---|---|
| 1 | 02 Sep 2022 | 9 am to 10:30 am | Probability | Video | Slides |
| 2 | 06 Sep 2022 | 9 am to 10:30 am | Python, Numpy, Plotting | Video | Notebook |
| 3 | 09 Sep 2022 | 9 am to 10:30 am | Linear Algebra | Video | Notebook |
| 4 | 19 Oct 2022 | 9 am to 10:30 am | PyTorch | Video | Notebook |
| 5 | 09 Nov 2022 | 10 am to 11:30 am | RLHive | Video |