Lecture notes are available here.

Lecture scribes are available here.

Final Exam
Date: 13th December, 1.30 pm

Class Schedule

Lec Date Topic Lecture Notes Lecture Videos Reference Readings
1 01 Sep 2021 Introduction to RL, Immediate RL Logistics Slides, Chapter-1, Chapter-2 Part1, Part2, Part3 SB Chapter 1 and 2 Computing Machinery and Intelligence by Alan Turing,
Probability Review,
Linear Algebra Review
2 08 Sep 2021 Immediate RL, Markov Decision Processes Chapter-2, Chapter-3 Part1, Part2, Part3 SB Chapter 2 and 3 Reward is enough
3 15 Sep 2021 MDP, Dynamic Programming Chapter-3, Chapter-04 Part1, Part2, Part3 SB Chapter 3 and 4  
4 22 Sep 2021 Monte-Carlo Methods Chapter-05 Part1, Part2, Part3 SB Chapter 5  
5 29 Sep 2021 TD Learning, n-step Bootstrapping Chapter-6 Part1, Part2, Part3 SB Chapter 6 and 7  
6 06 Oct 2021 Function Approximation with on-policy methods Check the lecture scribes. Part1, Part2, Part3 SB Chapter 9 and 10  
7 20 Oct 2021 Non-linear Function Approximation, FA with off-policy methods, Average Reward RL Chapter-8 Part1, Part2, Part3 SB Chapter 10 and 11 DQN, Deadly Triad
8 27 Oct 2021 Policy Gradient Methods Check the lecture scribes. Part1, Part2, Part3 SB Chapter 13 A3C
9 03 Nov 2021 Planning and Model-based RL Check the lecture scribes. Part1, Part2, Part3 SB Chapter 8  
10 10 Nov 2021 Model Predictive Control, Eligibility Traces Slides, Check scribbles for MPC. Part1, Part2, Part3 SB Chapter 12  
11 17 Nov 2021 Hierarchical RL Chapter-12 Part1, Part2, Part3 Hierarchical Approaches, HRL Survey  
12 24 Nov 2021 Partial Observability, POMDPs, Offline RL Check the lecture scribes. Part1, Part2, Part3 POMPD Solutions, Offline RL Review  
13 01 Dec 2021 Course Summary, Further Challenges in RL Check the lecture scribes. Part1, Part2, Part3    

Tutorials

Lec Date Time Topic Lecture Videos Lecture Materials  
1 03 Sep 2021 9 am to 10 am Linear Algebra Video Linear Algebra  
2 10 Sep 2021 4 pm to 5.30 pm Python, NumPy, Matplotlib Video Colab Notebook  
3 14 Sep 2021 9 am to 10 am Probability Video Probabilty  
4 27 Oct 2021 4 pm to 5.30 pm Pytorch Video Slides and Colab Notebook