Course Syllabus for

Deep Reinforcement Learning (Fall 2021)


Course Description

This course introduces deep reinforcement learning (RL), one of the most modern techniques of machine learning. Deep RL is often seen as the third area of machine learning, in addition to supervised and unsupervised areas, in which learning of an agent occurs as a result of its own actions and interaction with the environment.Deep RL has attracted the attention of many researchers and developers in recent years due to its wide range of applications in a variety of fields such as robotics, game playing, computer vision, natural language processing,consumer modeling and healthcare.This course will provide a solid introduction to the field of deep RL and students will learn about the core challenges and approaches.

Prerequisites

Introductory deep learning,probability and statistics, calculus, linear algebra, and proficiency in Python programming. We formulate value (cost) functions and perform optimization. Students are expected to be comfortable taking derivatives. Basic knowledge of probability theory (in particular, conditional probability distributions and conditional expectations) is necessary.

Course Goals

By the end of this course, students should be able to do the following:
· Learn how to define deep RL tasks and the core principals behind deep RL, including policies, value functions,deriving Bellman equations;
· Implement in code common algorithms following code standards and libraries used in deep RL;
· Understand and work with value-based methods and approximate solutions (deep Q network based algorithms);
· Learn the policy gradient methods and Actor-Critic methods;
· Explore imitation learning tasks and solutions;
· Recognize current advanced techniques and applications in deep RL.

Topics


Session Lecture Contents Time
1

 Introduction to Reinforcement Learning

-RL basics and Course overview

 Week 1
2

 Policy Gradients

 Week 2
3

 Actor-Critic Algorithms

 Week 3
4

 Value Function Methods

 Week 4
5

 Deep RL with Q-functions

 Week 5
6

 Model-based Reinforcement Learning

 Week 6
7

 Exploration

 Week 7
8

 Midterm Review

 Week 8
9

 Offline Reinforcement Learning

 Week 9
10

 Deep RL Theory and Algorithm Design

-Theoretical problems and Algorithmic problems
-Advanced policy gradient (PPO, TRPO, DDPG)

 Week 10
11

 Variational Inference andGenerative Models

-Inference/Connection between Inference and Control

Week 11
12

 Imitation Learning

 Week 12
13

 InverseReinforcement Learning

 Week 13
14

 Transfer Learning and Multi-Task Learning

 Week 14
15

 Meta-Learning

 Week 15
16

 Final Review and Project Due

 Week 16

Instructors

·Dr. Donglin Wang WangDongLin@westlake.edu.cn
·Teaching Assistants: Jinxin Liu, Zifeng Zhuang.

Credit and Hours

·Terms: Fall 2021
·This course has 2 credits and takes 32 class hours for teaching.

Textbook

·Reinforcement Learning: An Introduction Richard S. Sutton and Andrew G. Barto. Second edition, in progress.
·Deep Reinforcement LearningFundamentals, Research and Applications Dong, Hao, Ding, Zihan, Zhang, Shanghang. Springer，2020.

Grading Scheme

The due date of quizzes, course project and final grading scheme are given as follows:

Attendance Course Homework
10% 40% 50%



Westlake University
Shilongshan ST #18, Xihu District, Hangzhou, Zhejiang Province, CN
中国浙江省杭州市西湖区云栖小镇石龙山街18号
2021 BY Westlake Deep Transfer and Meta Learning  浙ICP备16029590号