Study Guide

Objective

- Knowledge and understanding:
Students will learn some basic concepts and algorithms of Reinforcement Learning (RL) for intelligent control of robots, and they will understand how to implement RL systems using Python. In an RL system, the "agent" learns "the most rewarding behavior in the environment" and automatically acquires various behavior patterns. Students will gain knowledge of Q-learning and Deep Q-Network as typical and popular techniques of RL.

- Skills:
Students will be able to use Google Colaboratory and OpenAI Gym for some simulations of RL implementation through Python programming. Students will also learn how to build an offline environment with relevant frameworks and tools.

- Practice and Lab work:
Based on sample programs, students will learn how to implement Python program codes for RL. In practice, students will implement several deep-RL systems using libraries such as PyTorch, OpenAI Gym, and Stable Baselines.

Content

1. Studying the overall framework of reinforcement learning and implementing simple programs using Google Colaboratory and OpenAI Gym. (Lecture + Lab work)

2. Studying the theory and principle of temporal difference learning algorithms including Q-learning, and practicing their implementations. (Lecture + Lab work)

3. Studying the principle of Deep Q-learning and practicing its implementation with the hands-on guidance of PyTorch. (Lecture + Lab work)

4. Building a Deep Q-learning application of an autonomous mobile robot simulation. (Lecture + Lab work)

5. Studying the Policy-gradient method for continuous robot control and practicing its implementation through a robot arm control simulation. (Lecture + Lab work)

Evaluation scale

0-5

Assessment criteria, satisfactory (1)

Students know the concepts and structures of typical RL methods including Q-learning and Deep Q-learning. Students can understand Python programs of the typical RL algorithms. Students know the application examples of RL.

Assessment criteria, good (3)

Students can understand the concepts and structures of typical RL methods including Q-learning and Deep Q-learning. Students can partly implement Python programs of the typical RL algorithms by modifying sample codes. Students can explain the application examples of RL.

Assessment criteria, excellent (5)

Students can understand and explain the concepts and structures of typical RL methods including Q-learning and Deep Q-learning. Students can completely implement Python programs of the typical RL algorithms based on sample codes. Students can explain the application examples of RL and have their own ideas about an original application.

Assessment criteria, approved/failed

Students know the concepts and structures of typical RL methods including Q-learning and Deep Q-learning. Students can understand Python programs of the typical RL algorithms. Students know the application examples of RL.

Assessment methods and criteria

Active attendance at all laboratory sessions. Completing practical exercises.

Prerequisites

It is preferable to have basic programming skills in Python and to be able to use basic data structures and algorithms. It is further preferable to have a basic knowledge of deep learning