Course info

Language:
English

Appointments:

Description:

This course provides students with the required basic background on machine learning to independently carry out research projects on the hot topic of reinforcement learning, e.g. within the scope of a Bachelor's or Master's thesis. In particular, this class aims at providing the students with fundamental understanding of reinforcement learning algorithms and applications within deep learning.

This class can be taken as a substitute for robot learning this semester. Robot learning will be offered again in the winter semester 2019.

There are two configurations in which students can take this class:

1. Lecture
2. Lecture + Seminar + Project

• Lecture means attending lectures and taking the final exam. Exercises will be provided for self study but will neither be graded nor corrected.
• Seminar means writing a report either alone or in groups of 2-3 people

• Project means writing well-documented code + report in groups of 2-3 people

Contents

(Lecture) Reinforcement Learning: From Foundations to Deep Approaches:

• Review of machine learning background
• Black box Reinforcement Learning
• Modelling as bandit, Markov Decision Processes and Partially Observable Markov Decision Processes
• Optimal control
• System identification
• Learning value functions
• Policy search
• Deep value functions methods
• Deep policy search methods
• Exploration vs exploitation
• Hierarchical reinforcement learning
• Intrinsic motivation

(Seminar) Reinforcement Learning Algorithms and Platforms:
This seminar will cover learning methods and their application in intelligent technical systems. In the context of this seminar, students will train the ability to write a scientific article and present its content similar as at scientific conference.

(Project) Application of Reinforcement Learning Methods:
In this project, students get hands-on experience in reinforcement learning research conducted by a team of students. Small groups of students pursue their own Reinforcement Learning experiment, involving standard platforms (Cartpole, Furuta-Pendulum, etc). Starting from a project idea, students are guided by the lecturer through the whole process of developing the experiment, collecting and analysing data and writing a research report/paper which is ready to publish.

Requirements

(Lecture) Reinforcement Learning: From Foundations to Deep Approaches:
Good programming in Python. Lecture Statistical Machine Learning is helpful but not mandatory.

(Seminar) Reinforcement Learning Algorithms and Platforms:
Simultaneous Participation in "Reinforcement Learning: From Foundations to Deep Approaches" and "Application of Reinforcement Learning Methods".

(Project) Application of Reinforcement Learning Methods:
Simultaneous Participation in "Reinforcement Learning: From Foundations to Deep Approaches" and "Reinforcement Learning Algorithms and Platforms".

Moodle Class

All further information and announcements regarding the lecture, seminar and project will be made public over the Moodle system of the computer science department: https://moodle.informatik.tu-darmstadt.de/course/view.php?id=434

Application Platforms

The platforms used in the class:

• Furuta Pendulum: Boris Belousov boris@robot-learning.de
• Magnetic Levitation: Hany Abdulsamad hany@robot-learning.de
• Ball on a Plate: Fabio Muratore fabio@robot-learning.de
• Inverted Pendulum: Samuele Tosatto samuele@robot-learning.de
• Haptic Manipulator: Simone Parisi simone@robot-learning.de

Feel free to contact us if you are interested in any of these systems.

Algorithms

Students are welcome to suggest their own algorithms/papers. Here we provide a list of the most fundamental and recent successful algorithms for reference:

• DQN
• DDPG
• A3C
• TRPO
• PPO
• NPG
• LSPI
• REPS
• MORE
• MPO
• NAC
• iLQG
• MPC
• PILCO
• RS
• NES