| Unit name | Learning in Autonomous Systems |
|---|---|
| Unit code | COMSM0305 |
| Credit points | 10 |
| Level of study | M/7 |
| Teaching block(s) |
Teaching Block 1 (weeks 1 - 12) |
| Unit director | Dr. Tim Kovacs |
| Open unit status | Not open |
| Pre-requisites |
None |
| Co-requisites |
None |
| School/department | Department of Computer Science |
| Faculty | Faculty of Engineering |
The aim of this unit is to equip students with knowledge of the fundamental features, challenges and methods of learning from reinforcement. This unit introduces the Reinforcement Learning (RL) paradigm, in which feedback to the learner consists only of rewards and punishments. Training a dog is an appropriate analogy. This contrasts with the more common supervised learning paradigm in which a teacher coaches the learner on a set of correctly labeled examples, for example training a classifier to diagnose disease in X-ray images based on a set of examples labeled by a doctor. The subtle difference in feedback between the two paradigms has many implications. We define RL broadly and include all methods which learn from reward signals including, for example, genetic algorithms. RL can model a wide range of learning scenarios, from the simpler case of finding the optimum of an unknown function to the more complex case of a set of autonomous agents interacting within some environment, each trying to achieve their own goals. Applications of RL range from learning control systems for autonomous agents, to optimisation of industrial processes, to modelling human and animal cognition. We will cover a range of approaches including stochastic search, dynamic programming, temporal difference, and monte carlo methods and explore their relationship. We will cover both theoretical foundations and implementation.
After successfully completing this unit you will be able to: Understand how reinforcement learning differs from supervised learning, and the additional challenges it involves; Model appropriate optimisation problems as reinforcement learning tasks; Select appropriate reinforcement learning methods for a given task; Implement basic methods, and some advanced methods, of reinforcement learning.
Approximately 20 hours of lectures. 80 hours are nominally set aside for coursework, private study, etc.
100% coursework. This may include a multiple choice class test, but not a formal exam.
Reinforcement Learning. An Introduction. R. Sutton and A. Barto. MIT Press. 1998. ISBN: 0-262-19398-1 Essential
