Skip to main content

Unit information: Learning in Autonomous Systems in 2016/17

Please note: you are viewing unit and programme information for a past academic year. Please see the current academic year for up to date information.

Unit name Learning in Autonomous Systems
Unit code COMSM0305
Credit points 10
Level of study M/7
Teaching block(s) Teaching Block 1 (weeks 1 - 12)
Unit director Dr. Tim Kovacs
Open unit status Not open




School/department Department of Computer Science
Faculty Faculty of Engineering


The aim of this unit is to equip students with knowledge of the fundamental features, challenges and methods of learning from reinforcement. This unit introduces the Reinforcement Learning (RL) paradigm, in which feedback to the learner consists only of rewards and punishments. Training a dog is an appropriate analogy. This contrasts with the more common supervised learning paradigm in which a teacher coaches the learner on a set of correctly labeled examples, for example training a classifier to diagnose disease in X-ray images based on a set of examples labeled by a doctor. The subtle difference in feedback between the two paradigms has many implications. We define RL broadly and include all methods which learn from reward signals including, for example, genetic algorithms. RL can model a wide range of learning scenarios, from the simpler case of finding the optimum of an unknown function to the more complex case of a set of autonomous agents interacting within some environment, each trying to achieve their own goals. Applications of RL range from learning control systems for autonomous agents, to optimisation of industrial processes, to modelling human and animal cognition. We will cover a range of approaches including stochastic search, dynamic programming, temporal difference, and monte carlo methods and explore their relationship. We will cover both theoretical foundations and implementation.

Intended learning outcomes

After successfully completing this unit you will be able to: Understand how reinforcement learning differs from supervised learning, and the additional challenges it involves; Model appropriate optimisation problems as reinforcement learning tasks; Select appropriate reinforcement learning methods for a given task; Implement basic methods, and some advanced methods, of reinforcement learning.

Teaching details

Approximately 20 hours of lectures. 80 hours are nominally set aside for coursework, private study, etc.

Assessment Details

50% coursework and 50% exam

Reading and References

Reinforcement Learning. An Introduction. R. Sutton and A. Barto. MIT Press. 1998. ISBN: 0-262-19398-1 Essential