Apprenticeship learning with few examples

Abdeslam Boularias, Brahim Chaib-draa

Research output: Contribution to journalArticle

3 Scopus citations

Abstract

We consider the problem of imitation learning when the examples, provided by an expert human, are scarce. Apprenticeship learning via inverse reinforcement learning provides an efficient tool for generalizing the examples, based on the assumption that the expert's policy maximizes a value function, which is a linear combination of state and action features. Most apprenticeship learning algorithms use only simple empirical averages of the features in the demonstrations as a statistics of the expert's policy. However, this method is efficient only when the number of examples is sufficiently large to cover most of the states, or the dynamics of the system is nearly deterministic. In this paper, we show that the quality of the learned policies is sensitive to the error in estimating the averages of the features when the dynamics of the system is stochastic. To reduce this error, we introduce two new approaches for bootstrapping the demonstrations by assuming that the expert is near-optimal and the dynamics of the system is known. In the first approach, the expert's examples are used to learn a reward function and to generate furthermore examples from the corresponding optimal policy. The second approach uses a transfer technique, known as graph homomorphism, in order to generalize the expert's actions to unvisited regions of the state space. Empirical results on simulated robot navigation problems show that our approach is able to learn sufficiently good policies from a significantly small number of examples.

Original languageEnglish (US)
Pages (from-to)83-96
Number of pages14
JournalNeurocomputing
Volume104
DOIs
StatePublished - Mar 15 2013
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Cognitive Neuroscience
  • Artificial Intelligence

Keywords

  • Bootstrapping
  • Imitation learning
  • Inverse reinforcement learning
  • Transfer learning

Fingerprint Dive into the research topics of 'Apprenticeship learning with few examples'. Together they form a unique fingerprint.

  • Cite this