CIS Department Talk - March 21, 2007

The Department of Computer and Information Science & The Society of Computer Science Present

Speaker:	Shimon Whiteson, University of Texas at Austin
Topic:	Adaptive Representations for Reinforcement Learning
Date:	Wednesday March 21, 2007, 11:30 am
Place:	John Mulcahy Hall, Room 403

In reinforcement learning, a computer, robot, or other agent seeks an effective behavioral policy for tackling a sequential decision task. One limitation of current methods is that they typically require a human to manually design a representation for the solution (e.g. the internal structure of a neural network). Since poor design choices can lead to grossly suboptimal policies, agents that automatically adapt their own representations have the potential to dramatically improve performance. This talk introduces two novel approaches for automatically discovering high-performing representations. The first approach, called evolutionary function approximation, uses evolutionary methods to optimize representations for neural network function approximators. Hence, it evolves agents that are better able to learn. The second approach, called adaptive tile coding, begins with coarse representations and gradually refines them during learning, analyzing the current policy and value function to deduce the best refinements. Empirical results in multiple domains demonstrate that these techniques can substantially improve performance over methods with fixed representations.

Shimon Whiteson is a doctoral candidate and assistant instructor in the Department of Computer Sciences at The University of Texas at Austin. His research focuses on reinforcement learning for real- world domains that are continuous and stochastic. For his thesis, he developed methods to improve the performance of function approximators for temporal difference methods by automatically optimizing their internal representations. In 2006, he received an IBM PhD Fellowship and two Best Paper Awards at the GECCO-06 conference. He plans to graduate in May 2007.