Integrating perception and problem solving to predict complex object behaviors

D.M.Lyons, S. Chaudhry, Marius Agica and John Vincent Monaco

One of the objectives of Cognitive Robotics is to construct robot systems that can be directed to achieve real-world goals by high-level directions rather than a complex, low-level robot programming. Such a system must have the ability to represent, problem-solve and learn about its environment as well as communicate with other agents. In previous work, we have proposed ADAPT, a Cognitive Architecture that views perception as top-down and goal-oriented, so that perception becomes part of the problem solving process. This approach is linked to a SOAR-based problem-solving and learning framework. In this paper, we focus on the architecture for the perceptive and world modelling components of ADAPT and report on experimental results using this architecture to perceive events in a complex, real-world scenario.

Consider an object, such as a ball, moving in cluttered environment and consider a robot given the objective of intercepting the object. The robot must track the object as it moves and determine how to intercept it. However, in a cluttered environment, it is highly likely the object will collide and rebound from the environment. Tracking can at most handle this on a collision-by-collision basis. In our approach a world modelling system - a 3D game engine - is intimately linked into the visual perception process. Predictions of complex object behavior such as repeated rebounds from the environment are generated from the world modelling system at very high rates and are available as 'imagined' perceptions, directly comparable with actual perceptions.

We present results from this approach using a Pioneer AT3 robot equipped with stereovision to syncronize its visual input with the game engine graphical output, and to use this syncronization to predict and travel to intercept locations for a ball after several rebounds.