My research focuses on integrating robotic perception and action. I believe that “Through predicting perceptual action consequences based on memory and observation, a robot would be capable of solving novel tasks in a human environment.” However, action and perception are often represented separately–as object models or maps in computer vision systems and as action templates in robotics controllers; due to this separation, classification is often based on labels instead of how the robot can interact with the environment, therefore limiting how learned action can be generalized.
Aspect Transition Graph Model
Instead of an independent object recognition system, I proposed an integrated model based on the aspect transition graph (ATG) representation that fuses information acquired from sensors and robot actions to achieve better recognition and understanding of the environment. An ATG summarizes how actions change viewpoints or the state of the object and thus, the post-action observation.
Belief Space Planning with ATGs
I address these dual problems of modeling and reasoning through the Aspect Transition Graph model that is grounded in the robot’s own actions and perceptions. The description of state is domain general, as it is computed directly from the status of executable actions and not hand built for a specific task. I present a planner that exploits the uniform description of state and the probabilistic models to plan efficiently in partially observed environments.
Object Manipulation with Visual Servoing
I introduce the notion of a slide as a metaphor for this kind of action that transitions from one set of states to another. Uncertainty of the state may increase after transitioning down a slide, but may still reach the goal state if a funnel-slide-funnel structure is carefully designed. I investigate how a sequence of these two kinds of controllers will change how an object is observed.