Learning and Execution of Action Categories

The aim of ACAT is to provide artificial systems with abstract, functional knowledge about relationships between actions objects from information sources made for humans. The potential of robots would be broadened greatly if they could make use of the incredible amount of knowledge available to humans, but most of these information sources presuppose “common” knowledge that does not need to be explicitly specified when interpreted by people. ACAT provides robots with this type of information and generates internal knowledge about tasks by creating and storing all required action information into “action categories”. This is done by generating a dynamic process memory by extracting and storing action categories from large bodies of text and image sources.

Action categories are designed to include both action encoding and context information, obtained by combining linguistic analysis with grounded exploration and action simulation. The power of action categories is that the rich context information allows for generalization (for example replacement of objects in an action), and addresses ambiguity and incompleteness in planning.

An example industrial application for ACAT would be to use manuals, made for human workers, to construct robot-executable instructions. This would enable the robot to do certain human tasks without time-consuming programming procedures. The result of the ACAT project is a robot-compiler which translates human understandable information into a program a robot can execute.

Our role within the project is to build an action-verb specific knowledge base using statistical first order representation, learning and reasoning, and create an ontology of action categories. We will also coordinate the planning and execution module of the project, and design and implement extensions of the Cognitive Robot Abstract Machine (CRAM) for concurrent reactive plan execution.

For more information and news, visit


Logo_komplett_r_vekt_Farbe Kopie [Konvertiert]