Project “Learning and Execution of Action Categories (ACAT)”

VMU participates FP7 ACAT: “Learning and Execution of Action Categories (ACAT)”, Theme ICT-2011.2.1 (Cognitive Systems and Robotics), Grant Agreement No. 600578 (2013-2016).

Partners

About the Project:

ACAT focuses on how artificial systems (robots) can understand and utilize information made for humans.

Every day we (humans!) are using many sources of information, for example books, videos, the internet, etc., from which we extract information also about new tasks. For example, if you have never fixed a punctured bicycle tire before, you can access wikiHow (Mend-a-Puncture-in-a-Bike-Tire ) to learn how to do this.

Any of these information sources, however, is vastly incomplete and leaves out all information “that goes without saying”. What does that mean? For example, if a text source tells you to use a hammer for a certain task it will usually not tell you how to hold the hammer (by the handle, not by the tool end), how to wield it (usually hitting with the flat part of the tool end – but there are exceptions to this), let alone how to shape the dynamics of your movement (making a smooth swing). Usually all of this is learned by us during childhood and if not we will – sometimes painfully – find out that a presumably good manual, textbook, internet page, cannot be used, because of some gap in information.

Thus, for many skills, humans have internalized and stored a lot of readily available information and our external textual or pictorial sources tacitly assume this and do not reiterate ‘the obvious’.

The goal of the ACAT project is to provide machines (robots) with this type of tacit information and to generate internal knowledge about individual task by way of creating and storing all required action information into so-called “Action Categories”.

To achieve this, ACAT generates a dynamic process memory by extraction and storage of action categories from large bodies of human compatible sources (text, images). Action categories are designed to include the actual action-encoding but also large amounts of context information (“background”). They are obtained by combining linguistic analysis with grounded exploration and action-simulation.

To make them available to a wide variety of robots ACAT will structure and store the action categories in an action-specific knowledge base.

The ACAT system then uses action-categories to compile robot-executable plans. Execution benefits strongly from the rich context information present in the action-categories because this allows for generalization (for example replacement of objects in an action). It also permits us to specifically address ambiguity, incompleteness and uncertainty in planning.

The ultimate purpose of ACAT is to equip the robot – on an ongoing basis – with abstract, functional knowledge, normally made for humans, about relations between actions and objects leading to a system which can act meaningfully.

As industrially relevant scenario, ACAT uses “instruction sheets” (manuals), made for human workers, and translates these into a robot-executable format. This way the robot will be able to partially take over human tasks without time-consuming programming procedures.

Similar to computer science, where the development of the first compilers had led to a major step forward, the main impact of this project is that ACAT develops a robot-compiler, which translates human understandable information into a robot-executable program.

The ACAT Project consists of researchers from six European institutions across four countries.

University of Göttingen Göttingen, Germany
University of Southern Denmark Odense, Denmark
Aalborg University Aalborg, Denmark
Vytautas Magnus University Kaunas, Lithuania
Jožef Stefan Institute Ljubljana, Slovenia
University of Bremen Bremen, Germany