| Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision |
| teaching:gsoc2018 [2018/01/22 09:49] – [RobCoG - Robot Commonsense Games] ahaidu | teaching:gsoc2018 [2018/01/22 13:35] – [Topic 6: CRAM -- Visualizing Robot's Imagined World in RViz] gkazhoya |
|---|
| [[http://robcog.org/|RobCoG]] (**Rob**ot **Co**mmonsense **G**ames) is a framework consisting of various open source games and plugins (https://github.com/robcog-iai) written in the Unreal Engine with the intention of collecting and equipping robots with commonsense and naive physics knowledge. Various Game prototypes are created where users are asked to execute kitchen related tasks. During gameplay developed game plugins automatically collecting symbolic and sub-symbolic data. The automatically annotated data is then stored in the web-based knowledge service [[http://www.open-ease.org/|openEASE]]. This allows robots to access it and reason about it. | [[http://robcog.org/|RobCoG]] (**Rob**ot **Co**mmonsense **G**ames) is a framework consisting of various open source games and plugins (https://github.com/robcog-iai) written in the Unreal Engine with the intention of collecting and equipping robots with commonsense and naive physics knowledge. Various Game prototypes are created where users are asked to execute kitchen related tasks. During gameplay developed game plugins automatically collecting symbolic and sub-symbolic data. The automatically annotated data is then stored in the web-based knowledge service [[http://www.open-ease.org/|openEASE]]. This allows robots to access it and reason about it. |
| |
| The games are split into two categories: (1) VR/Full Body Tracking with physics based interactions, where data as close as possible to reality is collected. The users are immersed in a virtual environment and are asked to perform tasks using natural movements. (2) FPS style, web-based games, where the users interact with objects using a keyboard and mouse. This allows for easy crowdsourcing capabilities since these games could be run from a browser (open-ease.org/robcogweb). The data will be less precise for more low level learning, however still valuable at a more higher level (e.g. positioning of objects, the order of executing actions etc.). | The games are split into two categories: (1) VR/Full Body Tracking with physics based interactions, where data as close as possible to reality is collected. The users are immersed in a virtual environment and are asked to perform tasks using natural movements. (2) FPS style, web-based games, where the users interact with objects using a keyboard and mouse. This allows for easy crowdsourcing capabilities since these games could be run from a browser (http://open-ease.org/robcogweb). The data will be less precise for more low level learning, however still valuable at a more higher level (e.g. positioning of objects, the order of executing actions etc.). |
| | |
| | |
| | ===== CRAM - Cognition-enabled Robot Executive ===== |
| | |
| | CRAM is a software toolbox for the design, implementation and deployment of cognition-enabled plan execution on autonomous robots. CRAM equips autonomous robots with lightweight reasoning mechanisms that can infer control decisions rather than requiring the decisions to be preprogrammed. This way CRAM-programmed autonomous robots are more flexible and general than control programs that lack such cognitive capabilities. CRAM does not require the whole reasoning domain to be stated explicitly in an abstract knowledge base. Rather, it grounds symbolic expressions into the perception and actuation routines and into the essential data structures of the control plans. CRAM includes a domain-specific language that makes writing reactive concurrent robot behavior easier for the programmer. It extensively uses the ROS middleware infrastructure. |
| | |
| | CRAM is an open-source project hosted on [[https://github.com/cram2/cram|GitHub.]] It has its own |
| | [[http://cram-system.org|project page]] that provides extensive documentation |
| | and tutorials that help to get started. |
| | |
| ===== Proposed Topics ===== | ===== Proposed Topics ===== |
| |
| Contact: [[team/ferenc_balint-benczedi|Ferenc Bálint-Benczédi]] | Contact: [[team/ferenc_balint-benczedi|Ferenc Bálint-Benczédi]] |
| |
| | ==== Topic 3: Unreal - ROS 2 Integration ==== |
| | |
| | {{ :teaching:gsoc:ue_ros2.png?nolink&200|}} |
| | |
| | TODO |
| | |
| | **Task Difficulty:** The task is to be placed in the medium difficulty level, as it requires programming skills of various frameworks (ROS, Linux, Unreal Engine). |
| | |
| | **Requirements:** Good programming skills in C++. Good knowledge of the Unreal Engine API. Experience with ROS, ROS 2, c++ library linkage in Unreal Engine. |
| | |
| | **Expected Results** We expect to have an integrated communication level with ROS 2 and Unreal Engine on Windows and Linux side. |
| | |
| | Contact: [[team/andrei_haidu|Andrei Haidu]] |
| | |
| | |
| | ==== Topic 4: Unreal Editor User Interface Development ==== |
| | |
| | {{ :teaching:gsoc:ue_editor.png?nolink&200|}} |
| | |
| | TODO |
| | |
| | **Task Difficulty:** The task is to be placed in the easy difficulty level, as it only requires familiarity with the [[https://docs.unrealengine.com/latest/INT/Programming/Slate/|SLATE]] framework from Unreal Engine. |
| | |
| | **Requirements:** Good programming skills in C++. Good knowledge of the Unreal Engine API. Experience with the [[https://docs.unrealengine.com/latest/INT/Programming/Slate/|SLATE]] framework. |
| | |
| | **Expected Results** We expect to have intuitive Unreal Engine UI Panels for editing, visualizing various RobCoG plugins data and features. |
| | |
| | Contact: [[team/andrei_haidu|Andrei Haidu]] |
| | |
| | |
| | ==== Topic 5: Unreal - openEASE Live Connection ==== |
| | |
| | {{ :teaching:gsoc:ue_oe.png?nolink&200|}} |
| | |
| | TODO |
| | |
| | **Task Difficulty:** The task is to be placed in the medium difficulty level, as it required knowledge of various frameworks/libraries (Unreal Engine, openEASE, c++ websocket communication) |
| | |
| | **Requirements:** Good programming skills in C++. Good knowledge of the Unreal Engine API. Experience with c++ websocket based communication. |
| | |
| | **Expected Results** We expect to have a live connection with between openEASE and the Unreal Engine editor. |
| | |
| | Contact: [[team/andrei_haidu|Andrei Haidu]], [[team/asil_kaan_bozcuoglu|Asil Kaan Bozcuoğlu]] |
| | |
| | |
| | ==== Topic 6: CRAM -- Visualizing Robot's Imagined World in RViz ==== |
| | |
| | {{ :teaching:fetch-left-in-hand-cropped.png?nolink&200|}} |
| | |
| | **Main Objective:** CRAM includes a fast simulation engine for developers to test their newly written plans and for robots to try out different parameters of an action before executing it in the real world. Currently, the world is only visualized using raw OpenGL rendering. The objective of this topic is to visualize the robot's simulation world in the ROS visualization tool RViz, including the state of the robot itself, the objects surrounding it and the reasoning processes involved in action execution. |
| | |
| | **Task Difficulty:** The task itself is simple assuming good understanding of ROS principles and basic knowledge of RViz. To that the challenge of learning a small chuck of an existing system (CRAM) is added. So overall task difficulty is considered to be medium. |
| | |
| | |
| | {{ :teaching:fetch-left-in-hand-real-cropped.jpg?nolink&200|}} |
| | |
| | **Requirements:** |
| | * Familiarity with functional programming paradigms: some functional programming experience is a requirement (preferred language is Lisp but Haskel, Scheme, OCaml, Clojure, Scala or similar will do); |
| | * Experience with ROS (Robot Operating System). |
| | |
| | **Expected Results:** We expect operational and robust contributions to the source code of the existing robot control system including documentation. |
| |
| | Contact: [[team/gayane_kazhoyan|Gayane Kazhoyan]] |
