Google Summer of Code 2018

pracmln is a toolbox for statistical relational learning (SRL) and reasoning and as such also includes tools for standard probabilistic graphical models. pracmln is an implementation of Markov logic networks (MLN) that supports efficient learning and inference in relational domains. Its learning and inference engines are entirely written in the Python programming language. Markov logic networks generalize both first-order logic and probabilistic graphical models and have proven successful in many real-world applications such as natural-language understanding and robot perception. This makes MLNs one of the most general and most powerful representation formalisms for uncertain knowledge.

pracmln was designed with the particular needs of technical systems in mind. Our methods are geared towards practical applicability and can easily be integrated into other applications. The availability of graphical tools makes both learning or inference sessions a user-friendly experience. Scripting support enables automation, and a browser-based web application is intended for teaching and getting familiar with MLNs.

pracmln is an open-source project hosted on GitHub. It has its own project page (http://www.pracmln.org) that provides extensive documentation and tutorials that facilitate getting started with MLNs. It is provided as a pip package in the Python package index (PyPI).

RoboSherlock is a common framework for cognitive perception, based on the principle of unstructured information management (UIM). UIM has proven itself to be a powerful paradigm for scaling intelligent information and question answering systems towards real-world complexity (i.e. the Watson system from IBM). Complexity in UIM is handled by identifying (or hypothesizing) pieces of structured information in unstructured documents, by applying ensembles of experts for annotating information pieces, and by testing and integrating these isolated annotations into a comprehensive interpretation of the document.

RoboSherlock builds on top of the ROS ecosystem and is able to wrap almost any existing perception algorithm/framework, and allows easy and coherent combination of the results of these. The framework has a close integration with two of the most popular libraries used in robotic perception, namely OpneCV and PCL. More details about RoboSherlock can be found on the project webpage.

In the following, we list our proposals for the Google Summer of Code topics that contribute to the aforementioned open-source projects.

Main Objective: The current implementation of pracmln is entirely written in pure Python. The main objective of this project is to port the computationally heavy parts of the learning and inference algorithms to Cython, an extension to Python that allows static compilation of Python modules to C libraries. Cython allows to add static type declarations to Python, which can significantly speed up execution (up to a factor of 1000 compared to regular Python code).

As most of the Python libraries for machine learning and scientific computing (e.g. scikit-learn, numpy) make use of static compilation, it is expected that the practical applicability of Markov logic networks will substanially be pushed to more demanding real-world scenarios.

Task Difficulty: There is alreay a high-quality code base, however, solving this task properly requires understanding and experience in gradient-based optimization, machine learning and logics.

Requirements: Good programming skills in the Python programming language (CPython/Cython), experience in Artificial Intelligence and Machine Learning (ideally SRL technques and logic)

Expected Results: The core components of pracmln, i.e. the learning and inference modules are expected to be ported to the Cython language exploiting the means of implementational optimization provided by Cython.

Contact: Daniel Nyga

Main Objective: In this topic we will develop algorithms that en- able robots in a human environment to recognize objects in diffi- cult and challenging scenarios. To achieve this the participant will develop annotators for RoboSherlock that are particularly aimed at object-hypotheses generation and merging. Generating a hypotheses essentially means to generate regions/clusters in our raw data that form a single object or object-part. In particular this entails the de- velopment of segmentation algorithms for visually challenging scenes or object properties, as the likes of transparent objects, or cluttered, occluded scenes. The addressed scenarios include stacked, occluded objects placed on shelves, objects in drawers, refrigerators, dishwash- ers, cupboards etc. In typical scenarios, these confined spaces also bare an underlying structure, which will be exploited, and used as background knowledge, to aid perception (e.g. stacked plates would show up as parallel lines using an edge detection). Specifically we would start from (but not necessarly limit ourselves to) the implemen- tation of two state-of-the-art algorithms described in recent papers:

[1] Aleksandrs Ecins, Cornelia Fermuller and Yiannis Aloimonos, Cluttered Scene Segmentation Using the Symmetry Constraint, International Conference on Robotics and Automation(ICRA) 2016 [2] Richtsfeld A., M ̈ orwald T., Prankl J., Zillich M. and Vincze M. - Segmentation of Unknown Objects in Indoor Environments. IEEE/RSJ International Conference on Intelligent Robots and Sys- tems (IROS), 2012.

Task Difficulty: The task is considered to be challenging, as it is still a hot research topic where general solutions do not exist.

Requirements: Good programming skills in C++ and basic knowledge of CMake and ROS. Experience with PCL, OpenCV is prefered.

Expected Results: Currently the RoboSherlock framework lacks good perception algorithms that can generate object-hypotheses in challenging scenarios(clutter and/or occlusion). The expected results are several software components based on recent advances in cluttered scene analysis that are able to successfully recognized objects in the scenarios mentioned in the objectives, or a subset of these.

Contact: Ferenc Bálint-Benczédi