Rosie has been built around the KUKA-omnidirectional base and KUKA-lightweight LWR-4 arms. Deployed in the Assistive Kitchen environment of the Intelligent Autonomous Systems Group ( it has been in the operation for over more than a year. The main objective of the research performed on- and around the robot is to develop a system with the very high degree of cognition involved. The latter thus envisions a system that can intelligently perceive environments, reason about them, autonomously infer best plausible actions and in the end, also successfully execute them. Following video demonstrates some of the so far developed capabilities:

Technically the robot consists of the following components:



  • mecanum-wheeled omnidirectional platform,
  • two KUKA-lightweight LWR-4 arms (7 DOF) mounted on this platform. they have torque sensors in the joints and provide impedance control using these sensors,
  • two four-fingered DLR-HIT hands mounted on the arms with torque sensors and joint impedance control as well.
  • a Schunk Powercube Pan-Tilt head


  • Microsoft Kinect 3D camera
  • one Hokuyo UTM-30LX laser scanner mounted on a Powercube tilting module for 3D laser scans (shoulder scanner),
  • two Hokuyo URG-04LX laser scanners for navigation and localization
  • a Swiss-Ranger SR4000 Time-of-flight camera,
  • two SVS-Vistek eco274CVGE 2Mpixel gigabit-Ethernet cameras forming a stereo setup,
  • FLIR thermal camera,
  • fingertip sensors for slip, and proximity detection.


All the hardware is integrated in the Robot Operating System and (partly) in YARP. The code can be found in the tum-ros-pkg repository ( Notable software components are grouped into stacks and include: