The Blender RobotDesigner of the Neurorobotics Platform (NRP)
Introduction
Welcome
This is the documentation of the RobotDesigner belonging to the Neurorobotics Platform (NRP) 1 of the Human Brain Project (HBP) developed in the sub-project Neurorobotics 2. It is realized as a plugin for the Blender 3D modeling suite written in the Python language. This document provides information about the plugin, its installation, its features and usage, as well as the programming reference for developers.
About
The NRP RobotDesigner software developed in the scope of the Human Brain Project in the sub-project SP10 Neurorobtics. It is part of the Neurorobotics Platform which aims at providing easy accessible and usable tools to neuro-scientists and neuro-roboticists for simulating robots that are controlled by artificial brains in virtual environments. The RobotDesinger is a modeling tool that allows to generate geometric, kinematic and dynamic models, and sensor placement that can be used in the simulation environment of NRP–or any ROS/Gazebo-based environment. While other services of the NRP are created as web-based services that do not require installation and, therefore, administration expertise, the RobotDesigner is a plugin for the freely available (GPL licensed) Blender 3D modeling suite. This design decision has been made due to three reasons:
The full capabilities of a feature-rich modeling suite such as Blender is an enormous undertaking.
Robot design is a process that requires advanced engineering expertise and is time consuming justifying a more complex installation process and interruption of the web-based workflow.
Blender is freely available for a wide range of platforms and installation is very easy.
It is easily extendible.
Under the hood, the plugin extends the built-in data types and provides an improved and clear interface to the modeling software. Future development plans of the robot designer, however, include provide a simplified web-based interface integrated into the NRP to construct robots from building blocks. Therefore, the code base of the standalone software is planned to be used and will be further maintained.
A comprehensive description of the Robot Designer can be found in the manuscript “The Neurorobotics Platform Robot Designer: Modeling Morphologies for Embodied Learning Experiments” 3.
The RobotDesigner project was started under the lead of the Intelligent Systems and Production Engineering department of the FZI Forschungszentrum Informatik in Karlsruhe. Currently the RobotDesigner is developed and maintained towards NRP compliance and biomimetic robot design capabilities under the lead of Benedikt Feldotto at the Chair of Robotics, Artificial Intelligence and Real-Time Systems at Technical University of Munich.
History
The foundations of the NRP RobotDesigner has been initially developed under the name OpenGRASP RobotEditor from 2008-2012 at the Humanoids and Intelligence Systems Lab (HIS) at the Institute for Anthropomatics and Robotics (IAR) of the Karlsruhe Institute of Technology during the European GRASP project 4 5 as port of the OpenGRASP software. This simulation environment has been published under the GPL v2 license. This Python plugin restricted to the Blender version 2.49 was mainly developed by Stefan Ulbrich (FZI). One of the most notable features of the RobotEditor was support for the Collada v1.5 data 3D asset exchange format. This file format has large support from the industry including companies such as Daimler and Sony. The RobotEditor was the only freely available tool creating valid models for Collada v1.5 which supported the new description facilities for kinematics and dynamics.
Afterwards, the RobotEditor 6 has been completely rewritten to comply the programming interface of newer Blender versions (>2.69) at the High Perfomance Humanoid Technologies Lab (H2T) at the (IAR) of KIT by the main authors Michael Bechtel and Stefan Ulbrich (FZI). Many new feature such as physical properties, joint dynamics, sensors for matching motion capture data support for the Simox robot simulator 7 have been added to the software. This software is still developed and available under the open source GPL license from here.
The NRP RobotDesigner is a fork of the RobotEditor, which has been chosen as the basis after a comparison to competing projects (e.g., phobos). It will enrich the existing project by components required for the NRP (e.g., for communication/file exchange), additional file formats, support for simulators, an installer and an adapted user interface.
Background
The Human Brain Project 8
The European Commission has selected the Human Brain Project as one of its two Flagship projects. Over the course of 10 years, the HBP will create an integrated system of six ICT platforms, one dedicated to Neurorobotics. The Neurorobotics Subproject of the HBP is coordinated by a research group led by Prof. Alois Knoll. The Human Brain Project is part of the FET Flagship Programme, which is a new initiative launched by the European Commission as part of its Future and Emerging Technologies (FET) initiative. The goal is to encourage visionary, “mission-oriented” research with the potential to deliver breakthroughs in information technology with major benefits for European society and industry. The Commission envisages the Flagship program as a highly ambitious initiative involving close collaboration with National and Regional funding agencies, industry and partners from outside the European Union. Read more…
The Neurorobotics Platform 9
The Neurorobotics sub-project will develop the Neurorobotics Platform which will offer scientists and technology developers a software and hardware infrastructure allowing them to connect pre-validated brain models to detailed simulations of robot bodies and environments and to use the resulting neurorobotic systems in in silico experiments and technology development. The Neurorobotics Platform allows researchers to design and run simple experiments in cognitive neuroscience using simulated robots and simulated environments linked to simplified versions of HBP brain models. The Neurorobotics Platform includes a Robot Designer, an Environment Builder, and a Closed-Loop engine, as well as the Neurorobotics Platform host facility. The Neurorobotics Platform will exploit the 3D modelling capabilities provided by commercial and open source gaming platforms. The HBP Neurorobotics Platform allows researchers to conduct closed-loop experiments, in which a virtual robot is connected to a brain model, running on the HPC platform or on neuromorphic hardware (E.g., the SpiNNaker boards ). Although the capabilities to model virtual robots and environments already exist, and although various labs have created closed-loop set-ups with simple brain models, the HBP platform is the first to couple robots to detailed models of the brain. This will make it possible to perform experiments exploring the link between low level brain circuitry and high-level function.
The Neurorobotics Platform will provide a flexible environment in which researchers can perform experiments using simulated robots connected to different classes of brain model – simplified versions of the models provided by the Brain Simulation Platform, high level models coming from the Cognitive Architectures and Theoretical Neuroscience sub-projects.
Installation
Installation with Installer Script
We currently support Blender version 2.8.
To install the plugin on Linux and Windows follow these steps:
Download a supported Blender version from the dedicated website. Windows users download and install the *.msi installer. Go to Blender download page
Download the Robot Designer installation file “robotdesigner_installer.blend” from our GitHub repository . Download installation file
Launch Blender and open the downloaded “robotdesigner_installer.blend” file, then click on the triangle above the script to execute it. The installation may take a bit of time while Blender is remaining passive. Once the installation is done you will see the Robot Designer plugin opening on the right side along with the “Tool”, “View” and “Edit” tabs, as shown in the image below.
After running the Robot Designer installation script (on the left) you will find the plugin opening up on the right side of the 3D view.
If the plugin does not pop up, you will find it by opening the pane on the right of the 3D view with clicking on the small arrow.
Troubleshooting
Start Blender from terminal to see log output during the installation process and runtime.
In newer versions you can open the terminal view via
Window->Toggle System Console.If the plugin does not show up or does not show up anymore after restarting Blender make sure it is enabled: Navigate to
Edit->Preferences->Add-Onstab. Search for the NRP Robot Designer add-on and check the box to enable the plugin. You may need to restart Blender thereafter.If you see the error
No module named 'pyxb'in the terminal try the following:In your terminal go to
[blender-version]/[version]/python/binand run./python3.7m -m pip install pyxb==1.2.5Additional logs during runtime of the plugin are written to
blender_module_path/robot_designer_plugin/resources/log.txtUbuntu:
blender_module_path = $HOME/.config/blender/[version]/scripts/addons/Windows:
blender_module_path = C:\Users\[profile]\AppData\Roaming\Blender Foundation\blender\[version]\scripts\addons
Features
Implemented Features
Installer
The installer comes in a form of a
.blendfile that contains an installer script that can be directly executed from within Blender. That way, it can detect the used operating system and Blender version and link the files to the correct location as well as select the correct binaries for the platform.Robot modeling
The robot designer adds functionality to the Blender software with respect to robotics:
Kinematic modeling in a scientific/engineering way (e.g., entering transformations in Denavit-Hartenberg or Euler convention)
Editing of dynamic properties (center of mass and distribution, friction, etc.) and automated computation from mesh
Automatic mesh generation
Creation of collision models using geometries with fixed size of vertices and safety distance
Convex hull computation
Conversion from deformable meshes to rigid bodies. This is useful to transform deformable actors such as those created by MakeHuman into robots. This is used to provide a standard humanoid robot model to the NRP.
Generation of links and joint geometries based on the kinematic description (still experimental).
Placing of sensors (*Note: Import and export of sensors is in development)
Graphically defining muscles and its pathpoints on skeletal models, including muscle wrapping surfaces.
Building of world files with multiple robots and defining environment parametrization.
File format and ROS/Gazebo support
In order to interchange models with the Neurorobotics platform the Robot designer has to support additional file formats.
In the file section robot models can be imported and exported as single files or zipped packages in the Simulator Description Format (SDF) file format. It will be enriched by additional parameter tags supported by the Gazebo simulator–especially for supporting a plugin developed for the NRP to include joint controllers directly in the robot description file. This file support relies on the PyXB package–a software that translates XML scheme definitions (XSD) into a Python document object model.
Plugin Core Framework
Although the RobotEditor is already feature rich, development is cumbersome due to Blender’s conventions for plugin design. The
Plugin Coreframework is a python package that abstracts and simplifies many of the boiler plate thus allowing the design of larger applications based on Blender. The RobotEditor, therefore, had to be refactored and rewritten in large parts to comply with this framework.Especially the dynamic nature of how functionality is added to Blender makes a modern Python development with IDEs (Integrated Development Environments) such as the excellent PyCharmTM which support code completion and refactoring difficult. By using decorators (PEP 0318) and handlers for Blender Operators and Properties, and extended exception handling and logging, developers can easily create even larger projects comfortably. Integration of external debugging is planned and currently under development. Further, mock ups for the Blender API can be generated and used for code completion.
Current development focuses on extending the framework to support static type checking using MyPy and code analysis (PyLint) on plugin loading.
Documentation and coding standards
The RobotDesigner comes with extensive documentation in form of a user’s and developers manual which explains all steps necessary to setup and run the software as well on how to extend it and use the
Plugin Corein general. The code aims at being well-documented (the original code base is currently in the process of being documented) and to meet coding standards such as PEP 8.
Planned features
One of the key aspects of the ongoing development is data persistence,
that is, the ability to store robot models in different file formats and
different storage mechanisms. More file formats will be supported upon demand.
By the inclusion and abstraction of the PyXB interface XML-based systems can be integrated easily.
As well, further model properties and in particular sensor modalities including its various sensor parameters will be implemented.
How to cite
If you find the Robot Designer helpful and use it in your work, please cite the Robot Designer paper: “The Neurorobotics Platform Robot Designer: Modeling Morphologies for Embodied Learning Experiments”
Feldotto B, Morin FO and Knoll A (2022) The Neurorobotics Platform Robot Designer:
Modeling Morphologies for Embodied Learning Experiments. Front. Neurorobot. 16:856727.
doi: 10.3389/fnbot.2022.856727
The corresponding bibtex entry is:
@ARTICLE{10.3389/fnbot.2022.856727,
AUTHOR={Feldotto, Benedikt and Morin, Fabrice O. and Knoll, Alois},
TITLE={The Neurorobotics Platform Robot Designer: Modeling Morphologies for Embodied Learning Experiments},
JOURNAL={Frontiers in Neurorobotics},
VOLUME={16},
YEAR={2022},
URL={https://www.frontiersin.org/article/10.3389/fnbot.2022.856727},
DOI={10.3389/fnbot.2022.856727},
ISSN={1662-5218},
License
Similar to its predecessor the RobotEditor the NRP RobotDesigner is published as open source software under the GPLv2 license.
Acknowledgement
This open source software code was developed in part or in whole in the Human Brain Project, funded from the European Union’s Horizon 2020 Framework Programme for Research and Innovation under Specific Grant Agreements No. 720270, 785907 and 945539 (Human Brain Project SGA1, SGA2 and SGA3).
Footnotes
- 1
Falotico, E., Vannucci, L., Ambrosano, A., Albanese, U., Ulbrich, S., Vasquez Tieck, J. C., Hinkel, G., Kaiser, J., Peric, I., Denninger, O., & others. (2017). Connecting artificial brains to robots in a comprehensive simulation framework: The Neurorobotics Platform. Frontiers in Neurorobotics, 11, 2.
- 2
Knoll, A., Gewaltig, M.-O., Sanders, J., & Oberst, J. (2016). Neurorobotics: a strategic pillar of the Human Brain Project. Science Robotics.
- 3
Feldotto B, Morin FO and Knoll A (2022) The Neurorobotics Platform Robot Designer: Modeling Morphologies for Embodied Learning Experiments. Front. Neurorobot. 16:856727. doi: 10.3389/fnbot.2022.856727
- 4
Funded by the European Commission through its Cognition Unit under the Information Society Technologies of the seventh Framework Programme (FP7)
- 5
B. Leon, S. Ulbrich, R. Diankov, G. Puche, M. Przybylski, A. Morales, T. Asfour, S. Moisio, J. Bohg, J. Kuffner and R. Dillmann, “OpenGRASP: A Toolkit for Robot Grasping Simulation”, 2nd International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR), November 15, 2010
- 6
N. Vahrenkamp, M. Kröhnert, S. Ulbrich, T. Asfour, G. Metta, R. Dillmann and G. Sandini, “Simox: A Robotics Toolbox for Simulation, Motion and Grasp Planning”, International Conference on Intelligent Autonomous Systems (IAS), pp. 585 - 594, 2012
- 7
C. Mandery, Ö. Terlemez, M. Do, N. Vahrenkamp and T. Asfour, “The KIT Whole-Body Human Motion Database”, International Conference on Advanced Robotics (ICAR), pp. 0 - 0, July, 2015
- 8
- 9