DARMA Bridge: Robotic simulation moving to real-world scenarios

DARMA is a collaboration group which aims at developing learning-based control strategies for robotic manipulation of deformable objects. The DARMA industry bridge project, now running for one year, has enabled the transition of the DARMA project from simulation-based reinforcement learning for robotic manipulation to real-world experimental scenarios.

DARMA (DAta-driven foundations for Robust deformable object MAnipulation) is a collaboration group formed under a WASP collaborative project grant which aims at developing learning-based control strategies for robotic manipulation of deformable objects. The project is led by Yiannis Karayiannidis, Associate Professor at Chalmers University of Technology and Florian Pokorny, Associate Professor at KTH Royal Institute of Technology in cooperation with their industrial partners ABB and Algoryx.

ABB YuMi robotic setup

Countless manufacturing and everyday tasks require handling of non-rigid objects. Yet, the dynamics of deformable objects are complex and inherently difficult to model and simulate. This makes robotic manipulation of such objects using conventional control a challenge. DARMA initially focused on simulation experiments, and the aim of the Bridge project was to enable the move to real-world experimental scenarios:

-The bridge project has facilitated experimental evaluation of our fundamental research results and expanded the range of possible applications for DARMA, says Yiannis Karayiannidis.

Within the framework of the DARMA industry bridge project, the team at Chalmers created a complete experimental setup that includes a workspace for demonstrating cable manipulation and cable routing problems, which are of high industrial interest.

The system consists of a dual-arm ABB YuMi equipped with both vision and haptic (F/T and tactile sensors) sensing capabilities. The DARMA industry bridge project was instrumental to purchase the sensors for the robotic setup.

Dataset for planar manipulation

At KTH, the focus has been on data-driven understanding of planar manipulation of linear deformable objects such as ropes and the prediction of linear deformable object/manipulator interactions.

As part of the project, the group iterated on simulation setups and large-scale simulation data collection, as well as custom hardware for data-collection for this purpose. A large simulation dataset was collected and have iterated on a custom data-collection setup which is now ready to be used to obtain a first large scale real-world dataset for planar manipulation that the group hope to release in the coming months.

Interactions with WARA Robotics

The experimental setup built at Chalmers has been utilized for cable routing use cases: planning for vision-based cable routing and force-based cable routing with tension. Currently, it is used to collect data for offline reinforcement learning using only real data.

The simulator will be used by a visiting PhD student for investing sim-to-real aspects for force-based bimanual cable routing. The researchers also plan to use the simulator for validating results on force-based in-hand object tracking.

The robotic experimental setup built at Chalmers with the support of Bridge has attracted the interest of industrial partners ABB and Algoryx and further strengthen the collaboration in parallel with their interactions through the research arena WARA Robotics:

-Interactions with industries enable us to take on real-worlds problems and transform our research to “products” that can benefit the society, says Yiannis Karayiannidis.

The experimental setup can complement WARA Robotics’ experimental repertoire and enable further collaboration with Algoryx for sim‐to‐real experiments in applications where force information is needed, not limited to deformable objects.

In addition, the setup made the activities visible to an international audience and the project has received requests from international PhD students to use the experimental and simulation setup.

The WASP Industry Bridge instrument supports initial steps in forming foundations for research collaborations between academy and industry. A call for Industry Bridge was open until January 10^th 2023.