This AI trainer achieved amazing results with dual-arm robot

The Bi-Touch system allows robots to perform manual tasks by interpreting commands from a digital assistant. The recent findings, disclosed in IEEE Robotics and Automation Letters, showcase an AI agent using tactile and proprioceptive feedback to control robotic behavior. This mastery allows precise sensing, gentle interaction, and effective object manipulation.

The tactile dual-arm robotic system was designed using the latest advancements in AI and robotic tactile sensing. The researchers constructed a virtual world with robot arms, tactile sensors, and a reward function to encourage learning.

The researchers utilized Deep Reinforcement Learning (Deep-RL), a cutting-edge technique in robot learning, similar to training a dog with rewards and punishments. This system enables robots to make decisions, learn from trial and error, and, over time, discover the most effective ways to perform tasks.

One remarkable achievement? The robot can safely lift objects as delicate as a single Pringle crisp.

In a press release, lead author Yijiong Lin, from the Faculty of Engineering, eloquently explained the power and efficiency of the Bi-Touch system. "We can easily train AI agents in a virtual world within a couple of hours to achieve bimanual tasks that are tailored towards the touch," Lin stated, adding that these virtual-trained agents could be applied directly to the real world without further training.

The Future of Bimanual Manipulation

Bimanual manipulation is essential for human-level robot dexterity. However, this area has remained underexplored due to the complexity and availability of suitable hardware. The Bi-Touch system transcends these barriers, heralding a new era in robotic technology.

Co-author Professor Nathan Lepora proudly stated, "Our Bi-Touch system showcases a promising approach with affordable software and hardware for learning bimanual behaviors with touch in simulation, which can be directly applied to the real world."

The open-source nature of the developed tactile dual-arm robot simulation promises to facilitate further research and development in various fields.

The Bi-Touch system signifies a monumental step in bridging the gap between the virtual world and real-world applications. Its ability to learn, adapt, and manipulate objects with a gentle touch presents boundless possibilities, reshaping our perspective on what robots can achieve.

With the advent of the Bi-Touch system, the horizon of technological innovation has expanded, promising an exciting future for industries and research alike.

The study was published in IEEE Robotics and Automation Letters

Study Abstract:

Bimanual manipulation with tactile feedback will be key to human-level robot dexterity. However, this topic is less explored than single-arm settings, partly due to the availability of suitable hardware along with the complexity of designing effective controllers for tasks with relatively large state-action spaces. Here we introduce a dual-arm tactile robotic system (Bi-Touch) based on the Tactile Gym 2.0 setup that integrates two affordable industrial-level robot arms with low-cost high-resolution tactile sensors (TacTips). We present a suite of bimanual manipulation tasks tailored towards tactile feedback: bi-pushing, bi-reorienting, and bi-gathering. To learn effective policies, we introduce appropriate reward functions for these tasks and propose a novel goal-update mechanism with deep reinforcement learning. We also apply these policies to real-world settings with a tactile sim-to-real approach. Our analysis highlights and addresses some challenges met during the sim-to-real application, e.g. the learned policy tended to squeeze an object in the bi-reorienting task due to the sim-to-real gap. Finally, we demonstrate the generalizability and robustness of this system by experimenting with different unseen objects with applied perturbations in the real world.