Open-TeleVision allows VR-type control of remote robot

A combined team of roboticists from MIT and the University of California, San Diego, has developed a new type of remote control for robots called Open-TeleVision that allows for VR-type control. The team has posted a paper describing their robot system on the arXiv preprint server. They also posted videos on YouTube showing the robot and its pilot in action.

AI technology applied to robotics has produced gradual improvements, but to date, there are no autonomous robots that operate as smoothly as a biological creature, including humans. And most in the field do not expect parity any time soon. Some in the field have returned to the idea of remotely controlled robots.

In this new study, the team from MIT and UCSD has made the experience of controlling a remote robot much more immersive by giving the pilot a VR-type experience.

Most VR consumer products are intended for gaming—a player dons a VR helmet/mask and uses control devices to wander around and manipulate objects in a virtual world. The researchers have taken this idea a step forward by using much the same technology to allow a pilot to wander around and manipulate real objects by using the senses and body of a remote robot.

With the Open-TeleVision system, a pilot dons a VR helmet, allowing him or her to see via the robot’s eyes in a stereoscopic manner. The pilot is able to move the robot’s head by turning his/her head, and to manipulate the robot’s arms, hands and fingers by moving his or her own.

Instead of wearing sensors on the arms, hands and fingers, movement of the limbs is captured using remote sensors similar to those in a Kinect system. The system allows a pilot to feel as if they are in a remote location performing activities such as picking up balls and placing them in cups while the robot does the actual work.

The research team demonstrated the utility of the system by allowing a team member at MIT to control a robot at UCSD, while feeling as if they themselves were at the remote site. They suggest, upon further refinement, systems based on Open-TeleVision could be used for applications including remote surgery, search and rescue efforts in dangerous places, or even to explore other planets.

© 2024 Science X Network