Project dates:

Project started in December 2019

Project contributors:

I am the sole project designer

Project hardware:

I use a virtual reality headset (the Oculus Quest), a PC running Windows, a laptop running Linux. I also use a UR3 robot, a claw gripper with servo, arduino and two cameras.

Project Description:

The goal of this project is to control a robot remotely (over the Internet) using a virtual reality headset and perform tasks such as pick and place of small objects and opening doors. This is called Telerobotics.

Applications of this system:

Remote teleoperation in hospitals and nursing homes, especially when caring for patients with infectious diseases like COVID-19. Nurses can stay at home, put on their VR headsets and remotely be transferred to robots that are in hospital and work with COVID-19 patients in the safety and convenience of their home. While not every delicate task in a hospital or nursing home can be remotely done by robots, there are a lot of low dexterity tasks that can be done this way. Like cleaning a table, taking the trash out and put it in bins, pushing a wheelchair, bringing food or water for the patient, etc.

Current state of the project

The following video shows the successful pick and place of several small objects. The operator on the left is 44 kilometres away from the robot arm.

The above video is in real-time and has not sped up.

The following video shows the successful opening of a door.

The above video is in real-time and has not sped up.