Full video of the project:

Project date:

July 2019

Project contributors:

I am the sole project designer

Project hardware:

I used a UR3 robot, a raspberry pi 3, a claw gripper with servo motor, two USB webcam and two laptops running Linux.

Project Description

The goal of this project is to pick up an apple remotely over the internet.

One laptop is connected to the robot and a raspberry pi and gripper and two webcams. One webcam is fitted inside the gripper, and the other webcam is attached to the base of the robot. All systems communicate using the Robot Operating System or ROS. This laptop is also connected to the internet.

The operator laptop is in another part of the UTS campus. But it could have been anywhere in the world with an internet connection. It uses Skype and Facebook messenger for video conferencing with the two webcams attached to the robot. It is also connected to the main program running on the laptop near the robot.

The operator has eight keyboard commands. These are forward, backward, left, right, up, down, open-gripper and close-gripper. Each linear command will move the end effector of the robot by half a centimetre. Each gripper command will rotate the fingers of gripper by 15 degrees. The commands will be sent in a string of varying size. Therefore, the operator can input any configurations of command before hitting the enter key for sending.

For example, the operator can press three left key, four down key and one forward key and then press enter. The program on the other laptop will execute each command in the same order. It calculates the required joint angles to move the end effector of the robot to the point that is 1.5 centimetres to the left, 2 centimetres down and 0.5 centimetres forward of the current point. In other words, it calculates the effects of each command in order and add them up and then calculates the joint angles to reach the final point in one single move. It then sends these joint angles to the robot to execute.

Also, at each step of adding commands, it checks for self-crash or crash to the ground. If it sees a crash due to a given command, it will only execute the commands that came before that command and ignores any command after that. If the command that causes the crash is the first command in the string, then no movement will be executed. This is to make sure the operator cannot crash the robot even if he/she tries.

It will do the same for the gripper. It will calculate what the angle of gripper should be based on the string of input commands and then send the results to the raspberry pi to execute. Of course, it will not allow the gripper to be closed or opened more than it can even if the operator commands it.

The Raspberry pi node is implemented in C, collision avoidance and kinematics algorithms are implemented in one node in MATLAB, and these nodes communicate with each other and the UR3 robot using the Robot Operating System or ROS.

The above video isn't in real-time and has sped up.

You see that the apple falls in the first try. This is because the gripper used here is a cheap and low-quality part. It sometimes cannot hold the weight of the apple even though it's fully closed.