How to Make a Gesture Controlled Robotic Arm
In this project I will show you how to make a hand-gesture controlled robotic arm from scratch. I will also show you in detail how it works and how all the electronic components are interfaced.
Before we get started, please consider checking out my Instagram page: roboticworx. It has frequent posts on all my new projects and is also a great way to support me if you enjoy my projects. But without any further ado, let’s get to the build!
A Quick Preview:
A video of the finished robot operating is below:
How To Make a Hand Controlled Robotic Arm is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
Summary of How It Works:
Each joint on the robotic arm moves in the direction relative to the rotation of a PCB (printed circuit board) mounted to a standard silk glove. This PCB was custom designed to read and transmit signals from a gyroscope as well as a flex sensor to determine when a hand is closed and how much that hand is rotated. These signals are picked up by a receiver and get interpreted into signals that will move the motors.
Building The Arm:
The robotic arm was designed by me in CAD (computer aided design) software and require only basic materials for assembly such as M3 bolts, nuts, washers, ball bearings and a few NEMA 17 stepper motors. All the components other for the arm are 3D printed in blue, using an Ender 3 Max 3D printer. However, most printers will do the job as long as they have a big enough print bed.
Breaking down the design:
Here is how the design works, the arm has three stepper motors to move each main joint. Two motors for the long arms controlling the pitch and one for the base rotation controlling the yaw. I will point them out here:
For the gripper, it’s just as simple. There are only two servo motors that control the entirety of the gripper. One for rotating the “claw” and the other for closing it. I will point those locations out here:
You may be curious why there appear to be some extra layers to the gripper that aren’t explained. Well, allow me. The two layers floating above where the servos are to be mounted are there to enable quick and easy gripper replacement to something such as a soldering iron holder, etc. They are fit for a magnet so you can easily pop them out and replace them with whatever your heart desires. The magnets I used were 1.26” diameter Neodymium Cup Magnets.
The Magic of Planetary Gearboxes:
You may have noticed that the robotic arm is actually pretty huge. At full stance its around 33 inches, or 84 centimeters. That’s really big for a robotic arm. For these tiny $14 NEMA 17 stepper motors to move joints on this thing is no easy task. Without any sort of speed reduction to create a greater torque, these motors wouldn’t stand a change at moving any of these joints even a tiny bit. Thankfully, we are able to use physics to our advantage. The gearboxes mounted to the arm achieve a gear ratio of around 132.7, meaning that they will move the motors 132.7 times slower in exchange for 132.7 times the torque. The kind if gearboxes I used in this design are called planetary gearboxes because they have 3 planetary gears moving around a sun gear. This is all happening inside a ring gear to keep them from slipping. When you attach a carrier to each of the planetary gears you will achieve a gear reduction of the ((ring gear + sun gear)/sun gear)). These are efficient because you are able to achieve a high gear ratio in a small amount of space.
How my Design Works Off This Concept:
The gearbox I made for the robotic arm uses this concept to achieve such a high gear ratio by using three planetary gearboxes in one. Please keep in mind that just one planetary gearbox normally can’t go past a gear ratio of around ten just on its own. Each of my planetary gearboxes have a gear ratio of 5.1, but by feeding one end into another you are able to multiply the torque of the first by that of the second, etc. Every planetary layer of my gearbox each have a gear ratio of 5.1, but because I feed one end into the other three times, I am able to achieve a gear ratio of 5.1^3 or 132.7 which enables them to have enough torque to lift each joint
More on how to actually assemble the gearbox and robotic arm (bolts and nuts wise) as well as where to press fit the bearings will be at the bottom of the post under “Assembling the Arm.” Please consider subscribing. It helps support my projects and allows me to spend more time making content for you all to enjoy and learn off of :)
Building The Transmitter:
The microcontrollers I used for this project are ESP32 WROOM modules. They function very similarly to Arduino microcontrollers but are smaller and have built in Wi-Fi for wireless communication. To get things started I’ve attached a circuit schematic below illustrating all the connections.
The 9V Vin would be best with 5V, since it would have to step down less voltage, but I used a 9V battery for convince. Just be sure you keep the Vin voltage below 12V, as that is the maximum the voltage regulator on the ESP32 can step down. If you exceed it, it could expose the microcontroller’s fragile components to an unrated voltage and destroy it. Another very important thing to keep in mind is that you cannot ever have the USB plugged into the ESP32 while its being powered externally. It will burn out!
For this part of the project, I developed a PCB to help make the circuit more compact to fit it all on the top of a glove. There are many different kinds of design software that can generate Gerber files from given schematics. Once it is lain out as you’d like, all you need to do is upload it to a PCB site and wait for it to arrive in the mail. Then after that, it’s time to get soldering.
How The Circuit Works:
The circuit consists of a ESP32 microcontroller, MPU6050, three LEDs, a flex sensor, a button and three resistors. The MPU6050 is an amazing chip which you can get rather cheap. It is a two-in-one three-axis accelerometer and gyroscope. However, for this project we will only be using the gyroscope function. The gyroscope works by using what is known as the Coriolis Effect, which causes vibrations that are amplified and filtered to produce a proportional voltage. This voltage is then output into the microcontroller where it is read and manipulated. In this case, it is manipulated by moving the stepper motors connected to the receiver at a rate that is a variable, originated and mapped by that voltage. In my case, I used the value range of 0-180 degrees for ease of understanding and the ESP-NOW wireless communication method. You will need a MPU6050 library for this. As for the connection to the microcontroller, it’s as simple as hooking up the internal clocks (SCL) to sync data transfer and the data line (SDA) for the actual data transfer. As well as the VCC (3.3V) and GND of course.
The flex sensor is even simpler. All it does is produce a resistance correlated to the bend radius. The more it bends, the higher resistance it becomes due to the resistant material becoming more condensed. This value can be picked up by the microcontroller with an analog read function, in which can be sent directly to the receiver wirelessly to control the gripper servo motor.
As far as the LEDs, I used two for telling me if there were any problems at any point during operation. I used a blue one to tell me if there was a problem with the wireless communication, and a red one to tell me if the board was getting power. You will have to use Ohm’s law to calculate the values of the resistors for your LEDs (V=IR), but mine needed >= 160ohms. It is also okay to go slightly above the value if you don’t have the exact resistors, it just may be slightly less bright. The button is used in the circuit so I can switch control of which axis I am moving through literally, a push of a button. What I did was write one pin high so when the button is closed, the input on the other pin it is connected to would be low. This way when it is pressed, so is that pin’s value which I can send to the receiver board to switch control of one joint to the other.
Building The Receiver:
The receiver for this project is nothing too complex. It consists of a ESP32, three A4988 stepper motor drivers, two resistors and two LEDs. However, looking back I highly recommend using heavier duty stepper drivers such as the TB6600. It was pretty foolish of me to use an A4988 here due to the low current rating, however the wiring will be the same. The LEDs are used for the same purpose as in the transmitter circuit. I’ve attached my circuit schematic below.
Breaking The Circuit Down:
The purpose of this circuit is to take the values sent by the transmitter and manipulate them into speeds/values for the motors to obey. For the stepper motors this is done using the AccelStepper library and stepper motor drivers. In my project I used the A4988 stepper motor drivers. However, these are very small and fragile drivers so if you plan on using the robotic arm for anything other than educational purposes, I highly recommend going with something a bit heavier duty to control the stepper motor. The purpose of the stepper drivers is to provide current control as well as communicate with the stepper motors without using a feedback system. This is how the stepper motor knows how much it needs to move. You can think of them as a motor encoder, but in reverse. They are also necessary because the control circuit for stepper motors require a much smaller voltage than the driving circuit. To hook them up effectively, all you need to do is connect the STEP and DIR pins to their respective GPIO pins as well as provide it with power and connect it to the stepper motor.
The STEP pin on the A4988 is generally where you will connect the number of steps you want the motor to do, while the DIR pin is where you will send an either high or low value to determine the direction you wish to move the motor. However, this is not the case in the AccelStepper library I used and likely some other libraries as well. So, make sure you interface all the pins correctly as corresponding to the library you will be coding it with. Also be sure that the sleep and reset pins are connected on the driver, it will not work if they aren’t. As far as powering everything goes, the VDD can come straight from the 3.3V output of the ESP32 and the VMOT will need to come from an external power source as rated for the stepper motor you are using. Just be sure it’s within the operating voltage of the driver (8-35V). Also be sure that everything has a common ground! It is an extremely common mistake when working with electronics that everything isn’t grounded correctly. If there is no common ground, there will be no functioning circuit. You will also need to adjust the on-board potentiometer on the driver to limit the current going to your motor. The NEMA 17s I used, have a max current rating of 2A, so I adjusted the potentiometer correspondingly. A good rule of thumb for this is Vref=Imax/2.5 where Vref is the voltage going through your multimeter when you place your multimeter’s positive to the potentiometer and negative to GND, and where Imax is the maximum allowed current for your stepper motor. However, keep in mind the max current for the drivers is also 2A, so it wouldn’t hurt to keep it a bit on the safe side
As for the servo motors, it is extremely simple. Most microcontrollers have excellent support for servos since all they need to do is move in a specific direction until the servo’s on-board potentiometer matches the value it is trying to move it to. Just be sure that they servos also have a common ground with the rest of the circuit and that it’s getting external power as per it’s rated voltage. For the MG996Rs this is 6V.
Important Soldering Tips:
When soldering the boards and making connections, especially for the receiver, I highly recommend soldering a connection between the board and the fragile components such as the ESP32 and A4988 stepper drivers. The drivers especially are extremely fragile and although they work decently, they aren’t super heavy duty and things will go wrong. I myself probably shorted out at least five of the stepper motor drivers while prototyping. This can be extremely frustrating, especially when the drivers are soldered into the board because they can be very hard to remove and ruin the circuit if you are unable to. I highly recommend doing something like below to allow easy part replacement if anything does happen to go wrong.
This is also very useful in ensuring the hot soldering iron doesn’t damage anything while soldering components in place. As you know soldering irons get very, very hot and pins are excellent conductors. Soldering components directly could cause these pins to heat up too much and break connections, destroying the fragile ICs.
Building The Power Bank:
The power bank I made for this project is pretty simple and there are many ways it can be made. I just used a 11.1V 5200mAh LiPo battery and connected it to three different outputs. Two of the outputs had a stepped-down voltage and one had a stepped-up voltage. It is important to adjust these voltages so that each component that requires a different voltage level is able to get what they need to perform properly. If a voltage is too high, it will break the component. If it is too low, the component will not be getting enough power to do its job properly.
Each of the regulators (reg.) and step-up modules output to a different part of the receiver board to supply power. In the case, I’ve connected all the servos to the 6V output, all the stepper motors to the 24V output and the ESP32 to the 5V output. Simple voltage reductions can be made with the LM78XX series of ICs, however, keep in mind they have a small max current that can be ran through it (normally 1.2A), so they won’t work for heavy purposes. It is for this reason I used a buck converter with a max current of 3A for the servo motors to use and a step-up module with an even higher max current for the stepper motors. This way I can be sure everything is getting as much power as they need and don’t have to worry about them burning out. Just be sure to have common ground within all the different outputs and the receiver board.
Here I will describe the basics of how the microcontrollers are programmed using the pseudocode below:
For The Transmitter:
Declare variables to store the data that is to be read by the MPU6050 and flex sensor.
Structure variables to be sent through ESP-NOW.
Check for errors in communication.
Check if MPU6050 is found.
Declare pin modes.
Filter bandwidth and set ranges for the gyroscope as per the MPU6050 library.
Write one pin of the button high.
Check if button is pressed.
Map values from the MPU6050 into what can be interpreted into degrees (0-180).
Map values from the flex sensor.
Assign those values to the data variables that are to be sent through ESP-NOW.
Check if that data has been received.
If it wasn’t, write data LED low.
For The Receiver:
Declare variables to store data for when values are sent by the transmitter.
Setup steppers as a library function from AccelStepper.
Declare data that was sent over by the transmitter. (Rotation, bend, etc.)
Map the raw data sent over into the correct value ranges from previously declared variables.
Declare outputs of pins and check for errors in communication.
Attach servos to pins.
Finish setup of library functions from AccelStepper.
Check for error, if there is an error write data LED low.
Check if switch is high/low from the transmitter to determine which joint to activate through if statements.
If the statement is true, assign mapped values from the gyroscope to values being sent to the drivers.
Check if the flex sensor is open or closed.
If in a closed state, assign values to the servo relative to the bend in the sensor.
Set speeds from previously assigned values to motors.
If you’d like to see the exact code I used in this project, it will be in my post titled “The CAD and Code”. Please also consider subscribing. It helps support my page and enables me to devote more time into writing articles like this. Not to mention the more subscribers means the more funding I have to make projects! (Remember, the more subscribers = the cooler projects!)
Assembling the Arm:
Here I will be going over the assembly of the robotic arm so you can build your very own! As you know, the robotic arm has three main axes of rotation each requiring a gearbox so I will be going over the assembly of those first since the arm wouldn’t work without them. The gearbox mainly consists of eight different parts. The NEMA17 mount, NEMA17 shaft casing, sun gear, planetary gear, ring gear, bottom carrier, top carrier and spacer. Do note however, that there are three planetary gears per layer of the gearbox. This design is able to be held together so stiffly do to a really neat kind of rivet nut that I’m able to make in the 3D printed plastic.
For this to work all you have to do is pop the M3 nut into the indent and screw a bolt through the desired part. This method works incredibly well and certainly wish I’d have thought of it sooner. Another way this can be done to make it even more fool-proof is by actually having the nut concealed in the plastic as shown below. This was used a few times in the project.
As far as ordering and assembling the other aspects of the gearbox, it’s really pretty simple. You would first put the shaft casing on the motor shaft and after that put the motor mount over it.
After that, simply stick the sun gear on the top of the motor shaft casing and stick a bolt through the hole provided once you’ve lined them up. This will remove all the axes of motion so the sun gear will move with the motor shaft. After that, you just put the planetary gears over the bottom carrier as shown below.
One that is done you put the sun gear through the center. There should be some room between them but that’s okay. This will make them run smoothly. You will also need to insert nuts into the small indents in the bottom carrier. This will keep them in one piece when you mount the top part on.
Then you put the ring gear over it.
Next, you will put the longer spacers between the holes in the ring gear and those in the motor mount. Be sure you’ve inserted the nuts in the bottom of the motor mount. I would only use four so you can use the other four for mounting the next piece. This will require at least M3x55 bolts.
I recommend using some sort of lubricant for the gears at this point. It’s very important in reducing the friction between the plastic parts. I used Vaseline but other things should work well too. Next, place the top carrier over so that the holes in the design go over the pillars sticking out. Once that is done, use some small M3 bolts to secure the top carrier to the bottom one. However, be sure no end of the bolts is sticking out. If they are they will catch on the holes and stop movement.
Once this is done repeat the steps starting with the bottom carrier for the other two layers and switch to the spacers that are smaller in length. Once the gearbox is completed, clamp in the bearings and screw in the nuts through the holes in the arm’s joints and line up the ending gear with the one indented in the joint. Please note the bearings I used in this project were the 6204RS bearings, 20x47x14mm
Once the gearbox and bearings have been mounted for both parts it should look something like this.
At this point we will begin mounting for the last axis of rotation, the yaw. The bearing used here is a type of lazy Susan, in which the top is mounted to the top of the bearing and the bottom mounted to the bottom of the bearing. This is done through the hole made in the base. You will have to rotate the base of the arm to move it to each position to get all four bolts in.
Please note that I ended up having to cut out some holes just so I could see where to slot in the bottom gearbox, but it isn’t completely necessary for the design which is why you will not see it in the STEP file.
Once that is in, you will need something to secure the gearbox in place as it is just dangling in space at the moment. This is done using the piece below which you will bolt in eventually when everything is aligned.
Next up is the gripper. These are the areas that are designed to fit the MG996R servo motors and can be mounted easily using four M3 bolts and nuts for each motor which can be fitted and secured using the holes made in the design. I’ve shown what that may look like here:
As far as the actual gripper assembly goes it is quite simple. The gripper is printed in multiple layers to allow swap out if needed. It is aligned like below.
Bolting everything together is pretty intuitive. You will just screw holes through the servo’s shaft attachment and into the holes provided in the design. The layers of the gripper can be mounted with ease using the holes provided on the side and some bolts. If you choose to use the magnet you will need to bolt that in as well, but it should just go straight through and into the base shaft on the servo. Assembling the two gripper pieces are no different. One will be mounted directly to the servo motor and the other free to rotate about an axis using a bolt. They are geared so they will move inverse to one another.
Congratulations! You now should have your completed build. I hope you enjoyed my project and wanted to remind you again all the CAD and code for any of my projects will be located in my post titled “The CAD and Code.” If you enjoyed the article, please consider subscribing. Support is the key to my success, so I really appreciate everyone who contributes. Make sure to follow along with me as I will have many other cool projects coming in the future. Next, I’m thinking about making a belted robot that can move anywhere in 2D space and be programmed to play games like chess as well as other tasks that require thinking. Let me know what you think.