Internet-of-things applications with hand motion for remote control : a case study on Home Automation and Robotic arm
Nguyen, Minh (2020)
Nguyen, Minh
2020
All rights reserved. This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:amk-2020052012546
https://urn.fi/URN:NBN:fi:amk-2020052012546
Tiivistelmä
This thesis aimed to introduce two possible implementations with Myo armband, which contains surface electromyography (sEMG) electrodes and a nine-axis Inertial Measurement Unit (IMU). sEMG is the reading of electrical signals generated in forearm muscles contractions, while IMU measures the angles of movement, thus when used together, they can predict the gesture. The proposed projects, namely, Home Assistant control and MeArm robot control, aimed to provide reusable and simple applications for future research about Myo gesture-recognition in Assistive technology. With regards to Myo’s feature extraction, pre-built Python libraries were used for handling the Bluetooth communication with Myo band and performing handler on the received raw data including sEMG, poses, and IMU readings. By utilizing the MQTT (Message Queue Telemetry Transport) protocol, the communication between Myo and external microcontrollers, including an ESP-12E (also known as Node MCU) and an Arduino UNO, can be established.
The purpose of the Home Assistant project was to develop a sandbox application to control smart devices. In the Home Assistant implementation, a desk lamp was calibrated and controlled by Node MCU. After setting up the MQTT broker server on Intel gateway, it was possible to subscribe and publish messages to control the remote ESP. Hence, the result was promising, which has proved that this application could be scaled up and control other smart devices such as Phillips Hue light bulb, smart air-conditioner, or smart TV.
With a similar technique, the implementation with the MeArm robot was successful in demonstrating a simple gesture controlled robotic application. The combination of pose and arm movements were used to compose the set of command protocol with seven gestures. However, there were drawbacks to the current design. Firstly, connectivity with Myo Bluetooth dongle can sometimes be unstable. Secondly, Myo built-in gesture-recognition does not always make correct predictions. Lastly, only two readings from Myo’s IMU (inertial measurement unit) were used in the implementation. Nevertheless, performing data processing and pattern recognition for different pose categorization would be possible for further development in the field of machine learning.
The purpose of the Home Assistant project was to develop a sandbox application to control smart devices. In the Home Assistant implementation, a desk lamp was calibrated and controlled by Node MCU. After setting up the MQTT broker server on Intel gateway, it was possible to subscribe and publish messages to control the remote ESP. Hence, the result was promising, which has proved that this application could be scaled up and control other smart devices such as Phillips Hue light bulb, smart air-conditioner, or smart TV.
With a similar technique, the implementation with the MeArm robot was successful in demonstrating a simple gesture controlled robotic application. The combination of pose and arm movements were used to compose the set of command protocol with seven gestures. However, there were drawbacks to the current design. Firstly, connectivity with Myo Bluetooth dongle can sometimes be unstable. Secondly, Myo built-in gesture-recognition does not always make correct predictions. Lastly, only two readings from Myo’s IMU (inertial measurement unit) were used in the implementation. Nevertheless, performing data processing and pattern recognition for different pose categorization would be possible for further development in the field of machine learning.