Using a TensorFlow-Based Neural Network for Gesture Recognition and Control of a Bionic Prosthesis
DOI:
https://doi.org/10.31649/1997-9266-2024-174-3-71-77Keywords:
neural network, TensorFlow, gesture recognition, bionic prosthesis, electromyography, recognition accuracyAbstract
This study analyzes in detail the recognition accuracy of various gestures using a neural network built on TensorFlow to control a bionic prosthesis. By conducting a series of experiments using data sets and model parameters, it is investigated how effectively the neural network recognizes various gestures in the context of prosthesis control. The results of the experiments will allow us to understand the potential limitations and opportunities of this technology for practical application in real conditions. The neural network model and architecture, activation functions and parameters used for training are described. Learning parameters that affect the effectiveness of model learning are also considered. To evaluate the effectiveness of the model, results such as model evaluation metrics, model performance graphs, and confusion matrix are used to evaluate the level of reliability and performance of the trained neural network. Method of collecting electromyography (EMG) data is proposed, which consists in the use of electrical signals arising in muscles during their contraction. This process involves placing electrodes on the surface of the skin over the muscles being analyzed to record the electrical signals that occur at the moment of muscle activity. This approach allows collecting objective data on muscle activity and their movements, which can then be used to train a neural network and further use it in controlling a bionic prosthesis. A series of experiments was conducted to assess the accuracy of recognition of various movements using the trained model. Analyzing the results of the experiments, it is possible to understand how efficiently and reliably the trained model works in real conditions and how suitable it is for practical use in control systems of bionic prostheses.
References
A. M. Elbreki, S. Ramdan, F. Mohamed, K. Alshari, Z. Rajab, and B. Elhub, “Practical Design of an Upper Prosthetic Limb Using Three Dimensional Printer with an Artificial Intelligence Based Controller,” in Proceedings of the 2022 International Conference on Engineering & MIS (ICEMIS), Istanbul, Turkey, Jul. 4-6, 2022, pp. 1-6.
O. Kerdjidj, K. Amara, F. Harizi, and H. Boumridja, “Implementing Hand Gesture Recognition using EMG on the Zynq Circuit,” IEEE Sensors Journal, vol. 9, pp. 10054-10061, 2023. https://doi.org/10.1109/JSEN.2023.3259150 .
N. M. Hye, U. Hany, S. Chakravarty, L. Akter, and I. Ahmed, “Artificial Intelligence for sEMG-based Muscular Movement Recognition for Hand Prosthesis," IEEE Access, vol. 11, pp. 38850-38863, 2023. https://doi.org/ 10.1109/ACCESS.2023.3267674 .
L. C. Zayia, and P. Tadi, “Neuroanatomy, motor neuron,” StatPearls, StatPearls Publishing, Tampa, FL, USA, 2021.
L. Mc Manus, G. De Vito, and M.M. Lowery, “Analysis and biophysics of surface EMG for physiotherapists and kinesiologists: Toward a common language with rehabilitation engineers,” Frontiers in Neurology, vol. 11, p. 576729, 2020. https://doi.org/10.3389/fneur.2020.576729 .
D. Mahmood, H. N. Riaz, and H. Nisar, “Advances in Non-Invasive Biomedical Signal Sensing and Processing with Machine Learning: Introduction to Non-Invasive Biomedical Signals for Healthcare,” Springer International Publishing, Cham, Switzerland, 2023.
V. Asanza, E. Peláez, F. Loayza, L. L. Lorente-Leyva, and D. H. Peluffo-Ordóñez, “Identification of Lower-Limb Motor Tasks via Brain–Computer Interfaces: A Topical Overview,” Sensors, vol. 22, p. 2028, 2022. https://doi.org/ 10.3390/s22052028 .
I. Daimiel Naranjo et al., “Radiomics and Machine Learning with Multiparametric Breast MRI for Improved Diagnostic Accuracy in Breast Cancer Diagnosis,” Diagnostics, vol. 11, p. 919, 2021. https://doi.org/ 10.3390/diagnostics11060919 .
M. A. Gulum, C. M. Trombley, and M. Kantardzic, “A Review of Explainable Deep Learning Cancer Detection Models in Medical Imaging,” Applied Sciences, vol. 11, p. 4573, 2021. https://doi.org/10.3390/app11104573 .
M. Zivkovic, et al. “A Hybrid CNN and XGBoost Model Tuned by Modified Arithmetic Optimization Algorithm for COVID-19 Early Diagnostics from X-ray Images,” Electronics, vol. 11, p. 3798, 2022. https://doi.org/ 10.3390/electronics11223798
A. Toro-Ossaba, J. Jaramillo-Tigreros, J. C. Tejada, A. Peña, A. López-González, and R.A. Castanho, “LSTM Recurrent Neural Network for Hand Gesture Recognition Using EMG Signals,” Applied Sciences, vol. 12, p. 9700, 2022. https://doi.org/10.3390/app12199700 .
Á. L. Valdivieso Caraguay, J. P. Vásconez, L. I. Barona López, and M. E. Benalcázar, “Recognition of Hand Gestures Based on EMG Signals with Deep and Double-Deep Q-Networks,” Sensors, vol. 23, p. 3905, 2023. https://doi.org/10.3390/s23083905 .
Y. Xu, P. C. Barbosa, J. S. da Cunha Neto, L. Zhang, V. H. C. de Albuquerque, V. Shanmuganathan, and S. Pasupathi, “Development of intelligent and integrated technology for pattern recognition in EMG signals for robotic prosthesis command,” Expert Systems, vol. 40, p. e13109, 2023. https://doi.org/10.1111/exsy.13109 .
K. Dokic, M. Martinovic, and D. Mandusic, “Inference speed and quantisation of neural networks with TensorFlow Lite for Microcontrollers framework,” in Proceedings of the 2020 5th South-East Europe Design Automation, Computer Engineering, Computer Networks and Social Media Conference (SEEDA-CECNSM), Corfu, Greece, Sep. 25-27, 2020, pp. 1-5.
Y. Fang, J. Yang, D. Zhou, and Z. Ju, “Modelling EMG driven wrist movements using a bio-inspired neural network,” Neurocomputing, vol. 470, pp. 89-98, 2022. https://doi.org/10.1016/j.neucom.2021.10.104 .
A. K. Mukhopadhyay, S. Samui, “An experimental study on upper limb position invariant EMG signal classification based on deep neural network,” Biomed. Signal Process. Control. 2020;55:101669. https://doi.org/10.1016/j.bspc.2019.101669 .
D. Buongiorno, et al., “Deep learning for processing electromyographic signals: A taxonomy-based survey,” Neurocomputing, vol. 452, pp. 549-565, 2021. https://doi.org/10.1016/j.neucom.2020.06.139 .
V. Asanza, et al., “SSVEP-EEG Signal Classification based on Emotiv EPOC BCI and Raspberry Pi,” IFAC-PapersOnLine, vol. 54, pp. 388-393, 2021. https://doi.org/10.1016/j.ifacol.2021.10.287 .
A. Constantine, V. Asanza, F. L. Loayza, E. Peláez, and D. Peluffo-Ordóñez, “BCI System using a Novel Processing Technique Based on Electrodes Selection for Hand Prosthesis Control,” IFAC-PapersOnLine, vol. 54, pp. 364-369, 2021. https://doi.org/10.1016/j.ifacol.2021.10.283.
D. Tinoco-Varela, J. A. Ferrer-Varela, R. D. Cruz-Morales, and E. A. Padilla-García, “Design and Implementation of a Prosthesis System Controlled by Electromyographic Signals Means, Characterized with Artificial Neural Networks,” Micromachines, vol. 13, p. 1681, 2022. https://doi.org/10.3390/mi13101681 .
O. K. Kolesnytskyj, I. V. Bokotsey, and S. S. Yaremchuk, “Optoelectronic implementation of pulsed neurons and neural networks using bispin-devices,” Opt. Mem. Neural Networks, no. 19, pp. 154-165, 2010. https://doi.org/10.3103/S1060992X10020062 .
Downloads
-
PDF (Українська)
Downloads: 59
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
