Computer system for converting gestures to text and audio messages

Simple item page
dc.citation.epage97
dc.citation.issueVolume 6, № 2
dc.citation.journalTitleAdvances in Cyber-Physical Systems
dc.citation.spage90
dc.contributor.affiliationLviv Polytechnic National University
dc.contributor.affiliationMedBridge Inc
dc.contributor.authorBorovets, Dmytro
dc.contributor.authorPavych, Tetyana
dc.contributor.authorParamud , Yaroslav
dc.date.accessioned2022-12-01T09:00:12Z
dc.date.available2022-12-01T09:00:12Z
dc.date.issued2021
dc.date.submitted2022
dc.description.abstractToday, there are quite a large number of deafmute and hard-of-hearing people which communicate using gestures. Therefore, it is simply necessary to provide them with modern means of communication with the surrounding world. This paper creates a holistic computer system architecture for converting gestures into text and audio messages. The principles of construction and basic design solutions of a computer system based on a modern element base with increased productivity and minimization of hardware costs and energy consumption have been developed. The most popular existing solutions for gesture recognition are considered and analyzed. The operation of the main components has been described, the principle of functioning of the entire system has been analyzed, and their advantages and disadvantages have been compared. The latest structural components for building a computer system (both physical and software) have been selected and investigated. Physical features include: the state-of-the-art Arduino Nano computing platform, the HC-05 Bluetooth module, the ADXL335 accelerometer, and the latest ZD10-100 Information sensor (flexibility sensor). Software features include: firmware for the Arduino Nano hardware platform, Python-based software for splitting the flow of letters into words, displaying them, and voicing them. The methods of Google Media Translation API and Google Text-to-speech (gTTS) have been analyzed. The expediency of conducting research has improved performance through the use of a new information sensor, which is a flexibility sensor ZD10-100 500 g. The general structural scheme of all systems has been designed.
dc.format.pages90-97
dc.identifier.citationBorovets D. Computer system for converting gestures to text and audio messages / Dmytro Borovets, Tetyana Pavych, Yaroslav Paramud // Advances in Cyber-Physical Systems. – Lviv : Lviv Politechnic Publishing House, 2021. – Volume 6, № 2. – P. 90–97 . – Bibliography: 15 titles.
dc.identifier.doihttps://doi.org/10.23939/acps2021.02.090
dc.identifier.urihttps://ena.lpnu.ua/handle/ntb/57238
dc.language.isoen
dc.publisherLviv Politechnic Publishing House
dc.relation.ispartofAdvances in Cyber-Physical Systems
dc.relation.references[1] Gov.ua. (2017). World and Ukrainian statistics of the deaf and dumb, International Day of the Deaf. [online] Available at: https://www.kmu.gov.ua/news/250293661 (Accessed: 5 October 2021) [2] Uapatents.com. (2014). Description Enable talk system. [online] Available at: https://uapatents.com/9-86748-sistema enable-talk-inejjbl-tolk.html (Accessed: 6 October 2021) [3] Anthrotronix.com. (2018). Description NuGlove system, Interact system, Droid Agent system. [online] Available at: https://www.anthrotronix.com/ (Accessed: 6 October 2021) [4] Signall.us. (2021). SignALL translate program. [online] Avai lable at: https://www.signall.us/ (Accessed: 6 October 2021) [5] D. Mheill. “Gesture and Thought”, University of Chicago Press, Degruyter, 2005. DOI: 10.7208/9780226514642. [6] Ap.uu.edu.ua. (2004). The bilingual method is an indispensable condition for the learning of the deaf. [online] Available at: http://ap.uu.edu.ua/article/404 (Accessed: 5 October 2021) [7] Y. Badamasi. “The working principle of an Arduino”, ieeexplore, 2014. DOI: 10.1109/ICECCO.2014.6997578. [8] Analog.com. (2010). Specification analogue accelerometer ADXL335. [online] Available at: https://www. analog.com/ ru/products/adxl335.html#product-overview (Accessed: 7 October 2021) [9] V. Sing, A. Shahy, A. Beg, B. Chay, S. Kymar. “The Specification Bluetooth HC-05 module”, ieeexplore, 2019 . DOI: 10.1109/ICACAT.2018.8933698. [10] Desertcart.com.py. (2021). Thin film flex and pressure sensor ZD10-100 500g [online] Available at: https://www.desertcart.com.py/products/107132867-thin-film pressure-sensor-flex-bend-sensor-zd-10-100-500-g-resistance type-fsr-sensor-thin-film-pressure-sensor-force-sensing-resistor force-sensitive-resistor (Accessed: 7 October 2021) [11] J. Crammer. “Python -The Fastest Growing Programming Language”, International Research Journal of Engineering and Technology (IRJET), vol. 4, iss. 12, 2017. DOI: 2395- 0072/pan.2017.12. [12] Pypi.org. (2021). Word segmentation module v1.3.1. [online] Available at: https://pypi.org/project/wordsegment/ (Accessed: 8 October 2021) [13] E. Vries, M. Schoonvelde, G. Schumacher. “Google Translate Works” Political Analysis, vol. 26, iss. 4, 2018, pp. 417–430. DOI: 10.1017/pan.2018.26. [14] Pypi.org. (2021). Google Text-to-speech module v2.2.3. [online] Available at: https://pypi.org/project/gTTS/ (Accessed: 8 October 2021) [15] Cloud.google.com (2021). Google Media Translation API. [online] Available at: https://cloud.google.com/media-translation (Accessed: 8 October 2021).
dc.subjectcomputer system, gesture, computing plat form, module, serial port, firmware, sensor, thresholding
dc.titleComputer system for converting gestures to text and audio messages
dc.typeArticle

Files

Original bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
010-017.pdf
Size:
527.08 KB
Format:
Adobe Portable Document Format
Download

Collections

Advances In Cyber-Physical Systems. – 2021. – Vol. 6, No. 2