Design and Implementation of a Multi-Language Hand Gesture Typing System in Unity
Ruuskanen, Juho (2024)
2024
All rights reserved. This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:amk-2024052315001
https://urn.fi/URN:NBN:fi:amk-2024052315001
Tiivistelmä
This thesis introduces and demonstrates a new approach to text input methods, utilizing hand gestures within the Unity engine. The thesis project aimed to develop a multilingual hand gesture typing system that enables text input in augmented and virtual reality environments.
For optimal user-computer interaction, this system in this project uses the Unity engine and integrates Ultraleap's pose detection algorithms with the Leap Motion Controller. In addition, the system uses 28 hand gestures mapped to either an alphanumeric character or a text editing command, providing users with a functional text input system.
The project is constructed in Unity by incorporating the hand gesture scripts into the pose detection algorithm and the pose event handler script, which handles the letter input to the text field. As a result, the pose detection algorithm binds these scripts together, forming the core components of this software design.
This thesis demonstrates the potential of the hand gesture typing system through careful research and development. The thesis' main findings include the successful integration of the Leap Motion Controller with Unity, the possibility to edit character sets to suit multiple languages and the identification of areas for further research and improvement.
This thesis offers insights into the potential of gesture-based interaction techniques to enhance user experiences and extend the capabilities of virtual and augmented reality environments. Through the development of gesture-based interaction and user interface design, this thesis presents new opportunities for research and innovation in the human-computer interaction field.
For optimal user-computer interaction, this system in this project uses the Unity engine and integrates Ultraleap's pose detection algorithms with the Leap Motion Controller. In addition, the system uses 28 hand gestures mapped to either an alphanumeric character or a text editing command, providing users with a functional text input system.
The project is constructed in Unity by incorporating the hand gesture scripts into the pose detection algorithm and the pose event handler script, which handles the letter input to the text field. As a result, the pose detection algorithm binds these scripts together, forming the core components of this software design.
This thesis demonstrates the potential of the hand gesture typing system through careful research and development. The thesis' main findings include the successful integration of the Leap Motion Controller with Unity, the possibility to edit character sets to suit multiple languages and the identification of areas for further research and improvement.
This thesis offers insights into the potential of gesture-based interaction techniques to enhance user experiences and extend the capabilities of virtual and augmented reality environments. Through the development of gesture-based interaction and user interface design, this thesis presents new opportunities for research and innovation in the human-computer interaction field.