Tecnologías de reconocimiento de manos para fomentar la inclusión: interpretación automática del lenguaje de señas chilena

There are people who suffer from either hearing or speech impairments, causing undeniable communication challenges in their lives. These individuals communicate using sign language; however, this language is not widely known to a large portion of the human population who primarily use spoken language or writing. This work focuses on the use of hand recognition technologies to interpret sign language, specifically Chilean Sign Language, employing computer vision and machine learning (ML) techniques with various object recognition libraries. These libraries were used in conjunction with a set of 2D hand images, which were prepared in a CSV file. Mediapipe was also used to detect key points on the hands, achieving high accuracy in gesture detection.
For greater effectiveness, OpenCV, Roboflow, and Mediapipe Hands were used to detect characteristic points on the hands to identify the hand (left or right) and perform procedures on it. The specialized Mediapipe Hands module of the Mediapipe framework uses pre-trained convolutional neural network models to identify 21 key points on the hands, enabling real-time gesture and hand detection, which is necessary to detect a hand and work with it on still images or video. OpenCV was used to capture RGB color frames per second and render still images. Roboflow was used to train the dataset for Chilean Sign Language recognition.