By: Pei-Yu Tsai
Having a conversation with deaf and mute individuals can be extremely challenging for those who do not understand sign language. However, imagine a technological invention that can solve this problem— a sign language translator.
An 18-year-old American teenager, Liam Sweeney, has designed a program that can recognize American Sign Language, or ASL, and convert into English. This product aims to quickly and accurately translate both static and moving gestures.
When Liam began creating the program, he had just graduated from Centaurus High School in Lafayette, Colo. In his part-time working experience, he encountered an interaction with a hearing-impaired customer asking complex questions that made him realize how difficult it is to communicate as a deaf person. This inspired his thoughtful project to translate ASL to text. (Source from Liam’s program introduction and Regeneron International Science & Engineering Fair’s report)
American Sign Language (ASL) is the third most widely used language in the United States, following English and Spanish, with approximately five million people using it as their native language. It encompasses fundamental language features such as grammar, vocabulary, and emotional expression.
Liam initially intended to use AI deep learning for language translation in his program, but found that the process was too time-consuming. Therefore, he adopted a gesture tracking approach that compares data from pre-recorded examples and matches the closest sign language gestures to display corresponding English text.
However, currently the system’s results are not as expected. The translator is slow and can only recognize one sign at a time. So far, the machine can only translate 15 signs in total. But the overall direction of the research can hopefully apply to several fields, such as medicine, customer service, and general crisis management.
Still, the future for the translator system is bright. Liam’s current computer program can recognize signs at about a 98 percent accuracy. The system will continue to be refined by Liam and his friend who knows ASL.
Resources:
https://projectboard.world/isef/project/soft012-american-sign-language-translator
https://symposium.foragerone.com/2023-csef/presentations/52954
https://news.tcnj.edu/2023/05/01/computer-science-asl-project/