Two to three of every 1,000 children are born deaf or hard of hearing in the US. Though 90 percent of these kids are born to hearing parents, 72 percent of families do not sign with their deaf children.
After learning about those statistics, creative studio Hello Monday decided to create a solution that would encourage more parents of deaf children to learn sign language. The studio designed an app that teaches and corrects hand positions in real-time to facilitate people’s efforts in learning the sign language alphabet.
Designed in partnership with the American Society for Deaf Children, Fingerspelling.xyz is an app that runs in web browsers and is able to assess its user’s attempts to learn words in real-time — much like the popular language-learning app Duolingo.
Making use of a person’s webcam, the app tracks their hand movements as they attempt the hand positions of the American Sign Language (ASL) Alphabet. An algorithm trained on images of correct hand positions then analyses the accuracy of the user’s attempts, giving them feedback until they achieve the correct alignment. In ASL, fingerspelling is used for proper nouns, or when a person doesn’t know the sign for a word.
“The fingerspelling game is a great way to introduce the basics of ASL in a fun and playful way,” says Anders Jessen, Hello Monday’s founding partner. According to Jessen, the app is designed as such as to offer a more engaging learning experience than the traditional methods of learning the sign language alphabet — through reading or watching videos.
“The game leverages advanced hand recognition technology, matched with machine learning, to give you real-time feedback via the webcam for each sign and word you spell correctly,” says Jessen. “It’s fun to think about how this highly trained machine-learning model, which researchers have spent countless hours on training, now does the opposite — it now trains us back, and makes us better at fingerspelling.”
Hello Monday’s new app is mainly targeted at parents of deaf children, enabling them to encourage their child’s development by introducing them to sign language at an early stage.
Image source: Fingerspelling.xyz
