Millions of sign-language users can’t communicate with your organization

Publish date:

Many people don’t understand why a person who uses sign language might need a machine to translate their sign language into text or speech. Can’t they just write down whatever they want to say?

Many sign language users do not know any spoken languages, especially if they have been deaf since birth – they can’t speak them and they can’t write them. In many cases, giving a sign-language user a piece of paper and a pen is about as helpful as asking me to communicate using Chinese pictograms.

It’s a common mistake to assume that a sign language is just a way of communicating in French or English or some other language using hand gestures instead of sounds or written characters. In fact, sign languages are entirely different to spoken languages. A person able to communicate in French Sign Language, for example, does not automatically also know how to communicate in spoken or written French.

The sound of one hand clapping

There are no reliable statistics for the number of sign-language users globally, but extrapolating from the number of people born deaf suggests hundreds of millions of people use a sign language as their first and often only language.

Organizations can overcome spoken language barriers by providing written translations or automated text translation, but there is no such shortcut for sign languages. Right now, organizations that routinely provide multi-language services have no way of providing the same convenience for sign-language users.

We are not the first to notice this deficiency. Many attempts have been made over the years to automatically translate sign languages into spoken languages using gloves that sense sign-language users’ finger movements.

The problem with this approach is that it only works with a specialized part of sign languages called manual alphabets. Manual alphabets are used to spell out words from spoken languages in much that same way that anyone who knows the Latin alphabet can spell out a word in any language that uses that alphabet, even if they don’t speak it.

Just because a sign language user can spell a word, doesn’t mean he or she understands it, in the same way that an English speaker can spell out a word in Spanish without knowing what it means.

The bigger picture

Sign languages commonly include hand, arm, head, shoulder, torso, and facial movements as part of their articulation. A sign language translation system that only looks at hand movements would be like a spoken language translation that only translates nouns – you might get a broad sense of the communication, but you would miss a lot of the meaning.

This is why visual recognition systems are now being used in sign language interpretation. Properly trained AI-based software can identify the hand and body movements of a sign-language and translate them into a spoken language. We have built a solution that does exactly this, enabling more people to communicate seamlessly in retail businesses, schools, hospitals  and a wide range of other public places.

The solution consists of in-store kiosks equipped with video cameras and screens. The recognition platform, based on TensorFlow and PoseNet technologies, interprets the user’s signs to text or audio for a remote or in-store customer service agent. Their reply can be in signs, text, or audio.

Sign up

It’s important to note that translating from a sign language to a spoken language is just as difficult as translating from one spoken language to another, if not more so. We cannot claim to have built a solution that will allow sign-language users to communicate flawlessly with non-sign-language users, but we have built a solution that makes a good level of communication possible in a situation where otherwise it is often impossible.

Any business that recognizes the competitive importance of giving their customers the best experience possible should also recognize that this applies to all their customers, not just the ones who know spoken languages. If your business is committed to inclusivity, get in touch with me and we can talk about building a hassle-free experience for every one of your customers.

Related Posts

Automation – a way to thrive in uncertain times

Ritesh Mehta
Date icon October 20, 2020

How you see automation – and knowing exactly what to automate – can make all the difference...

Why leaders are no longer needed in an agile context – truth or myth?

Susanne Katharina Ruff
Date icon October 16, 2020

“Agile” is THE buzzword of recent years. More and more organizations are trying to become...

Sustainability

Training AIs doesn’t have to hurt (the planet)

Gunnar Menzel
Date icon October 15, 2020

Bedazzled as we are by the myriad new advances it enables – from speech recognition to...