Gesture recognition for a safer, more inclusive society

AI combined with innovative devices provides the ability to interact with computers and devices without physically touching them. Researchers worldwide innovate in areas as diverse as hand wash monitoring, translation of hand gestures to speech, and wearable biosensors. In this article, our experts Rajashree Das, Chief Architect, Insights & Data, Capgemini India and Rekha Chandrasekar, Enterprise Architect, Insights & Data, Capgemini India highlight how gesture recognition can create a safer place, even when the pandemic is over, and enable more people to interact with computers, access information, and communicate with others.

Gestures are the most natural form of human communication. Gesture recognition is being used today for Human-Computer Interaction (HCI), a means of human-machine interaction using only physical actions without voice. The current focus is primarily on emotion recognition from face to hand gesture recognition, but it can include complete body gestures in the future.

Gesture Recognition Technology

Gesture control, or gesture recognition technology (GRT), is a computer science topic and language technology. It primarily falls under the subdiscipline of computer vision in the AI world and involves the complex integration of various components.

Unique interfaces that can capture gesture movements like cameras, use computer vision technology and deep learning algorithms to understand the underlying pattern, interpret sign language, and deploy it to the cloud for enhanced scalability and computing power.

The concept of recognizing gestures using hands and/or other body parts is based on three layers: Detection, tracking, and recognition.

Apart from the hand gesture example given above, the identification and recognition of posture, gait,

proxemics, and human behavior is the advanced subject of gesture recognition techniques. There are challenges in designing GRT:

• Standardization of Gesture Language – The need to bring in consistency concerning gestures globally to mean the same thing across geographies.
• Domain Data – Creating gesture recognition systems for each domain needs much data to train ML models.
• Accuracy of GRT Systems – Factors like insufficient background light, high background noise, or something else may lead to inaccurate reading/recognition of motions.
• Diverse and Complex Implementations – Capturing, processing, training models, and recognizing gestures requires advanced machine learning algorithms and intensive computing. Further, various device hardware and sensing mechanisms support different kinds of recognizable gestures.
• Society Acceptance – User willingness to perform gestures/ use various sign languages keeping in mind social context, acceptability, and the potential implications. We are moving from the world of interaction with people to the world of interactions with systems.

How is this technology benefiting Industry?

According to ‘Markets and markets’, the market for gesture recognition is expected to grow from USD 9.6 Billion in 2020 to USD 32.3 Billion in just five years. The integration of multiple technologies with gesture control is creating innovation opportunities in many sectors, including:

• Consumer Electronics – GRT is a natural evolution from touchscreens for the consumer electronics market. Home automation is a big field in which gesture recognition is being employed. For instance, smart TVs can sense finger movements and hand gestures, offer touchless control over lighting and audio systems.
• Automotive – Car infotainment systems leverage gestures to control music like changing radio channels and incoming calls, among many other things. Less interaction with the touchscreen makes the driving experience safer and more convenient.
• Healthcare – Hand-gesture recognition system enables doctors to manipulate digital images during medical procedures using hand gestures instead of touch screens or computer keyboards.
• Entertainment/Gaming – Augmented reality (AR) and virtual reality (VR) technologies in the gaming and entertainment industries have accelerated the adoption of gesture recognition products and solutions.
• Banking – Banks can use gesture recognition for enhanced security to allow only trusted employees/ customers to access secured areas and avoid robberies. KYC has become a challenge for banks and financial institutions during the pandemic. An e-KYC solution can ensure a frictionless customer experience with a combination of facial recognition and live emotion/ gesture capture.

STEP 1:

A new picture is taken of the ‘live’ person and matched against the one taken from the government-issued identification document. This is done to ensure that an actual person is undergoing the verification process. A total of 106 facial markers (or landmarks) are used to ensure that the captured image of the person matches the picture in the official document.

STEP 2:

There’s also an option for live motion, which will require the application to blink, smile, nod their head, and more to determine that there is a real person present at the time of verification Machine Learning algorithms are used to enhance the accuracy of this process. There are many more scenarios where you can use GRT, such as in kiosks in the airline industry, contactless store experiences, virtual in-store displays, vending machines, and other similar environments.

Key Technology/Market Players accelerating adoption

Today’s top researchers and producers of gesture interface products are:

• Intel – RealSense Touchless Control Software (TCS) seamlessly converts a touch interaction to a touchless one with an Intel RealSense Depth Camera D435.
• Microsoft – has a project to explore camera-based gesture recognition within surgical settings. This would allow a surgeon to view and manipulate a patient’s x-rays or lab reports without having to “break out scrubs.”
• Google – was awarded a patent for gesture recognition systems covering how online shoppers might use augmented reality to examine merchandise remotely.

Apple is increasingly active in the area as well, but so are specialized players such as Tobii Rex (infrared light-based eye-tracking device for disabled people, using it to point and interact with a computer). Another example is Hitachi’s finger vein technology, which authenticates a person when they display a hand biometrically.

Handwashing

The need of the hour may be found in “Handwash movement recognition Technology” to promote handwashing etiquette. Due to the pandemic, the general significance of handwashing as a measure to protect health has

become more evident. There is an urgent need to implement measures to ensure hygiene in food safety standards and ensure it is non-invasive. You can extend this technology to medical facilities, hospitals, schools, hotels, the foodservice industry, and event venues. Automated on-site handwashing recognition eliminates the need for intrusive, time-consuming visual confirmation and manual recording at worksites.

#gestures #gesturetechnology #computervision #gesturerecognition #ai4good

INNOVATION TAKEAWAYS

1. Power to the People: A new AI-driven ability to interact with devices and computers without physically touching them, achieved with simple gestures.
2. Fast Adoption: Industries adopt gesture technology to provide for new contact-less business models to assure safety to their consumers, especially during the pandemic.
3. Societal Impact: Gesture technology comes as a boon to society, providing contact-less, safe, and inclusive experiences. Still, the social and emotional impact of interacting through technology does need to be further explored.

Interesting read?

Data-powered Innovation Review | Wave 2 features 21 such powerful stories from our leading technology partners and global top experts, covering fields like data for a better society, autonomous systems, data mesh architecture, creative AI, and data sharing ecosystems which will inspire you and activate your innovation muscles. Download your copy here!

Rajashree Das is a Senior IT Architect and Leader with 2 decades of rich and diverse experience in Information Technology, Business, Consulting and Leadership. Expertise in Architecting Large Complex Software Systems, Software Products, Solution Conceptualization, Digital Transformation Assessments, and End to End Solution Delivery to conform adherence to Business Outcome. She also drives Thought leadership, Innovation, Architecture Governance Boards, Evaluate Emerging Technologies, Technology & Tool Standardization.

Rekha  Chandrashekar has two decades of experience in Enterprise architecture, Strategy, Solutioning, Digital Transformation, Emerging technologies, Data / DWH Architecture, Big data analytics, BI, Presales, Consulting & Advisory.