Project information
- Category: NLTK
- Role: Collaborator
- Project date: August 2021- May 2022
- Project URL: Github Repository
English to Indian Sign Language
A WHO report says around 60M people in India suffer from either complete or partial deafness, and of these, at least 5M are children.
There are very few certified sign language interpreters available and therefore, there is a need for a system that establishes a link between spoken and sign languages.
The model created has 3 parts. First, the english sentence is taken as input in the form of text or speech (using the Google text to speech API).
The sentence then undergoes pre-processing.
The primary language processing model includes tokenization, part of speech tagging, word eliminator and lemmatization.
In module 2, using the ISL grammar rules, the processed English sentence is converted to an ISL sentence. ISL essentially follows a Subject-Object-Verb agreement unlike English which follows a Subject-Verb-Object.
Module 3 includes the following steps:
1. Extraction of word’s Hamburg Notation System (HamNoSys) from the database.
2. Generation of Signing Gesture Markup Language (SiGML) files from its HamNoSys.
3. These SiGML files are accessed by the animation server for generating animation frames. Further, these frames are used to define the pose of an avatar.
4. Finally, the ISL sign is played in the form of 3D avatar animation.