Indian Sign Language Recoginition With Predictive Correction Using Deep Learning









Abstract

The world is hard to live without communication, no matter whether it is in the form of text, voice or visual expression. Sign language can be of two types: Isolated sign language (single gesture) and Continuous sign language (double gesture). So, here we proposed a system using continuous sign language which generates a meaningful sequence. The continuous signs in a video are subjected to key frame extraction, feature extraction and classification with 2D-CNN and correction of the symbol is carried out with the help of ANN which results in the use of hybrid neural network to achieve our goal. Indian sign language uses two-handed gestures unlike ASL which makes it a little tedious to predict it. Here, we have used datasets with signs of alphabets for training (static gestures) with over 500 2D images and testing (dynamic gestures) with over 26 videos. The training with CNN achieved accuracy of 98% with our model. ANN has been trained for 20 words and tested with continuous sign language. The training of our ANN has achieved up to 90% accuracy. The text is then converted to speech for a better verbal communication with options in three Indian languages Keywords: HNN, 2D-CNN, ANN, ISL, key frame extraction, feature extraction, classification


Modules


Algorithms


Software And Hardware

• Hardware: Processor: i3 ,i5 RAM: 4GB Hard disk: 16 GB • Software: operating System : Windws2000/XP/7/8/10 Anaconda,jupyter,spyder,flask Frontend :-python Backend:- MYSQL