Azerbaijani sign language recognition using machine learning approach

Samir Aliyev, Ali Abd Almisreb, Sherzod Turaev

Research output: Contribution to journalConference articlepeer-review

3 Citations (Scopus)

Abstract

Sign Language recognition is one of the essential and focal areas for researchers in terms of improving the integration of speech and hearing-impaired people into common society. The main idea is to detect the hand gestures of impaired people and convert them to understandable formats, such as text by leveraging advanced approaches. In this paper, we present our contribution to the improvement of Azerbaijani Sign Language (AzSL). We worked on AzSL Alphabet static signs real-time recognition. The method applied in this work is Object Classification and Recognition by leveraging pre-trained lightweight Convolutional Neural Networks models. At first, a dataset containing near to 1000 images has been collected, then interesting objects on images have been labeled with bounding boxing option. To build, train, evaluate and deploy the relevant model, TensorFlow Object Detection API with Python has been employed. MobileNet v2 pre-trained model has been leveraged for this task. In the trial experiment with four sign classes (A, B, C, E) and 5000 step numbers 15.2% training loss and 83% evaluation mean average precision results have been obtained. In the next step of model deployment experiments with all 24 static signs of AzSL, 49700 and 27700 steps (180 and 100 epochs, respectively) 6.4% and 18.2% training losses, 66.5% and 71.6% mAP outcomes gained, respectively.

Original languageEnglish
Article number012007
JournalJournal of Physics: Conference Series
Volume2251
Issue number1
DOIs
Publication statusPublished - May 17 2022
Event2nd International Conference on Robotics and Artificial Intelligence, RoAI 2021 - Virtual, Online
Duration: Nov 29 2021Nov 30 2021

Keywords

  • Azerbaijan Sign Language
  • Convolutional Neural Networks
  • MobileNet
  • TensorFlow

ASJC Scopus subject areas

  • Physics and Astronomy(all)

Fingerprint

Dive into the research topics of 'Azerbaijani sign language recognition using machine learning approach'. Together they form a unique fingerprint.

Cite this