INTELLIGENT TOUCHLESS SYSTEM BASED ON GESTURE RECOGNITION

In our rapidly advancing technological era, every industry is experiencing its revolution. As we navigate the challenges the current pandemic presents, there is a heightened interest in solutions that facilitate social distancing and contactless interactions. To address this challenge, we propose the development of an interactive and innovative platform that allows users to navigate through hand gestures. This touchless system can be customized to meet various needs and utilizes a set of standard hand gestures for simplicity and ease of use and can be implemented in multiple sectors such as airports, banking, retail, restaurants, and so on. To demonstrate the system's potential, we have created a mobile food ordering application that uses hand gestures as the primary means of interaction and uses a set of standard hand gestures to promote simplicity, familiarity, and user accessibility. This study will develop a mobile food ordering system to illustrate the proposed gesture-based touchless system. To build our gesture recognition model, we collected a dataset of common hand gestures by scraping images from the web. We then trained our models using the Efficient Net-Lite [0-4] algorithms, leveraging transfer learning and pre-trained deep learning models to reduce computational demands. We utilized transfer learning and pre-trained deep learning models to reduce the time and computational resources required for training. The trained models were evaluated using the mean average precision (mAP) and inference time and then converted into a lightweight format, TensorFlow Lite, for use on mobile devices such as kiosks for the mentioned scenario. Our evaluation results revealed that all the trained models achieved an mAP of 82% or higher, with the most complex model, EfficientNet-Lite4, reaching 87%. However, the inference time for the trained models was significantly longer, ranging from one to ten seconds. To balance performance and inference time, we chose the EfficientNet-Lite0 model with an inference time of just half a second for our hand gesture-based touchless system. This model provides an adequate level of accuracy for our hand gesture-based touchless system while minimizing any lag or delay that could impact user experience. In summary, our proposed system is a cutting-edge, user-friendly solution that meets the need for contactless interactions and social distancing. Using standardized hand gestures, we have created a platform that is intuitive and accessible for users. Our system has the potential to offer significant benefits across a wide range of industries and applications in the modern era.