Skin-Former: Mobile-Friendly Transformer for Skin Lesion Diagnosis

The outbreak of infectious skin diseases quickly becomes a significant public health concern due to their rapid spread. Therefore, early detection and diagnosis of skin infections through lesion analysis via publicly accessible AI-powered tools are crucial for effective treatment and management. Recent attempts at optimizing Vision Transformers (ViTs) for efficient applications on mobile devices have enabled them to perform complex computer vision tasks. While ViT and its variants have lower latency or more parameters than lightweight CNN models, they remain substantially more complicated and capable in terms of the representational capacity of visual information. Lower latency and memory footprint are crucial for deployment on resource-limited consumer devices like smartphones or tablets. In this study, we use efficient pre-Trained transformer models to accurately capture coarse-grained and fine-grained features from various skin lesions. We propose an efficient hybrid transformer architecture, 'Skin-Former' that is low latency and parameter efficient and can capture fine-grained and discriminative color and texture features of skin lesions. The evaluation results on three publicly available datasets reveal that the Skin-Former model achieves higher accuracy with lower computational cost than several State-of-The-Art (SoTA) models.