Indian Sign Language Real-Time Recognition System Using Yolov11 Aligned With A Keypoint Detection Approach

Abstract

This paper introduce a reliable Indian Sign Language (ISL) recognition framework which utilize the capabilities of  MediaPipe for hand pose estimation, YOLOv11 for data augmentation, and transfer learning for improved accuracy. The system addresses the challenges of variations in signing styles, lighting conditions, and background clutter. Here, we utilize a tailor-made ISL dataset and analyze the performance of our present work, demonstrating its effectiveness in recognizing a range of ISL signs. The combination of MediaPipe's efficient hand tracking, YOLOv11's augmentation capabilities, and transfer learning allows for a more accurate and adaptable ISL recognition system compared to existing methods. The model achieved notable performance metrics: precision of 97.75%, recall rate of 95.002%, F1 score of 96.358%, mean avg. Precision (mAP) of 97.635%, and mAP50-95 of 86.163%, underscoring its exceptional accuracy and sturdy capabilities.