Environment-Resilient Yoga Pose Estimation Using Stacked Hourglass Networks With Adaptive Hybrid Heatmaps

Abstract

This work proposes real time fixed yoga pose estimation and correction system. The pose estimation approaches, including MoveNet and OpenPose, are likely to suffer from low-quality keypoint localization when there are complex body poses or occlusions, and they trade off accuracy for real-time performance. In surpassing these drawbacks, this system improves keypoint prediction by incorporating anatomical priors into the heatmap generation process. The pipeline combines a lightweight MoveNet model for real-time inference with a stacked hourglass network trained offline on a yoga pose dataset. During training, hybrid heatmaps are generated using Kernel Density Estimation (KDE) along with PCA-aligned geometric masks (ellipse and stadium) to create more anatomically meaningful supervision signals. At inference time, MoveNet keypoints are transformed into hybrid heatmaps and compared against predictions using an Adaptive Squared Mean Loss function that emphasizes precision at keypoint peaks. Quantitative performance on the MPII dataset confirms that the proposed method obtains a PCK@0.5 of 89.3%, mAP of 74.2%, and NME of 0.101 — better than MoveNet (PCK 84.1%), HRNet-W32 (PCK 88.5%), and OpenPose (PCK 83.9%). The system also operates at 22 FPS, leading to real-time performance with improved accuracy. Ablation experiments confirm the efficacy of hybrid heatmaps and mask geometries since the proposed loss function learns 20 epochs ahead of Wing Loss and 30 ahead of MSE. The method demonstrates great promise for yoga training and fitness feedback application scenarios, where speed and accuracy are of equal worth