A Survey on Person Identification under Occlusion Using Face and Ear Modalities

Abstract

Recognition of persons based on facial features has acquired considerable advancements in highly controllable conditions. But the real world can certainly include occlusions in the form of sunglasses, face masks and pose variations and this can seriously impair the performance of a traditional face recognition system. The current survey looks into the viability and usefulness of adapting the ear as an added modality to achieve viable person identification in such a free and occluded environment. The introduction to state-of-the-art reviews of face recognition in the presence of occlusion in the first part of the survey is organized into four sections: a survey of benchmark datasets, types of occlusion, methodology of recognition methods, and results of reported performance. The second section discusses techniques that aim at remedying the lack on facial information like data augmentation and frontal face reconstruction and learning in occluded or taking into account occlusions. The considerations of these are critically judged on the basis of the used datasets, deep learning models used and the results obtained. The third part of this review addresses the challenge of integrating the biometrics of the ears as a secondary biometric used by relying on multimodal fusion methods. In this case, we look at how different fusion techniques are applied in combining faces and ear features such as early, late and hybrid fusion, the data sets involved and the comparison on the increase of performance yield. The paper ends by supporting the main hypothesis ear biometrics as a fusion option plays a major role in improving the identification accuracy during the scenarios where facial data is deeply compromised. The survey will inform future work regards to the possibility to use multimodal systems particularly those involving the use of ear and face modalities in addressing the issue of occlusion in biometric identification.