Araştırma Makalesi
BibTex RIS Kaynak Göster

Mask Fitting on Face Images Based on Morphing and Masked Face Recognition

Yıl 2023, Cilt: 27 Sayı: 1, 12 - 21, 25.04.2023

Asuman Günay Yılmaz Eyup Gedikli Omar Alhori

https://doi.org/10.19113/sdufenbed.1059761

Öz

Viruses such as Covid-19 cause serious respiratory diseases, making the use of face masks important. For this reason, it is expected that face recognition and verification systems can also work with masked faces. There are no masked faces in the data sets created for face recognition systems, and today, masks of different models and patterns can be used in daily life. These reduce the success of face recognition systems. In this study, first, a wide masked face data set was produced by fitting different types of masks to the images in the existing face datasets. In the production of masked faces close to nature, the automatic mask fitting process was carried out with the image morphing technique. Then, a deep learning based model was developed for the recognition of masked/unmasked faces, and the dataset created with the automatic mask fitting technique was used to train the model. In the experiments using CASIA-WebFace and LFW (Labeled Faces in the Wild) datasets, masked face recognition performance was achieved over 96.5%.

Anahtar Kelimeler

Masked face recognition Biometric identification Automatic mask fitting Image morphing Deep learning

Kaynakça

  • [1] Dlib C++ Kütüphanesi, 2021. http://dlib.net/ (Erişim Tarihi : 15.12.2021).
  • [2] Yüz Hizalama Detektörü GTX kütüphanesi, 2015. https://github.com/davisking/dlib-models (Erişim Tarihi : 22.09.2021).
  • [3] Schroff, F., Kalenichenko, D., Philbin, J. 2015. Facenet: A Unified Embedding for Face Recognition and Clustering. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 7-12 June, Boston, MA, USA, 815-823.
  • [4] Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A. A. 2017. Inception-v4, Inception-Resnet and The Impact of Residual Connections on Learning. Thirty-first AAAI Conference on Artificial Intelligence (AAAI17), 4-9 February, San Francisco, California USA, 4278-4284. 2017.
  • [5] Zhang, K., Zhang, Z., Li, Z., Qiao, Y. 2016. Joint face Detection and Alignment Using Multitask Cascaded Convolutional Networks. IEEE Signal Processing Letters, 23(10), 1499 – 1503.
  • [6] Qi, C., Yang, L. 2020. Face Recognition in The Scene of Wearing A Mask. 2020 International Conference on Advance in Ambient Computing and Intelligence (ICAACI). 12-13 September, Ottawa, ON, Canada, 77-80.
  • [7] Alrikabi, J.M., Alibraheemi, K. H. 2021. A Combination Approach for Masked Face Recognition Based on Deep Learning. AM, 9(3), 499–520.
  • [8] Wang, Z., Wang, G., Huang, B., Xiong, Z., Hong, Q., Wu, H., Yi, P., Jiang, K., Wang, N., Pei, Y., Chen, H., Miao,Y., Huang, Z., Liang, J. 2020. Masked Face Recognition Dataset and Application. arXiv preprint arXiv:2003.09093.
  • [9] Mazli Shahar M.S., Mazalan, L. 2021. Face Identity for Face Mask Recognition System. 2021 IEEE 11th IEEE Symposium on Computer Applications &Industrial Electronics (ISCAIE). 3-4 April, Penang, Malaysia, 42-47.
  • [10] Hariri, W. 2021. Efficient Masked Face Recognition Method during the COVID-19 Pandemic. arXiv preprint arXiv:2105.03026.
  • [11] Vu, H.N., Nguyen, M.H., Pham, C. 2021. Masked face recognition with convolutional neural networks and local binary patterns. Applied Intelligence, Springer, Online.
  • [12] Ejaz, M.S., Islam, M. R. 2019. Masked Face Recognition Using Convolutional Neural Network. 2019 International Conference on Sustainable Technologies for Industry 4.0 (STI), 24-25 December, Dhaka, Bangladesh, 1-6.
  • [13] Jeevan, G., Zacharias, G.C., Nair, M.S., Rajan, J. 2022. An Empirical Study of The Impact of Masks on Face Recognition. Pattern Recognition, Volume 122(2022) 108308.
  • [14] Luo, X., He, X., Qing, L., Chen, X., Liu, L., Xu, Y. 2020. EyesGAN: Synthesize Human Face from Human Eyes. Neurocomputing, 404(2020), 213-226.
  • [15] SMalakar, S., Chiracharit, W., Chamnongthai, K., Charoenpong, T. 2021. Masked Face Recognition Using Principal Component Analysis and Deep Learning. 2021 18th International Conference on Electrical Engineering/Electronics Computer Telecommunications and Information Technology (ECTI-CON), 19-22 May, Chiang, Thailand, 785-788.
  • [16] Anwar, A., Raychowdhury, A. 2020. Masked Face Recognition for Secure Authentication. arXiv preprint arXiv:2008.11104.
  • [17] Jang, Y., Gunes, H., Patras, I. 2019. Registration-Free Face-ssd: Single Shot Analysis of Smiles, Facial Attributes and Affect in The Wild. Computer Vision and Image Understanding, 182, 17–29.
  • [18] Menon, A. 2019. Towards Data Science. https://towardsdatascience.com/face-detection-in-2-minutes-using-opencv-python-90f89d7c0f81 (Erişim Tarihi: 22.04.2021)
  • [19] Szegedy. C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A. 2015. Going Deeper with Convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 7-12 June, Boston, MA, 1–9.
  • [20] Ioffe, S., Szegedy, C., 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. ICML, arXiv: 1502.03167v3.
  • [21] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z. 2016. Rethinking The Inception Architecture for Computer Vision. 2016 IEEE Conference on Computer Vision and Pattern Recognition, 27-30 June, Las Vegas, NV, USA, 2818–2826.
  • [22] He, K., Zhang, X., Ren, S., Sun, J. 2016. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition, 27-30 June, Las Vegas, NV, USA, 770–778.
  • [23] Yi, D., Lei, Z. Liao, S., Li, S. Z. 2014. Learning Face Representation From Scratch. arXiv:1411.7923.
  • [24] Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E. 2008. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments, Workshop on Faces in ’Real-Life’ Images: Detection, Alignment, and Recognition, Erik Learned-Miller and Andras Ferencz and Frédéric Jurie, October 2008, Marseille, France. ffinria-00321923.

Yüz Görüntülerine Morflemeye Dayalı Maske Giydirme ve Maskeli Yüz Tanıma

Yıl 2023, Cilt: 27 Sayı: 1, 12 - 21, 25.04.2023

Asuman Günay Yılmaz Eyup Gedikli Omar Alhori

https://doi.org/10.19113/sdufenbed.1059761

Öz

Covid-19 gibi virüslerin ciddi solunum yolu hastalıklarına neden olması, yüz maskelerinin kullanımını önemli hale getirmiştir. Bu nedenle yüzden kişi doğrulama ve tanıma yapan sistemlerin maskeli yüzler üzerinde de çalışabilmesi beklenmektedir. Yüz tanıma sistemleri için oluşturulan veri setlerinde maskeli yüzler olmamakla birlikte günümüzde farklı model ve desenlerde maskeler kullanılabilmektedir. Bunlar yüz tanıma sistemlerinin başarısını düşürmektedir. Bu çalışmada öncelikle maskeli yüz veri seti üretmek için mevcut yüz veri setlerine farklı tipteki maskelerin giydirilmesine çalışılmıştır. Morfleme tekniği kullanılarak yüzün pozisyonlarına uygun olarak doğala yakın otomatik giydirme işlemi gerçekleştirilmiştir. Daha sonra maskeli/maskesiz yüzlerin tanınması için derin
öğrenmeye dayalı bir model geliştirilmiş ve otomatik maske giydirme tekniği ile oluşturulan veri seti denenmiştir. CASIA-WebFace ve LFW (Labeled Faces in the Wild) veri setleri kullanılarak gerçekleştirilen deneylerde %96.5’in üzerinde maskeli yüz tanıma başarımı elde edilmiştir.

Anahtar Kelimeler

Maskeli yüz tanıma Biyometrik kimlik tanıma Otomatik maske giydirme Görüntü morfleme Derin öğrenme

Kaynakça

  • [1] Dlib C++ Kütüphanesi, 2021. http://dlib.net/ (Erişim Tarihi : 15.12.2021).
  • [2] Yüz Hizalama Detektörü GTX kütüphanesi, 2015. https://github.com/davisking/dlib-models (Erişim Tarihi : 22.09.2021).
  • [3] Schroff, F., Kalenichenko, D., Philbin, J. 2015. Facenet: A Unified Embedding for Face Recognition and Clustering. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 7-12 June, Boston, MA, USA, 815-823.
  • [4] Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A. A. 2017. Inception-v4, Inception-Resnet and The Impact of Residual Connections on Learning. Thirty-first AAAI Conference on Artificial Intelligence (AAAI17), 4-9 February, San Francisco, California USA, 4278-4284. 2017.
  • [5] Zhang, K., Zhang, Z., Li, Z., Qiao, Y. 2016. Joint face Detection and Alignment Using Multitask Cascaded Convolutional Networks. IEEE Signal Processing Letters, 23(10), 1499 – 1503.
  • [6] Qi, C., Yang, L. 2020. Face Recognition in The Scene of Wearing A Mask. 2020 International Conference on Advance in Ambient Computing and Intelligence (ICAACI). 12-13 September, Ottawa, ON, Canada, 77-80.
  • [7] Alrikabi, J.M., Alibraheemi, K. H. 2021. A Combination Approach for Masked Face Recognition Based on Deep Learning. AM, 9(3), 499–520.
  • [8] Wang, Z., Wang, G., Huang, B., Xiong, Z., Hong, Q., Wu, H., Yi, P., Jiang, K., Wang, N., Pei, Y., Chen, H., Miao,Y., Huang, Z., Liang, J. 2020. Masked Face Recognition Dataset and Application. arXiv preprint arXiv:2003.09093.
  • [9] Mazli Shahar M.S., Mazalan, L. 2021. Face Identity for Face Mask Recognition System. 2021 IEEE 11th IEEE Symposium on Computer Applications &Industrial Electronics (ISCAIE). 3-4 April, Penang, Malaysia, 42-47.
  • [10] Hariri, W. 2021. Efficient Masked Face Recognition Method during the COVID-19 Pandemic. arXiv preprint arXiv:2105.03026.
  • [11] Vu, H.N., Nguyen, M.H., Pham, C. 2021. Masked face recognition with convolutional neural networks and local binary patterns. Applied Intelligence, Springer, Online.
  • [12] Ejaz, M.S., Islam, M. R. 2019. Masked Face Recognition Using Convolutional Neural Network. 2019 International Conference on Sustainable Technologies for Industry 4.0 (STI), 24-25 December, Dhaka, Bangladesh, 1-6.
  • [13] Jeevan, G., Zacharias, G.C., Nair, M.S., Rajan, J. 2022. An Empirical Study of The Impact of Masks on Face Recognition. Pattern Recognition, Volume 122(2022) 108308.
  • [14] Luo, X., He, X., Qing, L., Chen, X., Liu, L., Xu, Y. 2020. EyesGAN: Synthesize Human Face from Human Eyes. Neurocomputing, 404(2020), 213-226.
  • [15] SMalakar, S., Chiracharit, W., Chamnongthai, K., Charoenpong, T. 2021. Masked Face Recognition Using Principal Component Analysis and Deep Learning. 2021 18th International Conference on Electrical Engineering/Electronics Computer Telecommunications and Information Technology (ECTI-CON), 19-22 May, Chiang, Thailand, 785-788.
  • [16] Anwar, A., Raychowdhury, A. 2020. Masked Face Recognition for Secure Authentication. arXiv preprint arXiv:2008.11104.
  • [17] Jang, Y., Gunes, H., Patras, I. 2019. Registration-Free Face-ssd: Single Shot Analysis of Smiles, Facial Attributes and Affect in The Wild. Computer Vision and Image Understanding, 182, 17–29.
  • [18] Menon, A. 2019. Towards Data Science. https://towardsdatascience.com/face-detection-in-2-minutes-using-opencv-python-90f89d7c0f81 (Erişim Tarihi: 22.04.2021)
  • [19] Szegedy. C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A. 2015. Going Deeper with Convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 7-12 June, Boston, MA, 1–9.
  • [20] Ioffe, S., Szegedy, C., 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. ICML, arXiv: 1502.03167v3.
  • [21] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z. 2016. Rethinking The Inception Architecture for Computer Vision. 2016 IEEE Conference on Computer Vision and Pattern Recognition, 27-30 June, Las Vegas, NV, USA, 2818–2826.
  • [22] He, K., Zhang, X., Ren, S., Sun, J. 2016. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition, 27-30 June, Las Vegas, NV, USA, 770–778.
  • [23] Yi, D., Lei, Z. Liao, S., Li, S. Z. 2014. Learning Face Representation From Scratch. arXiv:1411.7923.
  • [24] Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E. 2008. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments, Workshop on Faces in ’Real-Life’ Images: Detection, Alignment, and Recognition, Erik Learned-Miller and Andras Ferencz and Frédéric Jurie, October 2008, Marseille, France. ffinria-00321923.
Toplam 24 adet kaynakça vardır.

Ayrıntılar

Birincil Dil Türkçe
Konular Mühendislik
Bölüm Makaleler
Yazarlar

Asuman Günay Yılmaz 0000-0003-3960-5085

Eyup Gedikli 0000-0002-7212-5457

Omar Alhori Bu kişi benim

Yayımlanma Tarihi 25 Nisan 2023
Yayımlandığı Sayı Yıl 2023 Cilt: 27 Sayı: 1

Kaynak Göster

APA Günay Yılmaz, A., Gedikli, E., & Alhori, O. (2023). Yüz Görüntülerine Morflemeye Dayalı Maske Giydirme ve Maskeli Yüz Tanıma. Süleyman Demirel Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 27(1), 12-21. https://doi.org/10.19113/sdufenbed.1059761
AMA Günay Yılmaz A, Gedikli E, Alhori O. Yüz Görüntülerine Morflemeye Dayalı Maske Giydirme ve Maskeli Yüz Tanıma. SDÜ Fen Bil Enst Der. Nisan 2023;27(1):12-21. doi:10.19113/sdufenbed.1059761
Chicago Günay Yılmaz, Asuman, Eyup Gedikli, ve Omar Alhori. “Yüz Görüntülerine Morflemeye Dayalı Maske Giydirme Ve Maskeli Yüz Tanıma”. Süleyman Demirel Üniversitesi Fen Bilimleri Enstitüsü Dergisi 27, sy. 1 (Nisan 2023): 12-21. https://doi.org/10.19113/sdufenbed.1059761.
EndNote Günay Yılmaz A, Gedikli E, Alhori O (01 Nisan 2023) Yüz Görüntülerine Morflemeye Dayalı Maske Giydirme ve Maskeli Yüz Tanıma. Süleyman Demirel Üniversitesi Fen Bilimleri Enstitüsü Dergisi 27 1 12–21.
IEEE A. Günay Yılmaz, E. Gedikli, ve O. Alhori, “Yüz Görüntülerine Morflemeye Dayalı Maske Giydirme ve Maskeli Yüz Tanıma”, SDÜ Fen Bil Enst Der, c. 27, sy. 1, ss. 12–21, 2023, doi: 10.19113/sdufenbed.1059761.
ISNAD Günay Yılmaz, Asuman vd. “Yüz Görüntülerine Morflemeye Dayalı Maske Giydirme Ve Maskeli Yüz Tanıma”. Süleyman Demirel Üniversitesi Fen Bilimleri Enstitüsü Dergisi 27/1 (Nisan 2023), 12-21. https://doi.org/10.19113/sdufenbed.1059761.
JAMA Günay Yılmaz A, Gedikli E, Alhori O. Yüz Görüntülerine Morflemeye Dayalı Maske Giydirme ve Maskeli Yüz Tanıma. SDÜ Fen Bil Enst Der. 2023;27:12–21.
MLA Günay Yılmaz, Asuman vd. “Yüz Görüntülerine Morflemeye Dayalı Maske Giydirme Ve Maskeli Yüz Tanıma”. Süleyman Demirel Üniversitesi Fen Bilimleri Enstitüsü Dergisi, c. 27, sy. 1, 2023, ss. 12-21, doi:10.19113/sdufenbed.1059761.
Vancouver Günay Yılmaz A, Gedikli E, Alhori O. Yüz Görüntülerine Morflemeye Dayalı Maske Giydirme ve Maskeli Yüz Tanıma. SDÜ Fen Bil Enst Der. 2023;27(1):12-21.
 Kapak Resmi İndir

e-ISSN: 1308-6529