Cover
Vol. 22 No. 1 (2026)

Published: June 15, 2026

Pages: 46-55

Original Article

A Multi-Modal Convolutional Neural Network for Face Anti-Spoofing Detection

Hala S. Mahmood 1 Salah Al-Darraji 2
1 Dept. of Computer Science, College of Education for Pure Sciences, University of Basrah, Basrah, Iraq
2 Dept. of Computer Science, College of Computer Science & Information Technology, University of Basrah, Basrah, Iraq
History
  • Received: October 8, 2023
  • Revised: December 20, 2023
  • Accepted: December 21, 2023
  • Available Online: June 15, 2026
DOI

https://doi.org/10.37917/ijeee.22.1.5

Abstract

Recently, face recognition technology has become more prevalent in various applications, including mobile devices, access control, and financial transactions. Therefore, it is crucial to address potential vulnerabilities that attackers might exploit. In this study, a method for face presentation attack detection (PAD) is introduced. The method utilizes the diversity of modalities provided by some cameras and sensors to detect face spoofing using convolutional neural networks (CNN) within the context of deep learning. To assess the effectiveness of the proposed approach in real-world scenarios, the wide multi-channel presentation attack (WMCA) dataset is used. The presented method exploits the multi-modal data, including RGB, depth, IR, and thermal channels, to enhance system performance and explore different techniques for combining the results from each modality. Furthermore, this study explores diverse techniques for fusing results from each channel in two fusion scenarios, pre-fusion and post-fusion. In the pre-fusion scenario, data from the four channels is combined, resulting in an ACER value of 0.19%. In the post-fusion scenario, the results of each modality are fused using different fusion techniques, such as majority voting, weighted voting, average pooling, and a stacking classifier. The stacking classifier yields the most favorable outcome with an ACER ratio of 0.03%. This performance is notably superior when compared to state-of-the-art methodologies.

Keywords

References

  1. P. J. Phillips, A. N. Yates, Y. Hu, C. A. Hahn, E. Noyes, K. Jackson, J. G. Cavazos, G. Jeckeln, R. Ranjan, S. Sankaranarayanan, et al., “Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms,” Proceedings of the National Academy of Sciences, vol. 115, no. 24, pp. 6171–6176, 2018.
  2. Y. Guo, L. Zhang, Y. Hu, X. He, and J. Gao, “Ms-celeb- 1m: A dataset and benchmark for large-scale face recognition,” in European conference on computer vision, pp. 87–102, Springer, 2016.
  3. O. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition,” in BMVC 2015-Proceedings of the British Machine Vision Conference 2015, British Machine Vision Association, 2015.
  4. A. Hadid, “Face biometrics under spoofing attacks: Vulnerabilities, countermeasures, open issues, and research directions,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 113–118, 2014.
  5. A. Kadhim and S. Al-Darraji, “Face recognition system against adversarial attack using convolutional neural network.,” Iraqi Journal for Electrical & Electronic Engineering, vol. 18, no. 1, 2022.
  6. U. Scherhag, R. Raghavendra, K. B. Raja, M. Gomez- Barrero, C. Rathgeb, and C. Busch, “On the vulnerability of face recognition systems towards morphed face attacks,” in 2017 5th international workshop on biometrics and forensics (IWBF), pp. 1–6, IEEE, 2017.
  7. Z. Boulkenafet, J. Komulainen, L. Li, X. Feng, and A. Hadid, “Oulu-npu: A mobile face presentation attack database with real-world variations,” in 2017 12th IEEE international conference on automatic face & gesture recognition (FG 2017), pp. 612–618, IEEE, 2017.
  8. A. Costa-Pazo, S. Bhattacharjee, E. Vazquez-Fernandez, and S. Marcel, “The replay-mobile face presentationattack database,” in 2016 international conference of the Biometrics Special Interest Group (BIOSIG), pp. 1–7, IEEE, 2016.
  9. D. Wen, H. Han, and A. K. Jain, “Face spoof detection with image distortion analysis,” IEEE Transactions on Information Forensics and Security, vol. 10, no. 4, pp. 746–761, 2015.
  10. I. Chingovska, N. Erdogmus, A. Anjos, and S. Marcel, “Face recognition systems under spoofing attacks,” in Face Recognition Across the Imaging Spectrum, pp. 165– 194, Springer, 2016.
  11. N. Erdogmus and S. Marcel, “Spoofing in 2d face recognition with 3d masks and anti-spoofing with kinect,” in 2013 IEEE sixth international conference on biometrics: theory, applications and systems (BTAS), pp. 1–6, IEEE, 2013.
  12. A. George, Z. Mostaani, D. Geissenbuhler, O. Nikisins, A. Anjos, and S. Marcel, “Biometric face presentation attack detection with multi-channel convolutional neural network,” IEEE transactions on information forensics and security, vol. 15, pp. 42–55, 2019.
  13. A. Denisova, “An improved simple feature set for face presentation attack detection,” 2022.
  14. Y. Zheng and E. A. Essock, “A local-coloring method for night-vision colorization utilizing image analysis and fusion,” Information Fusion, vol. 9, no. 2, pp. 186–199, 2008.
  15. J. Kittler, M. Hatef, R. P. Duin, and J. Matas, “On combining classifiers,” IEEE transactions on pattern analysis and machine intelligence, vol. 20, no. 3, pp. 226–239, 2002.
  16. L. I. Kuncheva, Combining pattern classifiers: methods and algorithms. John Wiley & Sons, 2014.
  17. D. H. Wolpert, “Stacked generalization,” Neural networks, vol. 5, no. 2, pp. 241–259, 1992.
  18. I. Chingovska, A. Anjos, and S. Marcel, “On the effectiveness of local binary patterns in face anti-spoofing,” in 2012 BIOSIG-proceedings of the international conference of biometrics special interest group (BIOSIG), pp. 1–7, IEEE, 2012.
  19. Z. Zhang, J. Yan, S. Liu, Z. Lei, D. Yi, and S. Z. Li, “A face antispoofing database with diverse attacks,” in 2012 5th IAPR international conference on Biometrics (ICB), pp. 26–31, IEEE, 2012.
  20. Y. Liu, A. Jourabloo, and X. Liu, “Learning deep models for face anti-spoofing: Binary or auxiliary supervision,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 389–398, 2018.
  21. M. Fang, M. Huber, and N. Damer, “Synthaspoof: Developing face presentation attack detection based on privacy-friendly synthetic data,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1061–1070, 2023.
  22. N. Erdogmus and S. Marcel, “Spoofing face recognition with 3d masks,” IEEE transactions on information forensics and security, vol. 9, no. 7, pp. 1084–1097, 2014.
  23. H. Steiner, A. Kolb, and N. Jung, “Reliable face antispoofing using multispectral swir imaging,” in 2016 international conference on biometrics (ICB), pp. 1–8, IEEE, 2016.
  24. N. Kose and J.-L. Dugelay, “Countermeasure for the protection of face recognition systems against mask attacks,” in 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp. 1–6, IEEE, 2013.
  25. L. Feng, L.-M. Po, Y. Li, X. Xu, F. Yuan, T. C.-H. Cheung, and K.-W. Cheung, “Integration of image quality and motion cues for face anti-spoofing: A neural network approach,” Journal of Visual Communication and Image Representation, vol. 38, pp. 451–460, 2016.
  26. K. Patel, H. Han, and A. K. Jain, “Cross-database face antispoofing with robust feature representation,” in Chinese Conference on Biometric Recognition, pp. 611–619, Springer, 2016.
  27. J. Yang, Z. Lei, and S. Z. Li, “Learn convolutional neural network for face anti-spoofing,” arXiv preprint arXiv:1408.5601, 2014.
  28. W. Luo, P. Sun, F. Zhong,W. Liu, T. Zhang, and Y.Wang, “End-to-end active object tracking and its real-world deployment via reinforcement learning,” IEEE transactions on pattern analysis and machine intelligence, vol. 42, no. 6, pp. 1317–1332, 2019.
  29. L. Li, X. Feng, Z. Boulkenafet, Z. Xia, M. Li, and A. Hadid, “An original face anti-spoofing approach using partial convolutional neural network,” in 2016 Sixth international conference on image processing theory, tools and applications (IPTA), pp. 1–6, IEEE, 2016.
  30. Y. Kim, J. Na, S. Yoon, and J. Yi, “Masked fake face detection using radiance measurements,” Journal of the Optical Society of America A, vol. 26, no. 4, pp. 760–766, 2009.
  31. S. Bhattacharjee, A. Mohammadi, and S. Marcel, “Spoofing deep face recognition with custom silicone masks,” in 2018 IEEE 9th international conference on biometrics theory, applications and systems (BTAS), pp. 1–7, IEEE, 2018.
  32. Y. Li and Y. Yuan, “Convergence analysis of two-layer neural networks with relu activation,” Advances in neural information processing systems, vol. 30, 2017.
  33. A. C. Marreiros, J. Daunizeau, S. J. Kiebel, and K. J. Friston, “Population dynamics: variance and the sigmoid activation function,” Neuroimage, vol. 42, no. 1, pp. 147– 157, 2008.
  34. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
  35. M. W. Browne, “Cross-validation methods,” Journal of mathematical psychology, vol. 44, no. 1, pp. 108–132, 2000.
  36. J. D. Rodriguez, A. Perez, and J. A. Lozano, “Sensitivity analysis of k-fold cross validation in prediction error estimation,” IEEE transactions on pattern analysis and machine intelligence, vol. 32, no. 3, pp. 569–575, 2009.
  37. J. Brownlee and M. L. Mastery, Deep Learning with Python: Develop Deep Learning Models on Theano and TensorFlow Using Keras. Machine Learning Mastery, 2017.
  38. I. Standard, “Information technology–biometric presentation attack detection–part 3: testing and reporting,” International Organization for Standardization: Geneva, Switzerland, vol. 7, 2017.
  39. K. E. Ewald, L. Zeng, C. B. Mawuli, H. S. Abubakar, A. Victor, et al., “Applying cnn with extracted facial patches using 3 modalities to detect 3d face spoof,” in 2020 17th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), pp. 216–221, IEEE, 2020.
  40. X. Wu, R. He, Z. Sun, and T. Tan, “A light cnn for deep face representation with noisy labels,” IEEE transactions on information forensics and security, vol. 13, no. 11, pp. 2884–2896, 2018.