Cover
Vol. 20 No. 1 (2024)

Published: June 30, 2024

Pages: 84-94

Original Article

Transfer Learning Based Fine-Tuned Novel Approach for Detecting Facial Retouching

Kinjal R. Sheth 1 Vishal S. Vora 2
1Research Scholar, Electronics & Communication, Atmiya Institute of Technology, Atmiya University, Gujarat, India
2Head & Associate Professor, Atmiya Institute of Technology, Atmiya University, Gujarat, India
History
  • Received: July 13, 2023
  • Revised: August 25, 2023
  • Accepted: August 26, 2023
  • Available Online: November 11, 2023
DOI

https://doi.org/10.37917/ijeee.20.1.9

Abstract

Facial retouching, also referred to as digital retouching, is the process of modifying or enhancing facial characteristics in digital images or photographs. While it can be a valuable technique for fixing flaws or achieving a desired visual appeal, it also gives rise to ethical considerations. This study involves categorizing genuine and retouched facial images from the standard ND-IIITD retouched faces dataset using a transfer learning methodology. The impact of different primary optimization algorithms—specifically Adam, RMSprop, and Adadelta—utilized in conjunction with a fine-tuned ResNet50 model is examined to assess potential enhancements in classification effectiveness. Our proposed transfer learning ResNet50 model demonstrates superior performance compared to other existing approaches, particularly when the RMSprop and Adam optimizers are employed in the fine-tuning process. By training the transfer learning ResNet50 model on the ND-IIITD retouched faces dataset with the ”ImageNet” weight, we achieve a validation accuracy of 98.76%, a training accuracy of 98.32%, and an overall accuracy of 98.52% for classifying real and retouched faces in just 20 epochs. Comparative analysis indicates that the choice of optimizer during the fine-tuning of the transfer learning ResNet50 model can further enhance the classification accuracy.

Keywords

References

  1. N. Sharma, V. Jain, and A. Mishra, “An analysis of convolutional neural networks for image classification,” Procedia computer science, vol. 132, pp. 377–384, 2018.
  2. S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Transactions on knowledge and data engineering, vol. 22, no. 10, pp. 1345–1359, 2009.
  3. J. Szewczyk, “Photoshop law: Legislating beauty in the media and fashion industry,” Available at SSRN 3267772, 2014.
  4. N. Photograph, “Photography and law , notes queries,” vol. s1-XI, no. 27, pp. 16–16, 1855.
  5. I. News, “Supermodels without photoshop: Photoshop law puts focus on digitally altered images,” 2020.
  6. V. Vinolin and M. Sucharitha, “Hierarchical categoriza- tion and review of recent techniques on image forgery detection,” The Computer Journal, vol. 64, no. 11, pp. 1692–1704, 2021.
  7. W. Wang, J. Dong, and T. Tan, “A survey of passive image tampering detection,” in Digital Watermarking: 8th International Workshop, IWDW 2009, Guildford, (a)Retouched (b)Real face images UK, August 24-26, 2009. Proceedings 8, pp. 308–322, Springer, 2009.
  8. E. Kee and H. Farid, “A perceptual metric for photo retouching,” Proceedings of the National Academy of Sciences, vol. 108, no. 50, pp. 19907–19912, 2011.
  9. A. Bharati, R. Singh, M. Vatsa, and K. W. Bowyer, “De- tecting facial retouching using supervised deep learning,” IEEE Transactions on Information Forensics and Secu- rity, vol. 11, no. 9, pp. 1903–1913, 2016.
  10. A. Bharati, M. Vatsa, R. Singh, K. W. Bowyer, and X. Tong, “Demography-based facial retouching detec- tion using subclass supervised sparse autoencoder,” in 2017 IEEE international joint conference on biometrics (IJCB), pp. 474–482, IEEE, 2017.
  11. A. Jain, R. Singh, and M. Vatsa, “On detecting gans and retouching based synthetic alterations,” in 2018 IEEE 94 | Sheth & Vora 9th international conference on biometrics theory, appli- cations and systems (BTAS), pp. 1–7, IEEE, 2018.
  12. A. Dantcheva, C. Chen, and A. Ross, “Can facial cos- metics affect the matching accuracy of face recognition systems?,” in 2012 IEEE Fifth international conference on biometrics: theory, applications and systems (BTAS), pp. 391–398, IEEE, 2012.
  13. C. Chen, A. Dantcheva, and A. Ross, “Automatic facial makeup detection with application in face recognition,” in 2013 international conference on biometrics (ICB), pp. 1–8, IEEE, 2013.
  14. M.-L. Eckert, N. Kose, and J.-L. Dugelay, “Facial cos- metics database and impact analysis on automatic face recognition,” in 2013 IEEE 15th international workshop on multimedia signal processing (MMSP), pp. 434–439, IEEE, 2013.
  15. N. Kose, L. Apvrille, and J.-L. Dugelay, “Facial makeup detection technique based on texture and shape analy- sis,” in 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), vol. 1, pp. 1–7, IEEE, 2015.
  16. R. Singh, M. Vatsa, and A. Noore, “Effect of plastic surgery on face recognition: A preliminary study,” in 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 72–77, IEEE, 2009.
  17. C. Rathgeb, A. Botaljov, F. Stockhardt, S. Isadskiy, L. Debiasi, A. Uhl, and C. Busch, “Prnu-based detec- tion of facial retouching,” IET Biometrics, vol. 9, no. 4, pp. 154–164, 2020.
  18. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recog- nition, pp. 770–778, 2016.
  19. I. Kandel, M. Castelli, and A. Popoviˇc, “Comparative study of first order optimizers for image classification using convolutional neural networks on histopathology images,” Journal of imaging, vol. 6, no. 9, p. 92, 2020.
  20. P. Peng and J. Wang, “How to fine-tune deep neu- ral networks in few-shot learning?,” arXiv preprint arXiv:2012.00204, 2020.
  21. M. B. Hossain, S. H. S. Iqbal, M. M. Islam, M. N. Akhtar, and I. H. Sarker, “Transfer learning with fine- tuned deep cnn resnet50 model for classifying covid-19 from chest x-ray images,” Informatics in Medicine Un- locked, vol. 30, p. 100916, 2022.