Cover
Vol. 20 No. 2 (2024)

Published: December 31, 2024

Pages: 154-164

Review Article

Advancements and Challenges in Hand Gesture Recognition: A Comprehensive Review

Bothina Kareem Murad 1 Abbas H. Hassin Alasadi
1College of Computer Science and Information Technology, University of Basrah, Basrah, Iraq.
History
  • Received: July 23, 2023
  • Revised: September 24, 2023
  • Accepted: November 5, 2023
  • Available Online: July 3, 2024
DOI

https://doi.org/10.37917/ijeee.20.2.13

Abstract

Hand gesture recognition is a quickly developing field with many uses in human-computer interaction, sign language recognition, virtual reality, gaming, and robotics. This paper reviews different ways to model hands, such as vision-based, sensor-based, and data glove-based techniques. It emphasizes the importance of accurate hand modeling and feature extraction for capturing and analyzing gestures. Key features like motion, depth, color, shape, and pixel values and their relevance in gesture recognition are discussed. Challenges faced in hand gesture recognition include lighting variations, complex backgrounds, noise, and real-time performance. Machine learning algorithms are used to classify and recognize gestures based on extracted features. The paper emphasizes the need for further research and advancements to improve hand gesture recognition systems’ robustness, accuracy, and usability. This review offers valuable insights into the current state of hand gesture recognition, its applications, and its potential to revolutionize human-computer interaction and enable natural and intuitive interactions between humans and machines. In simpler terms, hand gesture recognition is a way for computers to understand what people are saying with their hands. It has many potential applications, such as allowing people to control computers without touching them or helping people with disabilities communicate. The paper reviews different ways to develop hand gesture recognition systems and discusses the challenges and opportunities in this area.

Keywords

References

  1. “Webster’s dictionary accessed: 12-oct-2022.”
  2. T. Zhang, Y. Ding, C. Hu, M. Zhang, W. Zhu, C. R. Bowen, Y. Han, and Y. Yang, “Self-powered stretch- able sensor arrays exhibiting magnetoelasticity for real- time human–machine interaction,” Advanced Materials, vol. 2203786, p. 2203786, 2022.
  3. F. A. Farid, N. Hashim, J. Abdullah, M. R. Bhuiyan, W. N. S. M. Isa, J. Uddin, M. A. Haque, and M. N. Husen, “A structured and methodological review on vision-based hand gesture recognition system,” Journal of Imaging, vol. 8, no. 6, p. 153, 2022.
  4. M. G. A. J. P. Rawat, L. Kane and S. Sehgal, “A re- view on vision-based hand gesture recognition targeting rgb-depth sensors,” International Journal of Informa- tion Technology and Decision Making, vol. 22, no. 01, pp. 115–156, 2023.
  5. S. Wu, Z. Li, S. Li, Q. Liu, and W. Wu, “An overview of gesture recognition,” in International Conference on Computer Application and Information Security (IC- CAIS 2022), vol. 12609, pp. 600–606, SPIE, Mar 2023.
  6. R. F. Pinto Jr, C. D. Borges, A. M. Almeida, and I. C. Paula Jr, “Static hand gesture recognition based on con- volutional neural networks,” Journal of Electrical and Computer Engineering, vol. 2019, no. 1, p. 4167890, 2019.
  7. A. K. H. AlSaedi and A. H. H. AlAsadi, “A new hand gestures recognition system,” Indonesian journal of elec- trical engineering and computer science, vol. 18, no. 1, pp. 49–55, 2020.
  8. P. Das, T. Ahmed, and M. F. Ali, “Static hand gesture recognition for american sign language using deep con- volutional neural network,” in 2020 IEEE Region 10 symposium (TENSYMP), pp. 1762–1765, IEEE, 2020.
  9. I. Papastratis, C. Chatzikonstantinou, D. Konstantinidis, K. Dimitropoulos, and P. Daras, “Artificial intelligence technologies for sign language,” Sensors, vol. 21, no. 17, p. 5843, 2021.
  10. L. I. Khalaf, S. A. Aswad, S. R. Ahmed, B. Makki, and M. R. Ahmed, “Survey on recognition hand gesture by using data mining algorithms,” in 2022 International Congress on Human-Computer Interaction, Optimiza- tion and Robotic Applications (HORA), pp. 1–4, IEEE, 2022.
  11. M. Oudah, A. Al-Naji, and J. Chahl, “Hand gesture recognition based on computer vision: a review of tech- niques,” Journal of Imaging, vol. 6, no. 8, p. 73, 2020.
  12. D. V. Suma, “Computer vision for human-machine interaction-review,” Journal of Trends in Computer Sci- ence and Smart Technology, vol. 1, no. 2, pp. 131–139, 2019.
  13. T. Vuletic, A. Duffy, L. Hay, C. McTeague, G. Campbell, and M. Grealy, “Systematic literature review of hand ges- tures in human-computer interaction interfaces,” Inter- national Journal of Human-Computer Studies, vol. 129, pp. 74–94, 2019.
  14. H. Kawashima, Active Appearance Models, pp. 1–5. 2020.
  15. T. H. Tsai, C. C. Huang, and K. L. Zhang, “Design of hand gesture recognition system for human-computer interaction,” Multimedia tools and applications, vol. 79, pp. 5989–6007, 2020.
  16. T. L. Dang, H. T. Nguyen, D. M. Dao, H. V. Nguyen, D. L. Luong, B. T. Nguyen, S. Kim, and N. Monet, “Shape: a dataset for hand gesture recognition,” Neural Computing and Applications, vol. 34, pp. 21849–21862, Dec 2022. 162 | Murad & Alasadi
  17. L. M. Dang, K. Min, H. Wang, M. J. Piran, C. H. Lee, and H. Moon, “Sensor-based and vision-based human activity recognition: A comprehensive survey,” Pattern Recognition, vol. 108, Dec 2020.
  18. D. R. Beddiar, B. Nini, M. Sabokrou, and A. Hadid, “Vision-based human activity recognition: a survey,” Multimedia Tools and Applications, vol. 79, pp. 30509– 30555, Nov 2020.
  19. C. Z. Dong and F. N. Catbas, “A review of computer vision–based structural health monitoring at local and global levels,” Structural Health Monitoring, vol. 20, pp. 692–743, Mar 2021.
  20. D. Dai, W. Zhuang, Y. Shen, L. Li, and H. Wang, “De- sign of intelligent mobile robot control system based on gesture recognition,” in Artificial Intelligence and Secu- rity: 6th International Conference, ICAIS 2020, Hohhot, China, pp. 101–111, Springer Singapore, 2020.
  21. W. Lin, C. Li, and Y. Zhang, “Interactive application of data glove based on emotion recognition and judgment system,” Sensors, vol. 22, Aug 2022.
  22. N. Magrofuoco, P. Roselli, and J. Vanderdonckt, “Two- dimensional stroke gesture recognition: A survey,” ACM Computing Surveys (CSUR), vol. 54, pp. 1–36, Jul 2021.
  23. N. Saleh, M. Farghaly, E. Elshaaer, and A. Mousa, “Smart glove-based gestures recognition system for ara- bic sign language,” in 2020 International Conference on Innovative Trends in Communication and Computer Engineering (ITCE), pp. 303–307, IEEE, Feb 2020.
  24. Q. Fu, J. Fu, J. Guo, S. Guo, and X. Li, “Gesture recog- nition based on bp neural network and data glove,” in 2020 IEEE International Conference on Mechatronics and Automation (ICMA), pp. 1918–1922, IEEE, Oct 2020.
  25. R. Rastgoo, K. Kiani, and S. Escalera, “Sign language recognition: A deep survey,” Expert Systems with Appli- cations, vol. 164, 2021.
  26. S. Mitra and T. Acharya, “Gesture recognition: A sur- vey,” IEEE Transactions on Systems, Man, and Cybernet- ics, Part C (Applications and Reviews), vol. 37, pp. 311– 324, Apr 2007.
  27. A. Dzedzickis, A. Kaklauskas, and V. Bucinskas, “Hu- man emotion recognition: Review of sensors and meth- ods,” Sensors, vol. 20, Jan 2020.
  28. A. K. Al-Saedi and A. H. Al-Asadi, “Survey of hand ges- ture recognition systems,” in Journal of Physics: Con- ference Series, vol. 1294, IOP Publishing, Sep 2019.
  29. L. I. Yang, J. Huang, T. I. Feng, W. A. Hong-An, and D. A. Guo-Zhong, “Gesture interaction in virtual reality,” Virtual Reality and amp; Intelligent Hardware, vol. 1, pp. 84–112, Feb 2019.
  30. A. Mujahid, M. J. Awan, A. Yasin, M. A. Mohammed, R. Damaˇseviˇcius, R. Maskeli¯unas, and K. H. Abdulka- reem, “Real-time hand gesture recognition based on deep learning yolov3 model,” Applied Sciences, vol. 11, May 2021.
  31. N. D. Binh, E. Shuichi, and T. Ejima, “Real-time hand tracking and gesture recognition system,” in Proc. GVIP, pp. 19–21, Dec 2005.
  32. J. Zhao, A. Lyons, A. C. Ulku, H. Defienne, D. Faccio, and E. Charbon, “Light detection and ranging with entan- gled photons,” Optics Express, vol. 30, no. 3, pp. 3675– 3683, 2022.
  33. B. Ac¸ıs¸ and S. G¨uney, “Classification of human move- ments by using kinect sensor,” Biomedical Signal Pro- cessing and Control, vol. 81, 2023.
  34. S. Machado, V. Mercier, and N. Chiaruttini, “Limeseg: a coarse-grained lipid membrane simulation for 3d image segmentation,” BMC bioinformatics, vol. 20, pp. 1–2, 2019.
  35. Y. Huang and J. Yang, “A multi-scale descriptor for real- time rgb-d hand gesture recognition,” Pattern Recogni- tion Letters, vol. 144, pp. 97–104, 2021.
  36. M. Singh, I. V. Tewari, and L. Sheth, Skin-Colour-Based Hand Segmentation Techniques, pp. 1–26. IGI Global, 2022.
  37. X. Larriva-Novo, C. S´anchez-Zas, V. A. Villagr´a, M. Vega-Barbas, and D. Rivera, “An approach for the application of a dynamic multi-class classifier for net- work intrusion detection systems,” Electronics, vol. 9, no. 11, 2020.
  38. Y. Zhou, H. Guo, L. Ma, Z. Zhang, and M. Skit- more, “Image-based onsite object recognition for au- tomatic crane lifting tasks,” Automation in Construction, vol. 123, 2021.
  39. Z. Zou, K. Chen, Z. Shi, Y. Guo, and J. Ye, “Object detection in 20 years: A survey,” Proceedings of the IEEE, 2023. 163 | Murad & Alasadi
  40. B. Noh, H. Park, S. Lee, and S. H. Nam, “Vision-based pedestrian’s crossing risky behavior extraction and anal- ysis for intelligent mobility safety system,” Sensors, vol. 22, no. 9, 2022.
  41. M. K. Hu, “Visual pattern recognition by moment invari- ants,” IRE Transactions on Information Theory, vol. 8, no. 2, pp. 179–187, 1962.
  42. S. Katoch, V. Singh, and U. S. Tiwary, “Indian sign language recognition system using surf with svm and cnn,” Array, vol. 14, 2022.
  43. Z. Ren, F. Fang, N. Yan, and Y. Wu, “State of the art in defect detection based on machine vision,” International Journal of Precision Engineering and Manufacturing- Green Technology, vol. 9, no. 2, pp. 661–691, 2022.
  44. N. Mirehi, M. Tahmasbi, and A. T. Targhi, “Hand ges- ture recognition using topological features,” Multimedia Tools and Applications, vol. 78, pp. 13361–13386, 2019.
  45. M. Wagh and P. K. Nanda, “Decision-theoretic rough sets based automated scheme for object and background classification in unevenly illuminated images,” Applied Soft Computing, vol. 119, 2022.
  46. W. Chen, C. Yu, C. Tu, Z. Lyu, J. Tang, S. Ou, Y. Fu, and Z. Xue, “A survey on hand pose estimation with wearable sensors and computer-vision-based methods,” Sensors, vol. 20, no. 4, p. 1074, 2020.
  47. J. Qi, K. Xu, and X. Ding, “Approach to hand posture recognition based on hand shape features for a human- robot interaction,” Complex & Intelligent Systems, 2021.
  48. M. Al-Hammadi, G. Muhammad, W. Abdul, M. Alsu- laiman, M. A. Bencherif, and M. A. Mekhtiche, “Hand gesture recognition for sign language using 3dcnn,” IEEE Access, vol. 8, pp. 79491–79509, 2020.
  49. A. Thakur and A. Konde, “Fundamentals of neural net- works,” International Journal for Research in Applied Science and Engineering Technology, vol. 9, pp. 407– 426, 2021.
  50. T. H. Maung, “Real-time hand tracking and gesture recognition system using neural networks,” Interna- tional Journal of Computer and Information Engineer- ing, vol. 3, no. 2, pp. 315–319, 2009.
  51. E. Stergiopoulou and N. Papamarkos, “Hand gesture recognition using a neural network shape fitting tech- nique,” Engineering Applications of Artificial Intelli- gence, vol. 22, no. 8, pp. 1141–1158, 2009.
  52. M. H. Ismail, S. A. Dawwd, and F. H. Ali, “Dynamic hand gesture recognition of arabic sign language using deep convolutional neural networks,” Indones. J. Electr. Eng. Comput. Sci., vol. 25, pp. 952–962, 2022.
  53. N. Rajawat, N. Gupta, and S. Lalwani, “A comprehen- sive review of hidden markov model applications in pre- dicting human mobility patterns,” International Journal of Swarm Intelligence, vol. 6, no. 1, pp. 24–47, 2021.
  54. D. Sarma and M. K. Bhuyan, “Methods, databases and recent advancement of vision-based hand gesture recog- nition for hci systems: A review,” SN Computer Science, vol. 2, no. 6, 2021.
  55. S. Mandal, Z. Li, T. Chatterjee, K. Khanna, K. Montoya, L. Dai, C. Petersen, L. Li, M. Tewari, A. Johnson-Buck, and N. G. Walter, “Direct kinetic fingerprinting for high- accuracy single-molecule counting of diverse disease biomarkers,” Accounts of Chemical Research, vol. 54, no. 2, pp. 388–402, 2020.
  56. J. Arora, K. Khatter, and M. Tushir, “Fuzzy c-means clustering strategies: A review of distance measures,” in Software Engineering: Proceedings of CSI 2015, pp. 153–162, 2019.
  57. R. S. Gaikwad and L. S. Admuthe, “A review of vari- ous sign language recognition techniques,” in Modeling, Simulation, and Optimization: Proceedings of CoMSO 2021, pp. 111–126, jun 2022.
  58. K. Taunk, S. De, S. Verma, and A. Swetapadma, “A brief review of the nearest neighbor algorithm for learning and classification,” in 2019 International Conference on Intelligent Computing and Control Systems (ICCS), pp. 1255–1260, IEEE, may 2019.
  59. S. Ghosh, A. Dasgupta, and A. Swetapadma, “A study on support vector machine-based linear and non-linear pattern classification,” in 2019 International Conference on Intelligent Sustainable Systems (ICISS), pp. 24–28, IEEE, feb 2019.
  60. M. Yu, J. Jia, C. Xue, G. Yan, Y. Guo, and Y. Liu, “A review of sign language recognition research,” Journal of Intelligent & Fuzzy Systems, vol. 43, no. 4, pp. 3879– 3898, 2022.
  61. U. Moser and D. Schramm, “Multivariate dynamic time warping in automotive applications: A review,” Intelli- gent Data Analysis, vol. 23, no. 3, pp. 535–553, 2019.
  62. D. Bhatt, C. Patel, H. Talsania, J. Patel, R. Vaghela, S. Pandya, K. Modi, and H. Ghayvat, “Cnn variants for 164 | Murad & Alasadi computer vision: History, architecture, application, chal- lenges, and future scope,” Electronics, vol. 10, no. 20, 2021.
  63. M. A. Khan, M. Mittal, L. M. Goyal, and S. Roy, “A deep survey on supervised learning based human detection and activity classification methods,” Multimedia Tools and Applications, vol. 80, pp. 27867–27923, jul 2021.
  64. W. Chen, Q. Sun, X. Chen, G. Xie, H. Wu, and C. Xu, “Deep learning methods for heart sound classification: A systematic review,” Entropy, vol. 23, no. 6, 2021.
  65. B. Xie, H. Liu, R. Alghofaili, Y. Zhang, Y. Jiang, F. D. Lobo, C. Li, W. Li, H. Huang, M. Akdere, and C. Mousas, “A review of virtual reality skill training applications,” Frontiers in Virtual Reality, vol. 2, apr 2021.
  66. C. Lewis and F. C. Harris Jr, “An overview of virtual reality,” in Proceedings of 31st International Conference, vol. 88, pp. 71–81, nov 2022.
  67. A. Rizzo, S. Koenig, and B. Lange, “Clinical virtual reality: The state of the science,” in APA Handbook of neuropsychology, Volume 2: Neuroscience and neuro methods, vol. 2, pp. 473–491, 2023.
  68. N. B. Ibrahim, H. H. Zayed, and M. M. Selim, “Ad- vances, challenges, and opportunities in continuous sign language recognition,” Journal of Engineering and Ap- plied Sciences, vol. 15, no. 5, pp. 1205–1227, 2020.
  69. J. Wachs, H. Stern, Y. Edan, M. Gillam, C. Feied, M. Smith, and J. Handler, “A hand gesture sterile tool for browsing mri images in the or,” Journal of the American Medical Informatics Association, vol. 15, pp. 321–323, may 2008.
  70. Z. Hosseinaee, M. Le, K. Bell, and P. H. Reza, “To- wards non-contact photoacoustic imaging,” Photoacous- tics, vol. 20, dec 2020.
  71. Y. Zhang, S. Q. Xie, H. Wang, and Z. Zhang, “Data analytics in steady-state visual evoked potential-based brain-computer interface: A review,” IEEE Sensors Jour- nal, vol. 21, pp. 1124–1138, aug 2020.
  72. M. B. Shaikh and D. Chai, “Rgb-d data-based action recognition: A review,” Sensors, vol. 21, jun 2021.
  73. J. Wan, Y. Zhao, S. Zhou, I. Guyon, S. Escalera, and S. Z. Li, “Chalearn looking at people rgb-d isolated and continuous datasets for gesture recognition,” in Proceed- ings of the IEEE Conference on computer vision and pattern recognition workshops, pp. 56–64, 2016.
  74. S. e. a. Escalera, “Chalearn multi-modal gesture recog- nition 2013: grand challenge and workshop summary,” in Proceedings of the 15th ACM on International con- ference on multimodal interaction, pp. 365–368, dec 2013.
  75. P. e. a. Molchanov, “Online detection and classification of dynamic hand gestures with recurrent 3d convolu- tional neural network,” in Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pp. 4207–4215, 2016.
  76. V. e. a. Athitsos, “The american sign language lexicon video dataset,” in 2008 IEEE Computer Society Con- ference on Computer Vision and Pattern Recognition Workshops, pp. 1–8, IEEE, jun 2008.
  77. S. e. a. Yuan, “Bighand2.2m benchmark: Hand pose dataset and state-of-the-art analysis,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4866–4874, 2017.
  78. E. P. Costa, A. C. Lorena, A. C. Carvalho, and A. A. Freitas, “A review of performance evaluation measures for hierarchical classifiers,” in Evaluation methods for machine learning II: papers from the AAAI-2007 Work- shop, vol. AAAI Technical Report WS-07-05, pp. 1–6, 2007.