Cover
Vol. 18 No. 2 (2022)

Published: December 31, 2022

Pages: 110-116

Original Article

Human Activity and Gesture Recognition Based on WiFi Using Deep Convolutional Neural Networks

Sokienah K. Jawad 1 Musaab Alaziz
1Department of Computer Engineering, University of Basrah, Basrah, Iraq
History
  • Received: July 4, 2022
  • Revised: August 26, 2022
  • Accepted: August 26, 2022
  • Available Online: September 15, 2022
DOI

https://doi.org/10.37917/ijeee.18.2.14

Abstract

WiFi-based human activity and gesture recognition explore the interaction between the human hand or body movements and the reflected WiFi signals to identify various activities. This type of recognition has received much attention in recent years since it does not require wearing special sensors or installing cameras. This paper aims to investigate human activity and gesture recognition schemes that use Channel State Information (CSI) provided by WiFi devices. To achieve high accuracy in the measurement, deep learning models such as AlexNet, VGG 19, and SqueezeNet were used for classification and extracting features automatically. Firstly, outliers are removed from the amplitude of each CSI stream during the preprocessing stage by using the Hampel identifier algorithm. Next, the RGB images are created for each activity to feed as input to Deep Convolutional Neural Networks. After that, data augmentation is implemented to reduce the overfitting problems in deep learning models. Finally, the proposed method is evaluated on a publicly available dataset called WiAR, which contains 10 volunteers, each of whom executes 16 activities. The experiment results demonstrate that AlexNet, VGG19, and SqueezeNet all have high recognition accuracy of 99.17 %, 96.25%, and 100 %, respectively.

Keywords

References

  1. L. Chen, J. Hoey, C. D. Nugent, D. J. Cook, and Z. Yu, “Sensor-based activity recognition,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 42, no. 6, pp. 790–808, 2012.
  2. Y. Wang, K. Wu, and L. M. Ni, “Wifall: Device-free fall detection by wireless networks,” IEEE Transactions on Mobile Computing, vol. 16, no. 2, pp. 581–594, 2016.
  3. S. Gaglio, G. Lo Re, and M. Morana, “Human activity recognition process using 3-D posture data,” IEEE Transactions on Human-Machine Systems, vol. 45, no. 5, pp. 586–597, 2014.
  4. H. Abdelnasser, M. Youssef, and K. A. Harras, “Wigest: A ubiquitous wifi-based gesture recognition system,” in 2015 IEEE conference on computer communications (INFOCOM), 2015, pp. 1472–1480.
  5. M. S. Aljumaily and G. A. Al-Suhail, “Towards ubiquitous human gestures recognition using wireless networks,” International Journal of Pervasive Computing and Communications, 2017.
  6. H. Salim, M. Alaziz, and T. Abdalla, “Human Activity Recognition Using The Human Skeleton Provided by Kinect,” Iraqi Journal for Electrical and Electronic Engineering, vol. 17, no. 2, pp. 183–189, 2021, doi: 10.37917/ijeee.17.2.20.
  7. M. Alaziz, Z. Jia, R. Howard, X. Lin, and Y. Zhang, “In-bed body motion detection and classification system,” ACM Transactions on Sensor Networks (TOSN), vol. 16, no. 2, pp. 1–26, 2020.
  8. H. A. Salim, M. Alaziz, and T. Y. Abdalla, “People Tracking in a Smart Room Using Kinect Sensor,” in 2021 International Conference on Communication & Information Technology (ICICT), 2021, pp. 221–226.
  9. A. Bulling, U. Blanke, and B. Schiele, “A tutorial on human activity recognition using body-worn inertial sensors,” ACM Computing Surveys (CSUR), vol. 46, no. 3, pp. 1–33, 2014.
  10. A. Avci, S. Bosch, M. Marin-Perianu, R. Marin-Perianu, and P. Havinga, “Activity recognition using inertial sensing for healthcare, wellbeing and sports applications: A survey,” in 23th International conference on architecture of computing systems 2010, 2010, pp. 1–10.
  11. Y. Wang, J. Liu, Y. Chen, M. Gruteser, J. Yang, and H. Liu, “E-eyes: device-free location-oriented activity identification using fine-grained wifi signatures,” in Proceedings of the 20th annual international conference on Mobile computing and networking, 2014, pp. 617–628.
  12. W. Wang, A. X. Liu, M. Shahzad, K. Ling, and S. Lu, “Understanding and modeling of wifi signal based human activity recognition,” in Proceedings of the 21st annual international conference on mobile computing and networking, 2015, pp. 65–76.
  13. W. Wang, A. X. Liu, and M. Shahzad, “Gait recognition using wifi signals,” in Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2016, pp. 363–373.
  14. Q. Pu, S. Gupta, S. Gollakota, and S. Patel, “Whole- home gesture recognition using wireless signals,” in Proceedings of the 19th annual international conference on Mobile computing & networking, 2013, pp. 27–38.
  15. K. Ali, A. X. Liu, W. Wang, and M. Shahzad, “Keystroke recognition using wifi signals,” in Proceedings of the 21st annual international conference on mobile computing and networking, 2015, pp. 90–102.
  16. C. C. Lee and X. C. Huang, “Human Activity Detection via WiFi Signals Using Deep Neural Networks,” in 2018 IEEE/ACM International Conference on Utility and Cloud Computing Companion (UCC Companion), 2018, pp. 3– 4.
  17. Z. Yang, Z. Zhou, and Y. Liu, “From RSSI to CSI: Indoor localization via channel response,” ACM Computing Surveys (CSUR), vol. 46, no. 2, pp. 1–32, 2013.
  18. Y. Wang, X. Jiang, R. Cao, and X. Wang, “Robust indoor human activity recognition using wireless signals,” Sensors, vol. 15, no. 7, pp. 17195–17208, 2015.
  19. C. Wu, Z. Yang, Z. Zhou, X. Liu, Y. Liu, and J. Cao, “Non-invasive detection of moving and stationary human with WiFi,” IEEE Journal on Selected Areas in Communications, vol. 33, no. 11, pp. 2329–2342, 2015.
  20. S. Arshad et al., “Wi-chase: A WiFi based human activity recognition system for sensorless environments,” in 2017 IEEE 18th International Symposium on A World of Wireless, Mobile and Multimedia Networks (WoWMoM), 2017, pp. 1–6.
  21. M. A. A. Al-qaness, “Device-free human micro-activity recognition method using WiFi signals,” Geo-spatial Information Science, vol. 22, no. 2, pp. 128–137, 2019.
  22. A. Virmani and M. Shahzad, “Position and orientation agnostic gesture recognition using wifi,” in Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services, 2017, pp. 252–264. Jawad & Alaziz
  23. L. Guo et al., “Wiar: A public dataset for wifi-based activity recognition,” IEEE Access, vol. 7, pp. 154935– 154945, 2019.
  24. P. Fard Moshiri, R. Shahbazian, M. Nabati, and S. A. Ghorashi, “A CSI-Based Human Activity Recognition Using Deep Learning,” Sensors, vol. 21, no. 21, p. 7225, 2021.
  25. B. Kellogg, V. Talla, and S. Gollakota, “Bringing gesture recognition to all devices,” in 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14), 2014, pp. 303–316.
  26. D. Jiang, M. Li, and C. Xu, “Wigan: A wifi based gesture recognition system with gans,” Sensors, vol. 20, no. 17, p. 4757, 2020.
  27. C. Shi, J. Liu, H. Liu, and Y. Chen, “WiFi-enabled user authentication through deep learning in daily activities,” ACM Transactions on Internet of Things, vol. 2, no. 2, pp. 1–25, 2021.
  28. J. Xiao, K. Wu, Y. Yi, and L. M. Ni, “FIFS: Fine-grained indoor fingerprinting system,” in 2012 21st international conference on computer communications and networks (ICCCN), 2012, pp. 1–7.
  29. X. Wang, L. Gao, S. Mao, and S. Pandey, “DeepFi: Deep learning for indoor fingerprinting using channel state information,” in 2015 IEEE wireless communications and networking conference (WCNC), 2015, pp. 1666–1671.
  30. Y. Lu, S. Lv, and X. Wang, “Towards location independent gesture recognition with commodity wifi devices,” Electronics, vol. 8, no. 10, p. 1069, 2019.
  31. L. Davies and U. Gather, “The identification of multiple outliers,” Journal of the American Statistical Association, vol. 88, no. 423, pp. 782–792, 1993.
  32. H. Li, W. Yang, J. Wang, Y. Xu, and L. Huang, “WiFinger: Talk to your smart devices with finger-grained gesture,” in Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2016, pp. 250–261.
  33. L. Perez and J. Wang, “The effectiveness of data augmentation in image classification using deep learning,” arXiv preprint arXiv:1712.04621, 2017.
  34. H. R. Roth et al., “Anatomy-specific classification of medical images using deep convolutional nets,” in 2015 IEEE 12th international symposium on biomedical imaging (ISBI), 2015, pp. 101–104.
  35. D. Tiwari, M. Dixit, and K. Gupta, “Deep Multi-View Breast Cancer Detection: A Multi-View Concatenated Infrared Thermal Images Based Breast Cancer Detection System Using Deep Transfer Learning.,” Traitement du Signal, vol. 38, no. 6, 2021.
  36. M. Naz, J. H. Shah, M. A. Khan, M. Sharif, M. Raza, and R. Damaševičius, “From ECG signals to images: a transformation based approach for deep learning,” PeerJ Computer Science, vol. 7, p. e386, 2021.
  37. Q. Bu, G. Yang, J. Feng, and X. Ming, “Wi-fi based gesture recognition using deep transfer learning,” in 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation(SmartWorld/SCALCOM/UIC/ATC/CBDCom/ IOP/SCI), 2018, pp. 590–595.
  38. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in neural information processing systems, vol. 25, 2012.
  39. F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, “SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size,” arXiv Prepr. arXiv1602.07360, 2016.