Cover
Vol. 22 No. 1 (2026)

Published: June 15, 2026

Pages: 388-400

Original Article

A Hybrid Approach Using Vector Field Histogram and Deep Reinforcement Learning for Dynamic Path Planning

Maysam Hameed Qasim 1 Salah Al-Darraji 1
1 Department of Computer Science, College of Computer Science & Information Technology, University of Basrah, Basrah, Iraq
History
  • Received: June 29, 2024
  • Revised: July 28, 2024
  • Accepted: August 28, 2024
  • Available Online: June 15, 2026
DOI

https://doi.org/10.37917/ijeee.22.1.34

Abstract

Autonomous mobile robots (AMRs) are becoming increasingly important in different domains such as healthcare, warehouse automation and household duties, but still encounter problems when it comes to moving around unfamiliar and dynamic environments. This study proposes an advanced robotic navigation system which combines the Soft Actor-Critic (SAC) approach and Vector Field Histogram (VFH) for path planning and avoidance obstacles in completely unknown environments. This system leverages the strengths of deep reinforcement learning and real-time obstacle detection to achieve robust and efficient navigation in certain scenarios. The SAC strategy optimizes robot navigation using policy networks and Q-networks, while the VFH method addresses obstacle avoidance by sensor data processing and dynamically adjusting the robot’s angular velocity to avoid collision. For testing and implementing this system, Gazebo simulation and Robot Operating System (ROS) are used. Experimental results demonstrated that the proposed method outperformed the standard technique and achieved a high success rate in path planning and obstacles avoidance.

Keywords

References

  1. S. Al-Ansarry and S. Al-Darraji, “Hybrid rrt-a*: An improved path planning method for an autonomous mobile robots.,” Iraqi Journal for Electrical & Electronic Engineering, vol. 17, no. 1, 2021.
  2. Z. Haruna, M. B. Mu’azu, A. Umar, and G. O. Ufuoma, “Path planning algorithms for mobile robots: A survey,” in Motion Planning for Dynamic Agents, IntechOpen, 2023.
  3. C. Liu, S. Xie, X. Sui, Y. Huang, X. Ma, N. Guo, and F. Yang, “Prm-d* method for mobile robot path planning,” Sensors, vol. 23, no. 7, p. 3512, 2023.
  4. J. A. Abdulsaheb and D. J. Kadhim, “Classical and heuristic approaches for mobile robot path planning: A survey,” Robotics, vol. 12, no. 4, p. 93, 2023.
  5. L. Bing, “Path planning of mobile robot based on improved artificial potential field method,” International Journal of Engineering Continuity, vol. 2, no. 2, pp. 55– 61, 2023.
  6. N. Abdalmanan, K. Kamarudin, M. A. A. Bakar, M. H. F. Rahiman, A. Zakaria, S. M. Mamduh, and L. M. Kamarudin, “2d lidar based reinforcement learning for multi-target path planning in unknown environment,” IEEE Access, vol. 11, pp. 35541–35555, 2023.
  7. A. Marashian and A. Razminia, “Mobile robot’s pathplanning and path-tracking in static and dynamic environments: Dynamic programming approach,” Robotics and Autonomous Systems, vol. 172, p. 104592, 2024.
  8. Y. Sung, J. Das, and P. Tokekar, “Decision-theoretic approaches for robotic environmental monitoring–a survey,” arXiv preprint arXiv:2308.02698, 2023.
  9. R. Raj and A. Kos, “An optimized energy and time constraints-based path planning for the navigation of mobile robots using an intelligent particle swarm optimization technique,” Applied Sciences, vol. 13, no. 17, p. 9667, 2023.
  10. L. Yang, P. Li, S. Qian, H. Quan, J. Miao, M. Liu, Y. Hu, and E. Memetimin, “Path planning technique for mobile robots: A review,” Machines, vol. 11, no. 10, p. 980, 2023.
  11. Y. Long, S. Liu, D. Qiu, C. Li, X. Guo, B. Shi, and M. S. AbouOmar, “Local path planning with multiple constraints for usv based on improved bacterial foraging optimization algorithm,” Journal of Marine Science and Engineering, vol. 11, no. 3, p. 489, 2023.
  12. P. G. Luan and N. T. Thinh, “Hybrid genetic algorithm based smooth global-path planning for a mobile robot,” Mechanics Based Design of Structures and Machines, vol. 51, no. 3, pp. 1758–1774, 2023.
  13. L. Liu, X. Wang, X. Yang, H. Liu, J. Li, and P. Wang, “Path planning techniques for mobile robots: Review and prospect,” Expert Systems with Applications, vol. 227, p. 120254, 2023.
  14. F. Li, Y.-C. Kim, and B. Xu, “Non-standard map robot path planning approach based on ant colony algorithms,” Sensors, vol. 23, no. 17, p. 7502, 2023.
  15. A. Jabini and E. A. Johnson, “A deep reinforcement learning approach to sensor placement under uncertainty,” IFAC-PapersOnLine, vol. 55, no. 27, pp. 178– 183, 2022.
  16. L. Chen, Z. Jiang, L. Cheng, A. C. Knoll, and M. Zhou, “Deep reinforcement learning based trajectory planning under uncertain constraints,” Frontiers in Neurorobotics, vol. 16, p. 883562, 2022.
  17. H. Liu, Y. Shen, S. Yu, Z. Gao, and T. Wu, “Deep reinforcement learning for mobile robot path planning,” arXiv preprint arXiv:2404.06974, 2024.
  18. K. Kasaura, S. Miura, T. Kozuno, R. Yonetani, K. Hoshino, and Y. Hosoe, “Benchmarking actor-critic deep reinforcement learning algorithms for robotics control with action constraints,” IEEE Robotics and Automation Letters, vol. 8, no. 8, pp. 4449–4456, 2023.
  19. J. Borenstein, Y. Koren, et al., “The vector field histogram-fast obstacle avoidance for mobile robots,” IEEE transactions on robotics and automation, vol. 7, no. 3, pp. 278–288, 1991.
  20. C. Chronis, G. Anagnostopoulos, E. Politi, G. Dimitrakopoulos, and I. Varlamis, “Dynamic navigation in unconstrained environments using reinforcement learning algorithms,” IEEE Access, 2023.
  21. R. Chai, H. Niu, J. Carrasco, F. Arvin, H. Yin, and B. Lennox, “Design and experimental validation of deep reinforcement learning-based fast trajectory planning and control for mobile robot in unknown environment,” IEEE Transactions on Neural Networks and Learning Systems, vol. 35, no. 4, pp. 5778–5792, 2022.
  22. R. Huang, C. Qin, J. L. Li, and X. Lan, “Path planning of mobile robot in unknown dynamic continuous environment using reward-modified deep q-network,” Optimal Control Applications and Methods, vol. 44, no. 3, pp. 1570–1587, 2023.
  23. A. N. Kumaar and S. Kochuvila, “Mobile service robot path planning using deep reinforcement learning,” IEEE Access, 2023.
  24. L. Zheng, Y. Wang, R. Yang, S. Wu, R. Guo, and E. Dong, “An efficiently convergent deep reinforcement learning-based trajectory planning method for manipulators in dynamic environments,” Journal of Intelligent & Robotic Systems, vol. 107, no. 4, p. 50, 2023.
  25. L. Yang, J. Bi, and H. Yuan, “Dynamic path planning for mobile robots with deep reinforcement learning,” IFACPapersOnLine, vol. 55, no. 11, pp. 19–24, 2022.
  26. H. Gong, P. Wang, C. Ni, and N. Cheng, “Efficient path planning for mobile robot based on deep deterministic policy gradient,” Sensors, vol. 22, no. 9, p. 3579, 2022.
  27. M. Kobayashi, H. Zushii, T. Nakamura, and N. Motoi, “Local path planning: Dynamic window approach with q-learning considering congestion environments for mobile robot,” IEEE Access, 2023.
  28. Y. Tan, Y. Lin, T. Liu, and H. Min, “Pl-td3: A dynamic path planning algorithm of mobile robot,” in 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 3040–3045, IEEE, 2022.
  29. Y. Cao, K. Ni, T. Kawaguchi, and S. Hashimoto, “Path following for autonomous mobile robots with deep reinforcement learning,” Sensors, vol. 24, no. 2, 2024.
  30. Z. He, L. Dong, C. Song, and C. Sun, “Multiagent soft actor-critic based hybrid motion planner for mobile robots,” IEEE transactions on neural networks and learning systems, vol. 34, no. 12, pp. 10980–10992, 2022.
  31. K. Katona, H. A. Neamah, and P. Korondi, “Obstacle avoidance and path planning methods for autonomous navigation of mobile robot,” Sensors, vol. 24, no. 11, p. 3573, 2024.
  32. L. Tai, G. Paolo, and M. Liu, “Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation,” in 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 31–36, IEEE, 2017.
  33. J. C. de Jesus, V. A. Kich, A. H. Kolling, R. B. Grando, M. A. d. S. L. Cuadros, and D. F. T. Gamarra, “Soft actor-critic for navigation of mobile robots,” Journal of Intelligent & Robotic Systems, vol. 102, no. 2, p. 31, 2021.
  34. J. C. Jesus, J. A. Bottega, M. A. Cuadros, and D. F. Gamarra, “Deep deterministic policy gradient for navigation of mobile robots in simulated environments,” in 2019 19th International Conference on Advanced Robotics (ICAR), pp. 362–367, IEEE, 2019.