Cover
Vol. 17 No. 1 (2021)

Published: June 30, 2021

Pages: 1-10

Review Article

Stochastic Local Search Algorithms for Feature Selection: A Review

Hayder Naser Khraibet Al-Behadili 1
1Department of Computer Science, Shatt Alarab University College, Basra, Iraq
History
  • Received: December 23, 2020
  • Revised: January 30, 2021
  • Accepted: January 31, 2021
  • Available Online: February 2, 2021
DOI

https://doi.org/10.37917/ijeee.17.1.1

Abstract

In today’s world, the data generated by many applications are increasing drastically, and finding an optimal subset of features from the data has become a crucial task. The main objective of this review is to analyze and comprehend different stochastic local search algorithms to find an optimal feature subset. Simulated annealing, tabu search, genetic programming, genetic algorithm, particle swarm optimization, artificial bee colony, grey wolf optimization, and bat algorithm, which have been used in feature selection, are discussed. This review also highlights the filter and wrapper approaches for feature selection. Furthermore, this review highlights the main components of stochastic local search algorithms, categorizes these algorithms in accordance with the type, and discusses the promising research directions for such algorithms in future research of feature selection.

Keywords

References

  1. T. R. Patil and S. Sherekar, “Performance Analysis of Naive Bayes and J48 Classification Algorithm for Data Classification,” Int. J. Comput. Sci. Appl., vol. 6, no. 2, pp. 256–261, 2013.
  2. H. N. K. Al-behadili, “Classification Algorithms for Determining Handwritten Digit,” Iraqi Journal for Electrical and Electronic Engineering, vol. 12, no. 1, pp. 96–102, 2016.
  3. N. K. Verma and A. Salour, “Feature selection,” in Studies in Systems, Decision and Control, 2020.
  4. M. M. Kabir, M. Shahjahan, and K. Murase, “A new hybrid ant colony optimization algorithm for feature selection,” Expert Syst. Appl., vol. 39, no. 3, pp. 3747– 3763, 2012.
  5. Y. B. W. Wah, N. Ibrahim, H. A. Hamid, S. Abdul- Rahman, and S. Fong, “Feature selection methods: Case of filter and wrapper approaches for maximising classification accuracy,” Pertanika J. Sci. Technol., vol. 26, no. 1, pp. 329–340, 2018.
  6. J. Li and H. Liu, “Challenges of Feature Selection for Big Data Analytics,” IEEE Intell. Syst., vol. 32, no. 2, pp. 9– 15, 2017.
  7. H. N. K. Al-behadili, K. R. Ku-Mahamud, and R. Sagban, “Hybrid Ant Colony Optimization and Iterated Local Search for Rules-Based Classification,” J. Theor. Appl. Inf. Technol., vol. 98, no. 04, pp. 657–671, 2020.
  8. A. M. Jabbar, K. R. Ku-Mahamud, and R. Sagban, “Modified ACS Centroid Memory for Data Clustering,” J. Comput. Sci., vol. 15, no. 10, pp. 1439–1449, 2019.
  9. A. Franzin and Thomas Stutzle, “Revisiting Simulated Annealing: a Component-Based Analysis,” Bruxelles, Belgium, 2018.
  10. H. H. Hoos and T. Stutzle, “Stochastic Local Search Algorithms: An Overview,” in Springer Handbook of Computational Intelligence, Springer Berlin Heidelberg, 2015, pp. 1085–1105.
  11. A. Albaghdadi and A. Ali, “An Optimized Complementary Filter For An Inertial Measurement Unit Contain MPU6050 Sensor,” Iraqi Journal for Electrical and Electronic Engineering, vol. 15, no. 2, pp. 71–77, 2019.
  12. T. Stützle and R. Ruiz, “Iterated Local Search,” in Handbook of Heuristics, Springer International Publishing, 2017, pp. 1–27.
  13. H. N. K. Al-behadili, K. R. Ku-Mahamud, and R. Sagban, “Ant colony optimization algorithm for rule- based classification: Issues and potential solutions,” J. Theor. Appl. Inf. Technol., vol. 96, no. 21, pp. 7139– 7150, 2018.
  14. H. N. K. Al-Behadili, R. Sagban, and K. R. Ku- Mahamud, “Adaptive Parameter Control Strategy for Ant-Miner Classification Algorithm,” Indones. J. Electr. Eng. Informatics, vol. 8, no. 1, pp. 149–162, 2020.
  15. A. M. Jabbar, K. R. Ku-Mahamud, and R. Sagban, “An improved ACS algorithm for data clustering,” Indones. J. Electr. Eng. Comput. Sci., vol. 17, no. 3, pp. 1506–1515, 2020.
  16. A. M. Jabbar and K. Ku-Mahamud, “Ant-based sorting and ACO-based clustering approaches: A review,” in In 2018 IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE), 2018, pp. 217–223.
  17. P. A. Estévez, M. Tesmer, C. A. Perez, and J. M. Zurada, “Normalized mutual information feature selection,” IEEE Trans. Neural Networks, vol. 20, no. 2, pp. 189–201, 2009.
  18. N. Kwak and C. H. Choi, “Input feature selection for classification problems,” IEEE Trans. Neural Networks, vol. 13, no. 1, pp. 143–159, 2002.
  19. K. Shin, D. Fernandes, and S. Miyazaki, “Consistency measures for feature selection: A formal definition, relative sensitivity comparison and a fast algorithm,” in IJCAI International Joint Conference on Artificial Intelligence, 2011, pp. 1491–1497.
  20. L. Yu and H. Liu, “Efficient feature selection via analysis of relevance and redundancy,” J. Mach. Learn. Res., vol. 5, pp. 1205–1224, 2004.
  21. D. Martens, M. Backer, R. Haesen, J. Vanthienen, M. Snoeck, and B. Baesens, “Classification with ant colony optimization,” IEEE Trans. Evol. Comput., vol. 11, no. 5, pp. 651–665, 2007.
  22. C. S. Yang, L. Y. Chuang, J. C. Li, and C. H. Yang, “Chaotic maps in binary particle swarm optimization for feature selection,” SMCia/08 - Proc. 2008 IEEE Conf. Soft Comput. Ind. Appl., pp. 107–112, 2008.
  23. B. Venkatesh and J. Anuradha, “A review of feature selection and its methods,” Cybern. Inf. Technol., vol. 19, no. 1, pp. 3–26, 2019. Hayder Al-Behadili | 9
  24. Y. Wah, N. Ibrahim, H. Hamid, S. Abdul-Rahman, and S. Fong, “Feature selection methods: case of filter and wrapper approaches for maximising classification accuracy,” Pertanika J. Sci. Technol., vol. 26, no. 1, pp. 329–340, 2018.
  25. I. A. Gheyas and L. S. Smith, “Feature subset selection in large dimensionality domains,” Pattern Recognit., vol. 43, no. 1, pp. 5–13, 2010.
  26. A. A. Abdoos, P. K. Mianaei, and M. R. Ghadikolaei, “Combined VMD-SVM based feature selection method for classification of power quality events,” Appl. Soft Comput. J., vol. 38, pp. 637–646, 2016.
  27. R. Kohavi and G. H. John, “Wrappers for feature subset selection,” Artif. Intell., vol. 97, no. 1–2, pp. 273–324, 1997.
  28. B. Chopard and M. Tomassini, “Simulated annealing,” in Natural Computing Series, 2018.
  29. S. Ledesma, G. Avia, and R. Sanchez, “Practical Considerations for Simulated Annealing Implementation,” in Simulated Annealing, 2008.
  30. M. Arokkiya and Jabasheela, “Performance Evaluation of Simulated Annealing based Feature Selection Approach on Depression Dataset,” J. Appl. Sci. Comput., vol. VI, no. I, pp. 3061–3068, 2019.
  31. J. C. W. Debuse and V. J. Rayward-Smith, “Feature Subset Selection within a Simulated Annealing Data Mining Algorithm,” J. Intell. Inf. Syst., vol. 9, no. 1, pp. 57–81, 1997.
  32. M. Laguna, “Tabu search,” in Handbook of Heuristics, 2018.
  33. H. Dhahri, I. Rahmany, A. Mahmood, E. Al Maghayreh, and W. Elkilani, “Tabu Search and Machine- Learning Classification of Benign and Malignant Proliferative Breast Lesions,” Biomed Res. Int., vol. 2020, pp. 1–10, 2020.
  34. Y. Wang, L. Li, J. Ni, and S. Huang, “Feature selection using tabu search with long-term memories and probabilistic neural networks,” Pattern Recognit. Lett., vol. 30, no. 7, pp. 661–670, 2009.
  35. H. Zhang and G. Sun, “Feature selection using tabu search method,” Pattern Recognit., vol. 35, no. 3, pp. 701–711, 2002.
  36. R. S. Parpinelli, G. F. Plichoski, R. S. Da Silva, and P. H. Narloch, “A review of techniques for online control of parameters in swarm intelligence and evolutionary computation algorithms,” Int. J. Bio-Inspired Comput., vol. 13, no. 1, pp. 1–20, 2019.
  37. A. Mukhopadhyay, U. Maulik, S. Bandyopadhyay, and C. A. C. Coello, “A survey of multiobjective evolutionary algorithms for data mining: Part I,” IEEE Trans. Evol. Comput., vol. 18, no. 1, pp. 4–19, 2014.
  38. G. Karafotias, M. Hoogendoorn, and A. E. Eiben, “Parameter Control in Evolutionary Algorithms: Trends and Challenges,” IEEE Trans. Evol. Comput., vol. 19, no. 2, pp. 167–187, 2015.
  39. B. Tran, B. Xue, and M. Zhang, “Genetic programming for feature construction and selection in classification on high-dimensional data,” Memetic Comput., vol. 8, no. 1, pp. 3–15, 2016.
  40. Q. U. Ain, B. Xue, H. Al-Sahaf, and M. Zhang, “Genetic Programming for Multiple Feature Construction in Skin Cancer Image Classification,” in International Conference Image and Vision Computing New Zealand, 2019, vol. 2019-Decem, pp. 732–745.
  41. A. Bakirtzis and S. Kazarlis, “Genetic algorithms,” in Advanced solutions in power systems: HVDC, facts, and artificial intelligence, 2016, pp. 845–902.
  42. S. Jadhav, H. He, and K. Jenkins, “Information gain directed genetic algorithm wrapper feature selection for credit rating,” Appl. Soft Comput. J., vol. 69, no. August, pp. 541–553, 2018.
  43. O. Babatunde, L. Armstrong, J. Leng, and D. Diepeveen, “A Genetic Algorithm-Based Feature Selection,” Int. J. Electron. Commun. Comput. Eng., vol. 5, no. 4, pp. 899–905, 2014.
  44. F. Ebadifard and S. M. Babamir, “A PSO‐based task scheduling algorithm improved using a load-balancing technique for the cloud computing environment,” Concurr. Comput. , vol. 30, no. 12, pp. 1–16, 2018.
  45. H. N. K. Al-behadili, K. R. Ku-Mahamud, and R. Sagban, “Rule pruning techniques in the ant-miner classification algorithm and its variants: A review,” in 2018 IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE), 2018, pp. 78–84.
  46. Y. Kabalci, S. Kockanat, and E. Kabalci, “A modified ABC algorithm approach for power system harmonic estimation problems,” Electr. Power Syst. Res., vol. 154, no. January, pp. 160–173, 2018.
  47. Y. Ren, T. Ye, M. Huang, and S. Feng, “Gray Wolf Optimization algorithm for multi-constraints second- order stochastic dominance portfolio optimization,” Algorithms, vol. 11, no. 5, 2018.
  48. T. M. Adis ALIHODZIC, “Bat Algorithm (BA) for Image Thresholding,” in Recent Researches in Telecommunications, Informatics, Electronics and Signal Processing, 2015, no. March 2015, pp. 364–369.
  49. D. Wang, D. Tan, and L. Liu, “Particle swarm optimization algorithm: an overview,” Soft Comput., vol. 22, no. 2, pp. 387–408, 2018.
  50. L. M. Abualigah, A. T. Khader, and E. S. Hanandeh, “A new feature selection method to improve the document clustering using particle swarm optimization algorithm,” J. Comput. Sci., vol. 25, no. September, pp. 456–466, 2018.
  51. O. S. Qasim and Z. Y. Algamal, “Feature selection using particle swarm optimization-based logistic regression model,” Chemom. Intell. Lab. Syst., vol. 182, pp. 41–46, 2018.
  52. S. Aslan, “A transition control mechanism for artificial bee colony (ABC) algorithm,” Comput. Intell. Neurosci., vol. 2019, pp. 1–24, 2019.
  53. A. D. Andrushia and A. T. Patricia, “Artificial bee colony optimization (ABC) for grape leaves disease detection,” Evol. Syst., vol. 11, no. 1, pp. 105–117, 2020.
  54. E. Hancer, B. Xue, D. Karaboga, and M. Zhang, “A binary ABC algorithm based on advanced similarity scheme for feature selection,” Appl. Soft Comput. J., vol. 36, pp. 334–348, 2015.
  55. X. S. Yang and X. He, “Bat algorithm: literature review and applications,” Int. J. Bio-Inspired Comput., vol. 5, Hayder Al-Behadili no. 3, p. 141, 2013.
  56. I. Gagnon, A. April, and A. Abran, “A critical analysis of the bat algorithm,” Eng. Reports, vol. 2, no. 8, pp. 1– 20, 2020.
  57. O. Ahmad Alomari, A. Tajudin Khader, M. Azmi Al- Betar, and L. Mohammad Abualigah, “Gene selection for cancer classification by combining minimum redundancy maximum relevancy and bat-inspired algorithm,” Int. J. Data Min. Bioinform., vol. 19, no. 1, pp. 32–51, 2017.
  58. R. Y. M. Nakamura, L. A. M. Pereira, K. A. Costa, D. Rodrigues, J. P. Papa, and X. S. Yang, “BBA: A binary bat algorithm for feature selection,” Brazilian Symp. Comput. Graph. Image Process., pp. 291–297, 2012.
  59. O. A. Alomari, A. T. Khader, M. A. Al-Betar, and L. M. Abualigah, “MRMR BA: A hybrid gene selection algorithm for cancer classification,” J. Theor. Appl. Inf. Technol., vol. 95, no. 12, pp. 2610–2618, 2017.
  60. S. Gupta and K. Deep, “A novel Random Walk Grey Wolf Optimizer,” Swarm Evol. Comput., vol. 44, pp. 101–112, 2019.
  61. W. Yamany, E. Emary, and A. E. Hassanien, “New rough set attribute reduction algorithm based on grey wolf optimization,” Adv. Intell. Syst. Comput., vol. 407, no. November, pp. 241–251, 2016.
  62. E. Emary, W. Yamany, A. E. Hassanien, and V. Snasel, “Multi-Objective Gray-Wolf Optimization for Attribute Reduction,” Procedia Comput. Sci., vol. 65, pp. 623–632, 2015.
  63. Y. Fang, Z. H. Liu, and F. Min, “A PSO algorithm for multi-objective cost-sensitive attribute reduction on numeric data with error ranges,” Soft Comput., vol. 21, no. 23, pp. 7173–7189, 2017.
  64. W. Li, X. Jia, L. Wang, and B. Zhou, “Multi-objective attribute reduction in three-way decision-theoretic rough set model,” Int. J. Approx. Reason., vol. 105, pp. 327– 341, 2019.
  65. G. Wang et al., “Multiple parameter control for ant colony optimization applied to feature selection problem,” Neural Comput. Appl., vol. 26, no. 7, pp. 1693–1708, 2015.
  66. A. E. Eiben, Z. Michalewicz, M. Schoenauer, and J. E. Smith, “Parameter control in evolutionary algorithms,” Stud. Comput. Intell., vol. 54, no. 2, pp. 19–46, 2007.
  67. T. Stutzle et al., “Parameter Adaptation in Ant Colony Optimization,” in Autonomous search, 2012, pp. 191– 215.
  68. R. Kamala and R. Thangaiah, “An improved hybrid feature selection method for huge dimensional datasets,” IAES Int. J. Artif. Intell., vol. 8, no. 1, pp. 77–86, 2019.