Skip to main content

Advertisement

SpringerLink
Log in

A new framework of multi-objective evolutionary algorithms for feature selection and multi-label classification of video data

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

There are few studies in the literature to address the multi-objective multi-label feature selection for the classification of video data using evolutionary algorithms. Selecting the most appropriate subset of features is a significant problem while maintaining/improving the accuracy of the prediction results. This study proposes a framework of parallel multi-objective Non-dominated Sorting Genetic Algorithms (NSGA-II) for exploring a Pareto set of non-dominated solutions. The subsets of non-dominated features are extracted and validated by multi-label classification techniques, Binary Relevance (BR), Classifier Chains (CC), Pruned Sets (PS), and Random k-Labelset (RAkEL). Base classifiers such as Support Vector Machines (SVM), J48-Decision Tree (J48), and Logistic Regression (LR) are performed in the classification phase of the algorithms. Comprehensive experiments are carried out with local feature descriptors extracted from two multi-label data sets, the well-known MIR-Flickr dataset and a Wireless Multimedia Sensor (WMS) dataset that we have generated from our video recordings. The prediction accuracy levels are improved by 6.36% and 25.7% for the MIR-Flickr and WMS datasets respectively while the number of features is significantly reduced. The results verify that the algorithms presented in this new framework outperform the state-of-the-art algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. http://ceng.metu.edu.tr/tr/node/3612

References

  1. Alpaydin E (2014) Introduction to machine learning. MIT Press, London

    MATH  Google Scholar 

  2. Miao J, Niu L (2016) A survey on feature selection. Proc Comput Sci 91:919–926

    Article  Google Scholar 

  3. Srivastava MS, Joshi MN, Gaur M (2014) A review paper on feature selection methodologies and their applications. IJCSNS 14(5):78

    Google Scholar 

  4. Bach FR (2008) Bolasso: model consistent lasso estimation through the bootstrap. In: Proceedings of the 25th international conference on machine learning, pp 33–40

  5. Cerri R, Basgalupp MP, Barros RC, de Carvalho AC (2019) Inducing hierarchical multi-label classification rules with genetic algorithms. Appl Soft Comput 77:584–604

    Article  Google Scholar 

  6. Gargiulo F, Silvestri S, Ciampi M, De Pietro G (2019) Deep neural network for hierarchical extreme multi-label text classification. Appl Soft Comput 79:125–138

    Article  Google Scholar 

  7. Deb K, Pratap A, Agarwal S, Meyarivan TAMT (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6(2):182–197

    Article  Google Scholar 

  8. Huiskes MJ, Lew MS (2008) The MIR flickr retrieval evaluation. In: Proceedings of the 1st ACM international conference on multimedia information retrieval, pp 39–43

  9. Costa AF, Traina AJM, Traina Jr C (2014) MFS-Map: efficient context and content combination to annotate images. In: Proceedings of the 29th annual ACM symposium on applied computing, pp 945–950

  10. Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3(Mar):1157–1182

    MATH  Google Scholar 

  11. Yin J, Tao T, Xu J (2015) A multi-label feature selection algorithm based on multi-objective optimization. In: 2015 international joint conference on neural networks (IJCNN), pp 1–7. IEEE, New York

  12. Zhang Y, Gong DW, Sun XY, Guo YN (2017) A PSO-based multi-objective multi-label feature selection method in classification. Sci Rep 7(1):1–12

    Article  Google Scholar 

  13. Dokeroglu T, Sevinc E, Kucukyilmaz T, Cosar A (2019) A survey on new generation metaheuristic algorithms. Comput Ind Eng 137:106040

    Article  Google Scholar 

  14. Vaishali R, Sasikala R, Ramasubbareddy S, Remya S, Nalluri S (2017) Genetic algorithm based feature selection and MOE Fuzzy classification algorithm on Pima Indians Diabetes dataset. In: 2017 international conference on computing networking and informatics (ICCNI), pp 1–5. IEEE, New York

  15. Vignolo LD, Milone DH, Scharcanski J (2013) Feature selection for face recognition based on multi-objective evolutionary wrappers. Expert Syst Appl 40(13):5077–5084

    Article  Google Scholar 

  16. Labani M, Moradi P, Jalili M, Yu X (2017) An evolutionary based multi-objective filter approach for feature selection. In: 2017 world congress on computing and communication technologies (WCCCT), pp 151–154. IEEE, New York

  17. Zhang P, Gao W, Liu G (2018) Feature selection considering weighted relevancy. Appl Intell 48(12):4615–4625

    Article  Google Scholar 

  18. Deniz A, Kiziloz HE, Dokeroglu T, Cosar A (2017) Robust multiobjective evolutionary feature subset selection algorithm for binary classification using machine learning techniques. Neurocomputing 241:128–146

    Article  Google Scholar 

  19. Saroj J (2014) Multi-objective genetic algorithm approach to feature subset optimization. In: Proceeding of the IEEE international advance computing conference (IACC), pp 544–548

  20. Hamdani TM, Won JM, Alimi AM, Karray F (2007) Multi-objective feature selection with NSGA II. In: International conference on adaptive and natural computing algorithms, pp 240–247. Springer, Berlin

  21. Khan MA, Ekbal A, Menca EL, Furnkranz J (2017) Multi-objective optimisation-based feature selection for multi-label classification. In: International conference on applications of natural language to information systems, pp 38–41. Springer, New York

  22. Li S, Wu H, Wan D, Zhu J (2011) An effective feature selection method for hyperspectral image classification based on genetic algorithm and support vector machine. Knowl-Based Syst 24(1):40–48

    Article  Google Scholar 

  23. Gaspar-Cunha A (2010) Feature selection using multi-objective evolutionary algorithms: application to cardiac SPECT diagnosis. In: Advances in bioinformatics, pp 85-92. Springer, Berlin

  24. Xue B, Zhang M, Browne WN (2012) Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans Cybern 43(6):1656–1671

    Article  Google Scholar 

  25. Zhang Y, Gong DW, Cheng J (2015) Multi-objective particle swarm optimization approach for cost-based feature selection in classification. IEEE/ACM Trans Comput Biol Bioinform 14(1):64–75

    Article  Google Scholar 

  26. Tangherloni A, Spolaor S, Cazzaniga P, Besozzi D, Rundo L, Mauri G, Nobile MS (2019) Biochemical parameter estimation vs. benchmark functions: a comparative study of optimization performance and representation design. Appl Soft Comput 81:105494

    Article  Google Scholar 

  27. Nalluri MR, Kannan K, Gao XZ, Roy DS (2019) Multiobjective hybrid monarch butterfly optimization for imbalanced disease classification problem. Int J Mach Learn Cybern 1–29

  28. Rundo L, Tangherloni A, Nobile MS, Militello C, Besozzi D, Mauri G, Cazzaniga P (2019) MedGA: a novel evolutionary method for image enhancement in medical imaging systems. Expert Syst Appl 119:387–399

    Article  Google Scholar 

  29. Rundo L, Tangherloni A, Cazzaniga P, Nobile MS, Russo G, Gilardi MC, Militello C (2019) A novel framework for MR image segmentation and quantification by using MedGA. Comput Methods Programs Biomed 176:159–172

    Article  Google Scholar 

  30. Thabtah FA, Cowling P, Peng Y (2004) MMAC: a new multi-class, multi-label associative classification approach. In: Fourth IEEE international conference on data mining (ICDM’04), pp 217–224. IEEE, New York

  31. Charte F, del Jesus MJ, Rivera AJ (2016) Multilabel classification: problem analysis, metrics and techniques. Springer, New York

    Google Scholar 

  32. Read J, Pfahringer B, Holmes G (2008) Multi-label classification using ensembles of pruned sets. In: 2008 eighth IEEE international conference on data mining, pp 995–1000. IEEE, New York

  33. Tsoumakas G, Vlahavas I (2007) Random k-labelsets: an ensemble method for multilabel classification. In: European conference on machine learning, pp 406–417. Springer, Berlin

  34. Lobato FS, Steffen V (2017) Multi-objective optimization problem. In: Multi-objective optimization problems, pp 9–23. Springer, Cham

  35. Zhou A, Qu BY, Li H, Zhao SZ, Suganthan PN, Zhang Q (2011) Multiobjective evolutionary algorithms: a survey of the state of the art. Swarm Evol Comput 1(1):32–49

    Article  Google Scholar 

  36. Stadler W (1979) A survey of multicriteria optimization or the vector maximum problem, part I: 1776–1960. J Optim Theory Appl 29(1):1–52

    Article  MathSciNet  Google Scholar 

  37. Miettinen K (2012) Nonlinear multiobjective optimization, vol 12. Springer, New York

    MATH  Google Scholar 

  38. Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85(3):333

    Article  MathSciNet  Google Scholar 

  39. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  40. Zeng ZQ, Yu HB, Xu HR, Xie YQ, Gao J (2008) Fast training support vector machines using parallel sequential minimal optimization. In: 2008 3rd international conference on intelligent system and knowledge engineering, vol 1, pp 997–1001. IEEE, New York

  41. Bishop CM (2006) Pattern recognition and machine learning. Springer, New York

    MATH  Google Scholar 

  42. Bhargava N, Sharma G, Bhargava R, Mathuria M (2013) Decision tree analysis on j48 algorithm for data mining. In: Proceedings of international journal of advanced research in computer science and software engineering, vol 3(6)

  43. Kaur G, Chhabra A (2014) Improved J48 classification algorithm for the prediction of diabetes. Int J Comput Appl 98(22)

  44. Dokeroglu T, Sevinc E (2019) Evolutionary parallel extreme learning machines for the data classification problem. Comput Ind Eng 130:237–249

    Article  Google Scholar 

  45. Cantu-Paz E (1998) A survey of parallel genetic algorithms. Calc Paralleles Reseaux Syst Repartis 10(2):141–171

    Google Scholar 

  46. Hadka D (2014) MOEA framework user guide

  47. Costa AF, Humpire-Mamani G, Traina AJM (2012) An efficient algorithm for fractal analysis of textures. In: 2012 25th SIBGRAPI conference on graphics, patterns and images, pp 39–46. IEEE, New York

  48. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  49. Bradski G, Kaehler A (2008) Learning OpenCV: computer vision with the OpenCV library. O’Reilly Media, Inc., Massachusetts

    Google Scholar 

  50. Read J, Reutemann P, Pfahringer B, Holmes G (2016) Meka: a multi-label/multi-target extension to weka. J Mach Learn Res 17(1):667–671

    MathSciNet  MATH  Google Scholar 

  51. Tan Q, Yu G, Domeniconi C, Wang J, Zhang Z (2018) Incomplete multi-view weak-label learning. In: IJCAI, pp 2703–2709

  52. Pearson K (1901) LIII. On lines and planes of closest fit to systems of points in space. Lond Edinb Dublin Philos Mag J Sci 2(11):559–572

    Article  Google Scholar 

  53. Jolliffe IT, Cadima J (2016) Principal component analysis: a review and recent developments. Philos Trans R Soc A Math Phys Eng Sci 374(2065):20150202

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This study is supported in part by NU Faculty development competitive research grants program, Nazarbayev University, Grant Number-110119FD4543 and in part by a research grant from TUBITAK (The Scientific and Technological Research Council of Turkey) with the Grant no. 114R082.

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Middle East Technical University, Ankara, Turkey

    Gizem Nur Karagoz & Adnan Yazici

  2. Department of Computer Science, Nazarbayev University, Nur-Sultan, Kazakhstan

    Adnan Yazici

  3. Department of Computer Engineering, TED University, Ankara, Turkey

    Tansel Dokeroglu

  4. Department of Computer Engineering, Ankara Bilim University, Ankara, Turkey

    Ahmet Cosar

Authors
  1. Gizem Nur Karagoz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adnan Yazici
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tansel Dokeroglu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ahmet Cosar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tansel Dokeroglu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karagoz, G.N., Yazici, A., Dokeroglu, T. et al. A new framework of multi-objective evolutionary algorithms for feature selection and multi-label classification of video data. Int. J. Mach. Learn. & Cyber. 12, 53–71 (2021). https://doi.org/10.1007/s13042-020-01156-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-020-01156-w

Keywords

Search

Navigation