Skip to main content
SpringerLink
Log in

Multicriteria Models for Learning Ordinal Data: A Literature Review

  • Chapter
Artificial Intelligence, Evolutionary Computing and Metaheuristics

Part of the book series: Studies in Computational Intelligence ((SCI,volume 427))

Abstract

Operations Research (OR) and Artificial Intelligence (AI) disciplines have been playing major roles on the design of new intelligent systems. Recently, different contributions from both fields have been made on the models design for problems with multi-criteria. The credit scoring problem is an example of that. In this problem, one evaluates how unlikely a client will default with his payments. Client profiles are evaluated, being their results expressed in terms of an ordinal score scale (Excelent ≻ Good ≻ Fair ≻ Poor). Intelligent systems have then to take in consideration different criteria such as payment history, mortgages, wages among others in order to commit their outcome. To achieve this goal, researchers have been delving models capable to render these multiple criteria encompassed on ordinal data.

The literature presents a myriad of different methods either on OR or AI fields for the multi-criteria models. However, a description of ordinal data methods on these two major disciplines and their relations has not been thoroughly conducted yet. It is key for further research to identify the developments made and the present state of the existing methods. It is also important to ascertain current achievements and what the requirements are to attain intelligent systems capable to capture relationships from data. In this chapter one will describe techniques presented for over more than five decades on OR and AI disciplines applied to multi-criteria ordinal problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abraham, A., Abraham, A., Falcn, R., Bello, R.: Rough Set Theory: A True Landmark in Data Analysis. Springer Publishing Company, Incorporated (2009)

    Google Scholar 

  2. Angilella, S., Greco, S., Matarazzo, B.: Non-additive robust ordinal regression: A multiple criteria decision model based on the choquet integral. European Journal of Operational Research 201(1), 277–288 (2010), doi:10.1016/j.ejor.2009.02.023

    MATH  Google Scholar 

  3. Baccianella, S., Esuli, A., Sebastiani, F.: Evaluation measures for ordinal regression. In: Proceedings of the Ninth International Conference on Intelligent Systems Design and Applications, pp. 283–287 (2009)

    Google Scholar 

  4. Baccianella, S., Esuli, A., Sebastiani, F.: Feature selection for ordinal regression. In: Proceedings of the 2010 ACM Symposium on Applied Computing, SAC 2010, pp. 1748–1754. ACM Press, New York (2010), doi:10.1145/1774088.1774461

    Google Scholar 

  5. Baccianella, S., Esuli, A., Sebastiani, F.: Selecting features for ordinal text classification. In: IIR, pp. 13–14 (2010)

    Google Scholar 

  6. Belacel, N.: Multicriteria assignment method PROAFTN: Methodology and medical application. European Journal of Operational Research 125(1), 175–183 (2000), doi:10.1016/S0377-2217(99)00192-7

    MATH  Google Scholar 

  7. Ben-David, A.: A lot of randomness is hiding in accuracy. Engineering Applications of Artificial Intelligence 20(7), 875–885 (2007), doi:10.1016/j.engappai.2007.01.001

    Google Scholar 

  8. Beuthe, M., Scannella, G.: Comparative analysis of UTA multicriteria methods. European Journal of Operational Research 130(2), 246–262 (2001), doi:10.1016/S0377-2217(00)00042-4

    MATH  Google Scholar 

  9. Bishop, C.M.: Pattern Recognition and Machine Learning (Information Science and Statistics), 1st edn. (2006); Corr., 2nd printing edn. Springer (2007)

    Google Scholar 

  10. Blaszczynski, J., Greco, S., Slowinski, R., Szelg, M.: Monotonic variable consistency rough set approaches. International Journal of Approximate Reasoning 50(7), 979–999 (2009), doi:10.1016/j.ijar.2009.02.011; Special Section on Graphical Models and Information Retrieval

    MathSciNet  MATH  Google Scholar 

  11. Bouveret, S., Lemaître, M.: Computing leximin-optimal solutions in constraint networks. Artificial Intelligence 173(2), 343–364 (2009), doi:10.1016/j.artint.2008.10.010

    MathSciNet  MATH  Google Scholar 

  12. Bradley, A.P.: The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern Recognition 30(7), 1145–1159 (1997), doi:10.1016/S0031-3203(96)00142-2

    Google Scholar 

  13. Branke, J., Deb, K., Miettinen, K., Slowinski, R. (eds.): Multiobjective Optimization: Interactive and Evolutionary Approaches. Springer, Heidelberg (2008)

    MATH  Google Scholar 

  14. Brefeld, U., Geibel, P., Wysotzki, F.: Support Vector Machines with Example Dependent Costs. In: Lavrač, N., Gamberger, D., Todorovski, L., Blockeel, H. (eds.) ECML 2003. LNCS (LNAI), vol. 2837, pp. 23–34. Springer, Heidelberg (2003)

    Google Scholar 

  15. Breiman, L., Friedman, J., Stone, C.J., Olshen, R.: Classification and Regression Trees. Chapman & Hall (1998)

    Google Scholar 

  16. Cao-Van, K., De Baets, B.: Consistent Representation of Rankings. In: de Swart, H., Orłowska, E., Schmidt, G., Roubens, M. (eds.) TARSKI. LNCS, vol. 2929, pp. 107–123. Springer, Heidelberg (2003)

    Google Scholar 

  17. Cardoso, J.S., Cardoso, M.J.: Towards an intelligent medical system for the aesthetic evaluation of breast cancer conservative treatment. Artificial Intelligence in Medicine 40, 115–126 (2007)

    Google Scholar 

  18. Cardoso, J.S., da Costa, J.F.P.: Learning to classify ordinal data: the data replication method. Journal of Machine Learning Research 8, 1393–1429 (2007)

    MATH  Google Scholar 

  19. Cardoso, J.S., Sousa, R.: Classification models with global constraints for ordinal data. In: Proceedings of The Ninth International Conference on Machine Learning and Applications, ICMLA (2010)

    Google Scholar 

  20. Cardoso, J.S., Sousa, R.: Measuring the Performance of Ordinal Classification. International Journal of Pattern Recognition and Artificial Intelligence (2011)

    Google Scholar 

  21. Cheng, J., Wang, Z., Pollastri, G.: A neural network approach to ordinal regression. In: IEEE International Joint Conference on Neural Networks, IJCNN 2008 (IEEE World Congress on Computational Intelligence), pp. 1279–1284 (2008), doi:10.1109/IJCNN.2008.4633963

    Google Scholar 

  22. Chu, W., Ghahramani, Z.: Gaussian Processes for Ordinal Regression. J. Mach. Learn. Res. 6, 1019–1041 (2005)

    MathSciNet  MATH  Google Scholar 

  23. Chu, W., Ghahramani, Z.: Preference learning with Gaussian processes. In: ICML 2005: Proceedings of the 22nd International Conference on Machine Learning, pp. 137–144. ACM, New York (2005), doi:10.1145/1102351.1102369

    Google Scholar 

  24. Chu, W., Sindhwani, V., Ghahramani, Z., Keerthi, S.S.: Relational Learning with Gaussian Processes. In: Schölkopf, B., Platt, J., Hoffman, T. (eds.) Advances in Neural Information Processing Systems 19, pp. 289–296. MIT Press, Cambridge (2007)

    Google Scholar 

  25. Cossock, D., Zhang, T.: Subset Ranking Using Regression. In: Lugosi, G., Simon, H.U. (eds.) COLT 2006. LNCS (LNAI), vol. 4005, pp. 605–619. Springer, Heidelberg (2006), doi:10.1007/11776420_44

    Google Scholar 

  26. da Costa, J.F.P., Sousa, R., Cardoso, J.S.: An all-at-once unimodal svm approach for ordinal classification. In: Proceedings of The Ninth International Conference on Machine Learning and Applications, ICMLA (2010)

    Google Scholar 

  27. da Costa, J.F.P., Alonso, H., Cardoso, J.S.: The unimodal model for the classification of ordinal data. Neural Networks 21(1), 78–91 (2008)

    Google Scholar 

  28. Delannay, N., Verleysen, M.: Collaborative filtering with interlaced generalized linear models. Neurocomputing 71(7-9), 1300–1310 (2008), doi http://dx.doi.org/10.1016/j.neucom.2007.12.021

    Google Scholar 

  29. Dembczyński, K., Greco, S., Kotłowski, W., Słowiński, R.: Statistical Model for Rough Set Approach to Multicriteria Classification. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) PKDD 2007. LNCS (LNAI), vol. 4702, pp. 164–175. Springer, Heidelberg (2007)

    Google Scholar 

  30. Dombi, J., Zsiros, A.: Learning multicriteria classification models from examples Decision rules in continuous space. European Journal of Operational Research 160(3), 663–675 (2005), doi:10.1016/j.ejor.2003.10.006; Decision Analysis and Artificial Intelligence

    MATH  Google Scholar 

  31. Doumpos, M., Kosmidou, K., Baourakis, G., Zopounidis, C.: Credit risk assessment using a multicriteria hierarchical discrimination approach: A comparative analysis. European Journal of Operational Research 138(2), 392–412 (2002), doi:10.1016/S0377-2217(01)00254-5

    MathSciNet  MATH  Google Scholar 

  32. Doumpos, M., Marinakis, Y., Marinaki, M., Zopounidis, C.: An evolutionary approach to construction of outranking models for multicriteria classification: The case of the electretri method. European Journal of Operational Research 199(2), 496–505 (2009), doi:10.1016/j.ejor.2008.11.035

    MATH  Google Scholar 

  33. Doumpos, M., Pasiouras, F.: Developing and testing models for replicating credit ratings: A multicriteria approach. Computational Economics 25, 327–341 (2005)

    MATH  Google Scholar 

  34. Doumpos, M., Salappa, A.: Feature selection algorithms in classification problems: an experimental evaluation. In: Proceedings of the 4th WSEAS International Conference on Artificial Intelligence, Knowledge Engineering Data Bases, pp. 36:1–36:6. World Scientific and Engineering Academy and Society, WSEAS (2005)

    Google Scholar 

  35. Doumpos, M., Zopounidis, C.: Multicriteria Decision Aid Classification Methods. Kluwer Academic Publishers, Dordrecht (2002)

    MATH  Google Scholar 

  36. Doumpos, M., Zopounidis, C.: A multicriteria classification approach based on pairwise comparisons. European Journal of Operational Research 158(2), 378–389 (2004), doi:10.1016/j.ejor.2003.06.011; Methodological Foundations of Multi-Criteria Decision Making

    MathSciNet  MATH  Google Scholar 

  37. Doumpos, M., Zopounidis, C.: A multicriteria decision support system for bank rating. Decision Support Systems 50(1), 55–63 (2010), doi:10.1016/j.dss.2010.07.002

    Google Scholar 

  38. Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley-Interscience (2001)

    Google Scholar 

  39. Duivesteijn, W., Feelders, A.: Nearest Neighbour Classification with Monotonicity Constraints. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008, Part I. LNCS (LNAI), vol. 5211, pp. 301–316. Springer, Heidelberg (2008)

    Google Scholar 

  40. Durbach, I.N.: The use of the SMAA acceptability index in descriptive decision analysis. European Journal of Operational Research 193(3), 1229–1237 (2009), doi:10.1016/j.ejor.2008.05.021

    Google Scholar 

  41. Ehrgott, M.: Multicriteria optimization. Lecture Notes in Economics and Mathematical Systems. Springer (2000)

    Google Scholar 

  42. Fernandez, E., Navarro, J., Bernal, S.: Multicriteria sorting using a valued indifference relation under a preference disaggregation paradigm. European Journal of Operational Research 198(2), 602–609 (2009), doi:10.1016/j.ejor.2008.09.020

    MATH  Google Scholar 

  43. Figueira, J., Greco, S., Ehrogott, M., Brans, J.P., Mareschal, B.: Promethee methods. In: Multiple Criteria Decision Analysis: State of the Art Surveys. International Series in Operations Research & Management Science, vol. 78, pp. 163–186. Springer, New York (2005), doi:10.1007/0-387-23081-5_5

    Google Scholar 

  44. Figueira, J., Greco, S., Ehrogott, M., Siskos, Y., Grigoroudis, E., Matsatsinis, N.: Uta methods. In: Hillier, F.S. (ed.) Multiple Criteria Decision Analysis: State of the Art Surveys. International Series in Operations Research & Management Science, vol. 78, pp. 297–334. Springer, New York (2005), doi:10.1007/0-387-23081-5_8

    Google Scholar 

  45. Fisher, R.A.: The Use of Multiple Measurements in Taxonomic Problems. Annals of Eugenics 7, 179–188 (1936)

    Google Scholar 

  46. Frank, E., Hall, M.: A Simple Approach to Ordinal Classification. In: Flach, P.A., De Raedt, L. (eds.) ECML 2001. LNCS (LNAI), vol. 2167, pp. 145–156. Springer, Heidelberg (2001)

    Google Scholar 

  47. Frasch, J.V., Lodwich, A., Shafait, F., Breuel, T.M.: A Bayes-true data generator for evaluation of supervised and unsupervised learning methods. Pattern Recognition Letters 32(11), 1523–1531 (2011), doi:10.1016/j.patrec.2011.04.010

    Google Scholar 

  48. Fürnkranz, J., Hüllermeier, E.: Pairwise Preference Learning and Ranking. In: Proceedings of the 14th European Conference on Machine Learning, pp. 145–156. Springer (2003)

    Google Scholar 

  49. Fürnkranz, J., Hüllermeier, E.: Pairwise preference learning and ranking. Tech. rep., Austrian Research Institute for Artificial Intelligence, Wien, Austria (2003)

    Google Scholar 

  50. Greco, S., Inuiguchi, M., Slowinski, R.: Fuzzy rough sets and multiple-premise gradual decision rules. International Journal of Approximate Reasoning 41(2), 179–211 (2006), doi:10.1016/j.ijar.2005.06.014; Advances in Fuzzy Sets and Rough Sets

    MathSciNet  MATH  Google Scholar 

  51. Greco, S., Matarazzo, B., Slowinski, R.: Rough sets theory for multicriteria decision analysis. European Journal of Operational Research 129(1), 1–47 (2001), doi:10.1016/S0377-2217(00)00167-3

    MathSciNet  MATH  Google Scholar 

  52. Greco, S., Mousseau, V., Slowinski, R.: Ordinal regression revisited: Multiple criteria ranking using a set of additive value functions. European Journal of Operational Research 191(2), 416–436 (2008), doi:10.1016/j.ejor.2007.08.013

    MathSciNet  MATH  Google Scholar 

  53. Harrington, E.F.: Online Ranking/Collaborative Filtering Using the Perceptron Algorithm. In: Proceedings of the 20th International Conference on Machine Learning, pp. 250–257 (2003)

    Google Scholar 

  54. Hastie, T., Tibshirani, R.: Generalized Additive Models. Statistical Science 1, 297–318 (1986)

    MathSciNet  Google Scholar 

  55. Haykin, S.: Neural Networks and Learning Machines, 3rd edn. Prentice Hall (2008)

    Google Scholar 

  56. Herbrich, R., Graepel, T., Obermayer, K.: Regression Models for Ordinal Data: A Machine Learning Approach. Tech. rep. (1999)

    Google Scholar 

  57. Herstein, I.N., Milnor, J.: An axiomatic approach to measurable utility. Econometrica 21(2), 291–297 (1953)

    MathSciNet  MATH  Google Scholar 

  58. Hillier, F.S., Lieberman, G.J., Hillier, F., Lieberman, G.: MP Introduction to Operations Research. McGraw-Hill Science/Engineering/Math (2004)

    Google Scholar 

  59. Huang, J., Ling, C.: Using auc and accuracy in evaluating learning algorithms. IEEE Transactions on Knowledge and Data Engineering 17(3), 299–310 (2005), doi:10.1109/TKDE.2005.50

    Google Scholar 

  60. Huédé, F., Grabisch, M., Labreuche, C., Savéant, P.: Integration and propagation of a multi-criteria decision making model in constraint programming. Journal of Heuristics 12(4-5), 329–346 (2006), doi:10.1007/s10732-006-8075-2

    MATH  Google Scholar 

  61. Iryna, Y.: Solving classification problems with multicriteria decision aiding approaches (2007)

    Google Scholar 

  62. Ishizaka, A., Balkenborg, D., Kaplan, T.: Does ahp help us make a choice? an experimental evaluation. JORS 62(10), 1801–1812 (2011), doi:10.1057/jors.2010.158

    Google Scholar 

  63. Ishizaka, A., Labib, A.: Analytic hierarchy process and expert choice: Benefits and limitations. OR Insight 22(4), 201–220 (2009), doi:10.1057/ori.2009.10

    Google Scholar 

  64. Ishizaka, A., Labib, A.: Review of the main developments in the analytic hierarchy process. Expert Systems with Applications 38(11), 14,336–14,345 (2011), doi:10.1016/j.eswa.2011.04.143

    Google Scholar 

  65. Jensen, R., Shen, Q.: Computational intelligence and feature selection: Rough and fuzzy approaches (2008)

    Google Scholar 

  66. Junker, U.: Preference-based search and multi-criteria optimization. Annals of Operations Research 130(1), 75–115 (2004)

    MathSciNet  MATH  Google Scholar 

  67. Junker, U.: Preference-based problem solving for constraint programming, pp. 109–126 (2008), doi:10.1007/978-3-540-89812-2_8

    Google Scholar 

  68. Kangas, J., Kurttila, M., Kajanus, M., Kangas, A.: Evaluating the management strategies of a forestland estate–the s-o-s approach. J. Environ. Manage. 69(4), 349–358 (2003), doi:10.1016/j.jenvman.2003.09.010

    Google Scholar 

  69. Kecman, V.: Learning and Soft Computing: Support Vector Machines, Neural Networks, and Fuzzy Logic Models. MIT Press, Cambridge (2001)

    MATH  Google Scholar 

  70. Kendall, M.: A new measure of rank correlation. Biometrika 30, 81–89 (1938)

    MathSciNet  MATH  Google Scholar 

  71. Kotlowski, W., Dembczynski, K., Greco, S., Slowinski, R.: Stochastic dominance-based rough set model for ordinal classification. Information Sciences 178(21), 4019–4037 (2008)

    MathSciNet  MATH  Google Scholar 

  72. Kramer, S., Widmer, G., Pfahringer, B., de Groeve, M.: Prediction of Ordinal Classes Using Regression Trees. Fundam. Inf. 47(1-2), 1–13 (2001)

    MATH  Google Scholar 

  73. Köksalan, M., Özpeynirci, S.B.: An interactive sorting method for additive utility functions. Computers & Operations Research 36(9), 2565–2572 (2009), doi:10.1016/j.cor.2008.11.006

    MATH  Google Scholar 

  74. Lahdelma, R., Miettinen, K., Salminen, P.: Ordinal criteria in stochastic multicriteria acceptability analysis (smaa). European Journal of Operational Research 147(1), 117–127 (2003), doi:10.1016/S0377-2217(02)00267-9

    MATH  Google Scholar 

  75. Lahdelma, R., Salminen, P.: Prospect theory and stochastic multicriteria acceptability analysis (SMAA). Omega 37(5), 961–971 (2009), doi:10.1016/j.omega.2008.09.001

    Google Scholar 

  76. Lahdelma, R., Salminen, P., Hokkanen, J.: Locating a waste treatment facility by using stochastic multicriteria acceptability analysis with ordinal criteria. European Journal of Operational Research 142(2), 345–356 (2002), doi:10.1016/S0377-2217(01)00303-4

    MATH  Google Scholar 

  77. Lakiotaki, K., Delias, P., Sakkalis, V., Matsatsinis, N.: User profiling based on multi-criteria analysis: the role of utility functions. Operational Research 9, 3–16 (2009), doi:10.1007/s12351-008-0024-4

    MATH  Google Scholar 

  78. Lakiotaki, K., Matsatsinis, N., Tsoukiàs, A.: Multicriteria user modeling in recommender systems. IEEE Intelligent Systems 26(2), 64–76 (2011), doi:10.1109/MIS.2011.33

    Google Scholar 

  79. Last, M., Kandel, A., Maimon, O.: Information-theoretic algorithm for feature selection. Pattern Recognition Letters 22(6-7), 799–811 (2001), doi:10.1016/S0167-8655(01)00019-8

    MATH  Google Scholar 

  80. Lavesson, N., Davidsson, P.: Evaluating learning algorithms and classifiers. Int. J. Intell. Inf. Database Syst. 1, 37–52 (2007), doi:10.1504/IJIIDS.2007.013284

    Google Scholar 

  81. Lee, J., Liu, D.Z.: Induction of ordinal decision trees. In: Proceedings of the International Conference on Machine Learning and Cybernetics, vol. 4, pp. 2220–2224 (2002)

    Google Scholar 

  82. Lee, K.H.: First Course On Fuzzy Theory And Applications. Springer (2004)

    Google Scholar 

  83. Lin, H.T., Li, L.: Combining ordinal preferences by boosting. In: Proceedings ECML/PKDD 2009 Workshop on Preference Learning, pp. 69–83 (2009)

    Google Scholar 

  84. Liu, H., Setiono, R.: Feature selection via discretization. IEEE Transactions on Knowledge and Data Engineering 9(4), 642–645 (1997), doi:10.1109/69.617056

    Google Scholar 

  85. Maccheroni, F., Marinacci, M., Rustichini, A.: Ambiguity aversion, robustness, and the variational representation of preferences. Econometrica 74(6), 1447–1498 (2006)

    MathSciNet  MATH  Google Scholar 

  86. Marichal, J.L.: Aggregation Operators for Multicriteria Decision Aid. Ph.D. thesis, Institute of Mathematics, University of Liège, Liège, Belgium (1998)

    Google Scholar 

  87. McCullagh, P.: Regression Models for Ordinal Data. Journal of the Royal Statistical Society 42(2), 109–142 (1980)

    MathSciNet  MATH  Google Scholar 

  88. McGeachie, M.: Msc.utility functions for ceteris paribus preferences. Master’s thesis, Department of Electrical Engineering and Computer Science, MIT (2002)

    Google Scholar 

  89. McGeachie, M., Doyle, J.: Efficient utility functions for ceteris paribus preferences. In: Eighteenth National Conference on Artificial Intelligence, pp. 279–284. American Association for Artificial Intelligence, Menlo Park (2002)

    Google Scholar 

  90. McGeachie, M., Doyle, J.: Utility functions for ceteris paribus preferences. Computational Intelligence 20(2), 158–217 (2002)

    MathSciNet  Google Scholar 

  91. Meyer, P., Roubens, M.: Choice, Ranking and Sorting in Fuzzy Multiple Criteria Decision Aid. In: Figueira, J., Greco, S., Ehrgott, M. (eds.) Multiple Criteria Decision Analysis: State of the Art Surveys, pp. 471–506. Springer, Boston (2005)

    Google Scholar 

  92. Miettinen, K.: Nonlinear Multiobjective Optimization. International Series in Operations Research and Management Science, vol. 12. Kluwer Academic Publishers, Dordrecht (1999)

    MATH  Google Scholar 

  93. Mousseau, V., Figueira, J., Naux, J.P.: Using assignment examples to infer weights for Electretri method: Some experimental results. European Journal of Operational Research 130(2), 263–275 (2001), doi:10.1016/S0377-2217(00)00041-2

    MATH  Google Scholar 

  94. Olafsson, S., Li, X., Wu, S.: Operations research and data mining. European Journal of Operational Research 187(3), 1429–1448 (2008), doi:10.1016/j.ejor.2006.09.023

    MathSciNet  MATH  Google Scholar 

  95. Oliveira, H.P., Magalhaes, A., Cardoso, M.J., Cardoso, J.S.: An accurate and interpretable model for bcct. core. In: Proceedings of the 32nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 6158–6161 (2010)

    Google Scholar 

  96. Ouyang, H., Gray, A.: Learning dissimilarities by ranking: from sdp to qp. In: International Conference on Machine Learning, pp. 728–735 (2008), doi:10.1145/1390156.1390248

    Google Scholar 

  97. Pawlak, Z.: Rough sets. International Journal of Computer and Information Sciences 11(5), 341–356 (1982)

    MathSciNet  MATH  Google Scholar 

  98. Pawlak, Z.: Rough set approach to knowledge-based decision support. European Journal of Operational Research 99(1), 48–57 (1997), doi:10.1016/S0377-2217(96)00382-7

    MATH  Google Scholar 

  99. Potharst, R., Bioch, J.C.: A decision tree algorithm for ordinal classification. In: Advances in Intelligent Data Analysis, pp. 187–198 (1999)

    Google Scholar 

  100. Potharst, R., Bioch, J.C.: Decision trees for ordinal classification. Intelligent Data Analysis 4(2), 97–111 (2000)

    MATH  Google Scholar 

  101. Potharst, R., Feelders, A.J.: Classification trees for problems with monotonicity constraints. SIGKDD Explorations Newsletter 4(1), 1–10 (2002), doi: http://doi.acm.org/10.1145/568574.568577

    Google Scholar 

  102. Presson, A., Yoon, N., Bagryanova, L., Mah, V., Alavi, M., Maresh, E., Rajasekaran, A., Goodglick, L., Chia, D., Horvath, S.: Protein expression based multimarker analysis of breast cancer samples. BMC Cancer 11(1), 230 (2011), doi:10.1186/1471-2407-11-230

    Google Scholar 

  103. Pyon, Y.S., Li, J.: Identifying gene signatures from cancer progression data using ordinal analysis. In: IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2009, pp. 136–141 (2009), doi:10.1109/BIBM.2009.18

    Google Scholar 

  104. Rebelo, A., Tkaczuk, J., Sousa, R., Cardoso, J.: Metric learning for music symbol recognition (2011)

    Google Scholar 

  105. Rietveld, P., Ouwersloot, H.: Ordinal data in multicriteria decision making: a stochastic dominance approach to siting nuclear power plants. European Journal of Operational Research 56(2), 249–262 (1992)

    Google Scholar 

  106. Rodriguez-Lujan, I., Huerta, R., Elkan, C., Cruz, C.S.: Quadratic programming feature selection. Journal of Machine Learning Research 11, 1491–1516 (2010)

    MATH  Google Scholar 

  107. Roy, B.: The outranking approach and the foundations of electre methods. Theory and Decision 31, 49–73 (1991), doi:10.1007/BF00134132

    MathSciNet  Google Scholar 

  108. Russell, S.J., Norvig, P.: Artificial Intelligence: A Modern Approach. Pearson Education (2003)

    Google Scholar 

  109. Saaty, T.L.: How to make a decision: The analytic hierarchy process. European Journal of Operational Research 48(1), 9–26 (1990), doi:10.1016/0377-2217(90)90057-I

    MATH  Google Scholar 

  110. Saaty, T.L., Vargas, L.G., Saaty, T.L., Vargas, L.G.: The seven pillars of the analytic hierarchy process. In: Models, Methods, Concepts & Applications of the Analytic Hierarchy Process. International Series in Operations Research & Management Science, vol. 34, pp. 27–46. Springer, US (2001), doi:10.1007/978-1-4615-1665-1_2

    Google Scholar 

  111. Schultz, M., Joachims, T.: Learning a distance metric from relative comparisons. In: NIPS. MIT Press (2004)

    Google Scholar 

  112. Seth, S., Príncipe, J.C.: Variable Selection: A Statistical Dependence Perspective. In: Proceeding of the Ninth International Conference on Machine Learning and Applications, pp. 931–936 (2010)

    Google Scholar 

  113. Shashua, A., Levin, A.: Ranking with large margin principle: Two approaches (2003)

    Google Scholar 

  114. Shen, L., Joshi, A.: Ranking and Reranking with Perceptron. Machine Learning 60, 73–96 (2005)

    Google Scholar 

  115. Siskos, Y., Grigoroudis, E., Matsatsinis, N.: Uta methods. In: Figueira, J., Greco, S., Ehrgott, M. (eds.) Multiple Criteria Decision Analysis: State of the Art Surveys, pp. 297–344. Springer, Boston (2005)

    Google Scholar 

  116. Siwik, L., Natanek, S.: Elitist evolutionary multi-agent system in solving noisy multi-objective optimization problems. In: IEEE Congress on Evolutionary Computation, CEC 2008 (IEEE World Congress on Computational Intelligence), pp. 3319–3326 (2008)

    Google Scholar 

  117. Siwik, L., Natanek, S.: Solving constrained multi-criteria optimization tasks using Elitist Evolutionary Multi-Agent System. In: IEEE Congress on Evolutionary Computation, CEC 2008 (IEEE World Congress on Computational Intelligence), pp. 3358–3365 (2008)

    Google Scholar 

  118. Smet, Y.D., Guzmán, L.M.: Towards multicriteria clustering: An extension of the k-means algorithm. European Journal of Operational Research 158(2), 390–398 (2004), doi:10.1016/j.ejor.2003.06.012; Methodological Foundations of Multi-Criteria Decision Making

    MathSciNet  MATH  Google Scholar 

  119. Chakrabarti, S., Ester, M., Fayyad, U., Gehrke, J., Han, J., Morishita, S., Piatetsky-Shapiro, G., Wang, W.: Data mining curriculum: a proposal, Version 1.0 (2006), www.kdd.org/curriculum/CURMay06.pdf (retrieved February 27, 2012)

  120. Sousa, R., Cardoso, J.S.: Ensemble of Decision Trees with Global Constraints for Ordinal Classification. In: 11th International Conference on Intelligent Systems Design and Applications (ISDA 2011), Cordoba, Spain, Spain (2011)

    Google Scholar 

  121. Sousa, R., Oliveira, H.P., Cardoso, J.S.: Feature selection with complexity measure in a quadratic programming setting. In: Proceedings of Iberian Conference on Pattern Recognition and Image Analysis (IbPRIA), pp. 524–531 (2011)

    Google Scholar 

  122. Spearman, C.: The proof and measurement of association between two things. American Journal of Psychology 15, 72–101 (1904)

    Google Scholar 

  123. Sridhar, P., Madni, A., Jamshidi, M.: Multi-criteria decision making in sensor networks. IEEE Instrumentation Measurement Magazine 11(1), 24–29 (2008), doi:10.1109/IM-M.2008.4449011

    Google Scholar 

  124. Sun, B.Y., Li, J., Wu, D., Zhang, X.M., Li, W.B.: Kernel discriminant learning for ordinal regression. IEEE Transactions on Knowledge and Data Engineering 22(6), 906–910 (2010), doi:10.1109/TKDE.2009.170

    Google Scholar 

  125. Tagliafico, A., Tagliafico, G., Tosto, S., Chiesa, F., Martinoli, C., Derchi, L.E., Calabrese, M.: Mammographic density estimation: Comparison among bi-rads categories, a semi-automated software and a fully automated one. The Breast 18(1), 35–40 (2009)

    Google Scholar 

  126. Taha, H.A.: Operations Research: An Introduction, 8th edn. Prentice-Hall, Inc., Upper Saddle River (2006)

    Google Scholar 

  127. Tervonen, T., Figueira, J.R.: A survey on stochastic multicriteria acceptability analysis methods. Journal of Multi-Criteria Decision Analysis 15, 1–14 (2008), doi:10.1002/mcda.407

    MATH  Google Scholar 

  128. Tervonen, T., Lahdelma, R.: Implementing stochastic multicriteria acceptability analysis. European Journal of Operational Research 178(2), 500–513 (2007), doi:10.1016/j.ejor.2005.12.037

    MATH  Google Scholar 

  129. Tutz, G.: Generalized Semiparametrically Structured Ordinal Models. Biometrics 59, 263–273 (2003)

    MathSciNet  MATH  Google Scholar 

  130. Ustinovichius, L., Zavadskas, E.K., Podvezko, V.: The application of a quantitative multiple criteria decision making (mcdm-1) approach to the analysis of investments in construction. Control and Cybernetics 36 (2007)

    Google Scholar 

  131. van Vanya, B., Kristiaan, P., Suykens Johan, A.K., van Sabine, H.: Learning transformation models for ranking and survival analysis. Journal of Machine Learning Research 12, 819–862 (2011)

    Google Scholar 

  132. Vapnik, V.N.: Statistical Learning Theory. Wiley-Interscience (1998)

    Google Scholar 

  133. Waegeman, W., Baets, B.D., Boullart, L.: Roc analysis in ordinal regression learning. Pattern Recognition Letters 29(1), 1–9 (2008), doi:10.1016/j.patrec.2007.07.019

    Google Scholar 

  134. Waegeman, W., De Baets, B., Boullart, L.: A comparison of different ROC measures for ordinal regression. In: Proceedings of the CML 2006 Workshop on ROC Analysis in Machine Learning (2006)

    Google Scholar 

  135. Waegeman, W., de Baets, B., Boullart, L.: Kernel-based learning methods for preference aggregation. 4OR: A Quarterly Journal of Operations Research 7, 169–189 (2009), doi:10.1007/s10288-008-0085-5

    MathSciNet  MATH  Google Scholar 

  136. Wang, J.J., Jing, Y.Y., Zhang, C.F.: Weighting methodologies in multi-criteria evaluations of combined heat and power systems. International Journal of Energy Research 33(12), 1023–1039 (2009), doi:10.1002/er.1527

    Google Scholar 

  137. Wang, J.J., Jing, Y.Y., Zhang, C.F., Zhao, J.H.: Review on multi-criteria decision analysis aid in sustainable energy decision-making. Renewable and Sustainable Energy Reviews 13(9), 2263–2278 (2009), doi:10.1016/j.rser.2009.06.021

    Google Scholar 

  138. Weinberger, K.Q., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. Journal of Machine Learning Research 10, 207–244 (2009)

    MATH  Google Scholar 

  139. Wolpert, D.H.: The supervised learning no-free-lunch theorems. In: Proc. 6th Online World Conference on Soft Computing in Industrial Applications, pp. 25–42 (2001)

    Google Scholar 

  140. Xia, F., Tao, Q., Wang, J., Zhang, W.: Recursive Feature Extraction for Ordinal Regression. In: International Joint Conference on Neural Networks, IJCNN 2007, pp. 78–83 (2007)

    Google Scholar 

  141. Xu, X., Zhou, C., Wang, Z.: Credit scoring algorithm based on link analysis ranking with support vector machine. Expert Syst. Appl. 36, 2625–2632 (2009), doi:10.1016/j.eswa.2008.01.024

    Google Scholar 

  142. Yang, L., Jin, R.: Distance metric learning: A comprehensive survey. Tech. rep., Department of Computer Science and Engineering, Michigan State University (2006)

    Google Scholar 

  143. Yu, S., Yu, K., Tresp, V., Kriegel, H.P.: Collaborative ordinal regression. In: Proceedings of the 23rd International Conference on Machine Learning, ICML 2006, pp. 1089–1096. ACM, New York (2007), doi:10.1145/1143844.1143981

    Google Scholar 

  144. Zhang, Z., Kwok, J.T., Yeung, D.Y.: Parametric distance metric learning with label information. In: Proceedings of the 18th International Joint Conference on Artificial Intelligence, pp. 1450–1452. Morgan Kaufmann Publishers Inc., San Francisco (2003)

    Google Scholar 

  145. Zopounidis, C., Doumpos, M.: Building additive utilities for multi-group hierarchical discrimination: The M.H.DIS method. Optimization Methods and Software 14(3), 219–240 (2000), doi:10.1080/10556780008805801

    MATH  Google Scholar 

  146. Zopounidis, C., Doumpos, M.: Multicriteria classification and sorting methods: A literature review. European Journal of Operational Research 138(2), 229–246 (2002), doi:10.1016/S0377-2217(01)00243-0

    MathSciNet  MATH  Google Scholar 

  147. Zopounidis, C., Pardalos, P.M.: Handbook of multicriteria analysis. Applied Optimization 103. Springer, Berlin (2010), doi:10.1007/978-3-540-92828-7

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. INESC TEC (formerly INESC Porto), Faculdade de Engenharia, Universidade do Porto, Porto, Portugal

    Ricardo Sousa & Jaime S. Cardoso

  2. Computer Science and Communication Research Center, Polytechnic Institute of Leiria, Leiria, Portugal

    Iryna Yevseyeva

  3. CMUP, Faculdade de Ciências, Universidade do Porto, Porto, Portugal

    Joaquim F. Pinto da Costa

Authors
  1. Ricardo Sousa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Iryna Yevseyeva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Joaquim F. Pinto da Costa
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jaime S. Cardoso
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ricardo Sousa .

Editor information

Editors and Affiliations

  1. , Mathematics and Scientific Computing, National Physical Laboratory, Hampton Road, Teddington, TW11 0LW, United Kingdom

    Xin-She Yang

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag GmbH Berlin Heidelberg

About this chapter

Cite this chapter

Sousa, R., Yevseyeva, I., da Costa, J.F.P., Cardoso, J.S. (2013). Multicriteria Models for Learning Ordinal Data: A Literature Review. In: Yang, XS. (eds) Artificial Intelligence, Evolutionary Computing and Metaheuristics. Studies in Computational Intelligence, vol 427. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29694-9_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-29694-9_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-29693-2

  • Online ISBN: 978-3-642-29694-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation