Skip to main content
SpringerLink
Log in

Multi-criteria Fuzzy Ordinal Peer Assessment for MOOCs

  • Conference paper
  • First Online:
Advances in Intelligent Networking and Collaborative Systems (INCoS 2018)

Part of the book series: Lecture Notes on Data Engineering and Communications Technologies ((LNDECT,volume 23))

Abstract

Due to the high number of students enrolled and the relatively small number of available tutors, the assessment of complex assignments is deemed as one of the most critical tasks in Massive Open On-line Courses (MOOCs). Peer assessment is becoming an increasingly popular tool to face this problem and many approaches have been proposed so far to make its outcomes more reliable. A promising approach is FOPA (Fuzzy Ordinal Peer Assessment) that adopts and integrates models coming from Fuzzy Set Theory and Group Decision Making. In this paper we propose a FOPA extension supporting multi-criteria assessment based on rubrics. Students are asked to rank a small number of peer submissions against specified criteria, provided rankings are then transformed in fuzzy preference relations, expanded to obtain missing values and aggregated to establish a global ranking between students’ works with respect to each criterion and globally. The absolute grades of all submissions are then calculated.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Glance, D.G., Forsey, M., Riley, M.: The pedagogical foundations of massive open online courses. First Monday 18(5) (2013)

    Google Scholar 

  2. Bouzidi, L., Jaillet, A.: Can online peer assessment be trusted? Educ. Technol. Soc. 12(4), 257–268 (2009)

    Google Scholar 

  3. Carlson, P.A., Berry, F.C.: Calibrated peer review™ and assessing learning outcomes. In: Proceedings of the 33rd International Conference Frontiers in Education (2003)

    Google Scholar 

  4. Walsh, T.: The peer rank method for peer assessment. In: Proceedings of the 21st European Conference on Artificial Intelligence (2014)

    Google Scholar 

  5. Capuano, N., Caballé, S.: Towards adaptive peer assessment for MOOCs. In: Proceedings of the 10th International Conference on P2P, Parallel, GRID, Cloud and Internet Computing (3PGCIC 2015), Krakow, Poland (2015)

    Google Scholar 

  6. Raman, K., Joachims, T.: Methods for ordinal peer grading. In: Proceedings of the 20th SIGKDD International Conference on Knowledge Discovery and Data Mining (2014)

    Google Scholar 

  7. Capuano, N., Loia, V., Orciuoli, F.: A fuzzy group decision making model for ordinal peer assessment. IEEE Trans. Learn. Technol. 10(2), 247–259 (2017)

    Article  Google Scholar 

  8. Capuano, N., Orciuoli, F.: Application of fuzzy ordinal peer assessment in formative evaluation. In: Proceedings of the 12th International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC 2017), Barcelona, Spain (2017)

    Google Scholar 

  9. Albano, G., Capuano, N., Pierri, A.: Adaptive peer grading and formative assessment. J. e-Learn. Knowl. Soc. 13(1), 147–161 (2017)

    Google Scholar 

  10. Reddy, Y., Andrade, H.: A review of rubric use in higher education. Assess. Eval. High. Educ. 35(4), 435–448 (2009)

    Article  Google Scholar 

  11. Lan, C.H., Graf, S., Lai, K.R., Kinshuk, K.: Enrichment of peer assessment with agent negotiation. IEEE Trans. Learn. Technol. 4(1), 35–46 (2011)

    Google Scholar 

  12. Capuano, N., Caballé, S., Miguel, J.: Improving peer grading reliability with graph mining techniques. Int. J. Emerg. Technol. Learn. 11(7), 24–33 (2016)

    Article  Google Scholar 

  13. Caragiannis, I., Krimpas, A., Voudouris, A.A.: Aggregating partial rankings with applications to peer grading in massive online open courses. In: Proceedings of the International Conference on Autonomous Agents and Multiagent Systems, Istanbul (2015)

    Google Scholar 

  14. Borda, J.C.: Memoire sur les elections au scrutin, Histoire de l’Académie Royale des Sciences (1781)

    Google Scholar 

  15. Pedrycz, W., Ekel, P., Parreiras, R.: Fuzzy Multicriteria Decision-Making: Models, Methods and Applications. Wiley, Hoboken (2010)

    Book  Google Scholar 

  16. Lu, J., Zhang, G., Ruan, D., Wu, F.: Multi-objective group decision making, methods, software and applications with fuzzy set techniques, World Scientific (2007)

    Google Scholar 

  17. Capuano, N., Chiclana, F., Fujita, H., Herrera-Viedma, E., Loia, V.: Fuzzy group decision making with incomplete information guided by social influence. In: IEEE Transactions of Fuzzy Systems. In Press

    Google Scholar 

  18. Zadeh, L.A.: A computational approach to fuzzy quantifiers in natural languages. Comput. Math Appl. 9, 149–184 (1983)

    Article  MathSciNet  Google Scholar 

  19. Wang, Y.M., Fan, Z.P.: Fuzzy preference relations: aggregation and weight determination. Comput. Ind. Eng. 53(1), 163–172 (2007)

    Article  MathSciNet  Google Scholar 

  20. Yager, R.R.: Families of OWA operators. Fuzzy Sets Syst. 59(2), 125–148 (1993)

    Article  MathSciNet  Google Scholar 

  21. Chiclana, F., Herrera, F., Herrera-Viedma, E.: Integrating three representation models in fuzzy multipurpose decision making based on fuzzy preference relations. Fuzzy Sets Syst. 97(1), 33–48 (1998)

    Article  MathSciNet  Google Scholar 

  22. Capuano, N., Chiclana, F., Fujita, H., Herrera-Viedma, E., Loia, V.: Fuzzy rankings for preferences modeling in group decision making. Int. J. Intell. Syst. 33(7), 1555–1570 (2018)

    Article  Google Scholar 

  23. Alonso, S., Chiclana, F., Herrera, F., Herrera-Viedma, E., Alcala-Fdez, J., Porcel, C.: A consistency-based procedure to estimate missing pairwise preference values. Int. J. Intell. Syst. 23(1), 155–175 (2008)

    Article  Google Scholar 

  24. Alonso, S., Herrera-Viedma, E., Chiclana, F., Herrera, F.: Individual and social strategies to deal with ignorance situations in multi-person decision making. Int. J. Inf. Technol. Decis. Making 8(2), 313–333 (2009)

    Article  Google Scholar 

  25. Jonsson, A., Svingby, G.: The use of scoring rubrics: reliability, validity and educational consequences. Educ. Res. Rev. 2(2), 130–144 (2007)

    Article  Google Scholar 

  26. Ekel, P., Queiroz, J., Parreiras, R., Palhares, R.: Fuzzy set based models and methods of multicriteria group decision-making. Nonlinear Anal.: Theor. Methods Appl. 71(12), 409–419 (2009)

    Article  Google Scholar 

  27. Chiclana, F., Herrera-Viedma, E., Herrera, F., Alonso, S.: Some induced ordered weighted averaging operators and their use for solving group decision-making problems based on fuzzy preference relations. Eur. J. Oper. Res. 182(1), 383–399 (2007)

    Article  Google Scholar 

Download references

Acknowledgement

This work has been supported by the project colMOOC “Integrating Conversational Agents and Learning Analytics in MOOCs”, co-funded by the European Commission within the Erasmus + Knowledge Alliances program (ref. 588438-EPP-1-2017-1-EL-EPPKA2-KA).

Author information

Authors and Affiliations

  1. Department of Information Engineering, Electric Engineering and Applied Mathematics, University of Salerno, Via Giovanni Paolo II 132, 84084, Fisciano, SA, Italy

    Nicola Capuano

  2. Faculty of Computer Science, Multimedia and Telecommunications, Universitat Oberta de Catalunya, Rambla Poblenou, 156, 08018, Barcelona, Spain

    Santi Caballé

Authors
  1. Nicola Capuano
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Santi Caballé
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicola Capuano .

Editor information

Editors and Affiliations

  1. Universitat Politècnica de Catalunya, Barcelona, Spain

    Fatos Xhafa

  2. Department of Information and Communication Engineering, Faculty of Information Engineering, Fukuoka Institute of Technology, Fukuoka-shi, Fukuoka-ken, Japan

    Leonard Barolli

  3. Faculty of Management, Comenius University in Bratislava, Bratislava, Slovakia

    Michal Greguš

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Capuano, N., Caballé, S. (2019). Multi-criteria Fuzzy Ordinal Peer Assessment for MOOCs. In: Xhafa, F., Barolli, L., Greguš, M. (eds) Advances in Intelligent Networking and Collaborative Systems. INCoS 2018. Lecture Notes on Data Engineering and Communications Technologies, vol 23. Springer, Cham. https://doi.org/10.1007/978-3-319-98557-2_34

Download citation

Publish with us

Policies and ethics

Search

Navigation