Skip to main content
SpringerLink
Log in

Computerbasiertes Assessment

  • Chapter
  • First Online:
Testtheorie und Fragebogenkonstruktion

  • 68k Accesses

Zusammenfassung

Das Kapitel gibt einen Überblick, wie mit Hilfe von Computern im weiteren Sinne Tests und Fragebogen realisiert und dabei die Möglichkeiten von klassischen Papier-und-Bleistift-Verfahren erweitert bzw. deutlich überschritten werden können. Dies betrifft beispielsweise die Entwicklung computerbasierter Items mit innovativen Antwortformaten und multimedialen Stimuli sowie die automatische Bewertung des gezeigten Antwortverhaltens. Des Weiteren ermöglicht der Computer eine flexiblere Testzusammenstellung, d. h., Items können automatisch unter Berücksichtigung inhaltlicher und statistischer Kriterien sequenziert werden. Das Kapitel behandelt außerdem die Frage, wie durch Logfiledaten das Analysepotential gesteigert und durch die automatische und zeitnahe Rückmeldung von Testdaten beispielsweise das Lernen unterstützt werden kann. Das Kapitel schließt mit Hinweisen auf einschlägige und frei zugängliche Softwarelösungen für Assessmentzwecke.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 69.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Literatur

  • Abele, S., Walker, F. & Nickolaus, R. (2014). Zeitökonomische und reliable Diagnostik beruflicher Problemlösekompetenzen bei Auszubildenden zum Kfz-Mechatroniker. Zeitschrift für Pädagogische Psychologie, 4, 167–179.

    Google Scholar 

  • Aleven, V., Roll, I., McLaren, B. & Koedinger, K. (2010). Automated, unobtrusive, action-by-action assessment of self-regulation during learning with an intelligent tutoring system. Educational Psychologist, 45, 224–233.

    Google Scholar 

  • Bassili, J. N. & Fletcher, J. F. (1991). Response-time measurement in survey research a method for CATI and a new look at nonattitudes, Public Opinion Quarterly, 55, 331–346.

    Google Scholar 

  • Brace, I. (2008). Questionnaire design: How to plan, structure and write survey material for effective market research (2nd ed). London; Philadelphia: Kogan Page.

    Google Scholar 

  • Braun, H., Bejar, I. I. & Williamson, D. M. (2006). Rule-based methods for automated scoring: Application in a licensing context. In D. M. Williamson, I. I. Bejar & R. J. Mislevy (Eds.), Automated scoring of complex tasks in computer-based testing (pp. 83–122). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Chen, M., Mao, S. & Liu, Y. (2014). Big Data: A Survey. Mobile Networks and Applications, 19, 171–209.

    Google Scholar 

  • Clark, D., Nelson, B., Sengupta, P. & D’Angelo, C. (2009). Rethinking science learning through digital games and simulations: Genres, examples, and evidence. An NAS commissioned paper. Retrieved from http://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_080068.pdf [20.12.2019]

  • Cor, K., Alves, C. & Gierl, M. (2009). Three applications of automated test assembly within a user-friendly modeling environment. Practical Assessment, Research & Evaluation, 14. Retrieved from http://pareonline.net/getvn.asp?v=14%26n=14 [20.12.2019]

  • Deroos, D., Deutsch, T., Eaton, C., Lapis, G. & Zikopoulos, P. (2012). Understanding big data: analytics for enterprise class Hadoop and streaming data. New York, NY: McGraw-Hill.

    Google Scholar 

  • Diao, Q. & van der Linden, W. J. (2011). Automated Test Assembly Using lp_Solve Version 5.5 in R. Applied Psychological Measurement, 35, 398–409.

    Google Scholar 

  • Dirk, J. & Schmiedek, F. (2016). Fluctuations in elementary school children’s working memory performance in the school context. Journal of Educational Psychology, 108, 722–739.

    Google Scholar 

  • Ebner-Priemer, U. W., Kubiak, T. & Pawlik, K. (2009). Ambulatory Assessment. European Psychologist, 14, 95–97.

    Google Scholar 

  • Frey, A. & Hartig, J. (2013). Wann sollten computerbasierte Verfahren zur Messung von Kompetenzen anstelle von Papier- und Bleistift-basierten Verfahren eingesetzt werden? Zeitschrift für Erziehungswissenschaft, 16, 53–57.

    Google Scholar 

  • Funke, U. (1995). Using complex problem solving tasks in personnel selection and training. In P. A. Frensch & J. Funke (Eds.), Complex problem solving: The European perspective (pp. 219–240). Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Gawrilow, C., Stadler, G., Langguth, N., Naumann, A. & Boeck, A. (2016). Physical activity, affect, and cognition in children with symptoms of ADHD. Journal of Attention Disorders, 20, 151–162.

    PubMed  Google Scholar 

  • Gierl, M. J. & Lai, H. (2013). Using automated processes to generate test items. Educational Measurement: Issues and Practice, 32, 36–50.

    Google Scholar 

  • Goldhammer, F., Kröhne, U., Keßel, Y., Senkbeil, M. & Ihme, J. M. (2014a). Diagnostik von ICT-Literacy: Multiple-Choice- vs. simulationsbasierte Aufgaben. Diagnostica, 60, 10–21.

    Google Scholar 

  • Goldhammer, F., Naumann, J., Rölke, H., Stelter, A. & Tóth, K. (2017). Relating product data to process data from computer-based competency assessment. In D. Leutner, J. Fleischer, J. Grünkorn & E. Klieme (Eds.), Competence Assessment in Education: Research, Models and Instruments (pp. 407–425). Berlin, Heidelberg: Springer.

    Google Scholar 

  • Goldhammer, F., Naumann, J., Stelter, A., Tóth, K., Rölke, H. & Klieme, E. (2014b). The time on task effect in reading and problem solving is moderated by task difficulty and skill: Insights from a computer-based large-scale assessment. Journal of Educational Psychology, 106, 608–626.

    Google Scholar 

  • Greiff, S., Niepel, C., Scherer, R. & Martin, R. (2016). Understanding students’ performance in a computer-based assessment of complex problem solving: An analysis of behavioral data from computer-generated log files. Computers in Human Behavior, 61, 36–46.

    Google Scholar 

  • Greiff, S., Wüstenberg, S. & Funke, J. (2012). Dynamic Problem Solving: A new measurement perspective. Applied Psychological Measurement, 36, 189–213.

    Google Scholar 

  • Hahnel, C., Goldhammer, F., Naumann, J. & Kröhne, U. (2016). Effects of linear reading, basic computer skills, evaluating online information, and navigation on reading digital text. Computers in Human Behavior, 55, 486–500.

    Google Scholar 

  • Han, K. T. & Guo, F. (2014). Multistage testing by shaping modules on the fly. In D. Yan, A. A. von Davier & Lewis (Eds.), Computerized multistage testing: Theory and applications (pp. 119–133). New York, NY: CRC Press.

    Google Scholar 

  • Holling, H., Bertling, J. P. & Zeuch, N. (2009). Automatic item generation of probability word problems. Studies in Educational Evaluation, 35, 71–76.

    Google Scholar 

  • Hornke, L. F., Wosnitza, M. & Bürger, K. (2013). SelfAssessment: Ideen, Hintergründe, Praxis und Evaluation, Wirtschaftspsychologie, 15, 5–16.

    Google Scholar 

  • Klein Entink, R. H., Fox, J.-P. & van der Linden, W. J. (2009). A multivariate multilevel approach to the modeling of accuracy and speed of test takers. Psychometrika, 74, 21–48.

    PubMed  Google Scholar 

  • Koedinger, K. R. & Aleven V. (2007). Exploring the assistance dilemma in experiments with cognitive tutors. Educational Psychology Review, 19, 239–264.

    Google Scholar 

  • Kong, X., Wise, S. L. & Bhola, D. S. (2007). Setting the response time threshold parameter to differentiate solution behavior from rapid-guessing behavior. Educational and Psychological Measurement, 67, 606–619.

    Google Scholar 

  • Kosinski, M., Stillwell, D. & Graepel, T. (2013). Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences, 110, 5802–5805.

    Google Scholar 

  • Kreuter, F. (2013). Improving surveys with paradata: Analytic uses of process information (Vol. 581). New York, NY: John Wiley & Sons.

    Google Scholar 

  • Kroehne, U. & Martens, T. (2011). Computer-based competence tests in the national educational panel study: The challenge of mode effects. Zeitschrift für Erziehungswissenschaft, 14, 169–186.

    Google Scholar 

  • Kuhn, J.-T. & Kiefer, T. (2013). Optimal test assembly in practice: The design of the Austrian Educational Standards Assessment in Mathematics. Zeitschrift für Psychologie, 221, 190–200.

    Google Scholar 

  • Kühnhausen, J., Leonhardt, A., Dirk, J. & Schmiedek, F. (2013). Physical activity and affect in elementary school children’s daily lives. Frontiers in Movement Science and Sport Psychology, 4, 456.

    Google Scholar 

  • Landauer, T. K., Foltz, P. W. & Laham, D. (1998). Introduction to Latent Semantic Analysis. Discourse Processes, 25, 259–284.

    Google Scholar 

  • Larson, R. & Csikszentmihalyi, M. (1983). The experience sampling method. In H. T. Reis (Ed.), Naturalistic approaches to studying social interaction. New directions for methodology of social and behavioral sciences (pp. 41–56). San Francisco, CA: Jossey-Bass.

    Google Scholar 

  • Luecht, R. L. & Sireci (2011). A review of models for computer-based testing. Research report 2011–2012. New York, NY: The College Board.

    Google Scholar 

  • Margolis, M. J. & Clauser, B. E. (2006). A regression-based procedure for automated scoring of a complex medical performance assessment. In D. M. Williamson, I. I. Bejar & R. J. Mislevy (Eds.), Automated scoring of complex tasks in computer-based testing (pp. 123–168). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Masters, J. (2010). Automated Scoring of an Interactive Geometry Item: A Proof-of-Concept. Journal of Technology, Learning, and Assessment, 8. Retrieved from https://ejournals.bc.edu/index.php/jtla/article/view/1626 [20.12.2019]

  • Mead, A. D. & Drasgow, F. (1993). Equivalence of Computerized and Paper-and-Pencil Cognitive Ability Tests: A Meta-Analysis. Psychological Bulletin, 114, 449–458.

    Google Scholar 

  • Mislevy, R. J. (2013). Evidence-centered design for simulation-based assessment. Military Medicine, 178, 107–114.

    PubMed  Google Scholar 

  • Mislevy, R. J., Almond, R. G. & Lukas, J. F. (2003). A brief introduction to evidence-centered design (Research Report 03-16). Princeton, NJ: Educational Testing Service.

    Google Scholar 

  • Mislevy, R. J., Behrens, J. T., Dicerbo, K. E. & Levy, R. (2012). Design and discovery in educational assessment: Evidence-centered design, psychometrics, and educational data mining. Journal of Educational Data Mining, 4, 11–48.

    Google Scholar 

  • Mislevy, R. J., Steinberg, L. S., Breyer, F. J., Almond, R. G. & Johnson, L. (2002). Making sense of data from complex assessments. Applied Measurement in Education, 15, 363–389.

    Google Scholar 

  • Moosbrugger, H. (2011). Lineare Modelle. Regressions- und Varianzanalysen (4. Aufl.). Bern: Huber.

    Google Scholar 

  • Moosbrugger, H. & Goldhammer, F. (2007). FAKT-II. Frankfurter Adaptiver Konzentrationsleistungs-Test II. Computerprogramm. Grundlegend neu bearbeitete und neu normierte 2. Auflage des FAKT von Moosbrugger und Heyden (1997). Bern: Huber.

    Google Scholar 

  • Organisation for Economic Co-operation and Development (OECD). (2011). PISA 2009 results Vol VI: Students on line – Digital technologies and performance. Paris: PISA, OECD Publishing. Retrieved from https://doi.org/10.1787/9789264112995-en [20.12.2019]

    Book  Google Scholar 

  • Organisation for Economic Co-operation and Development (OECD). (2013). OECD Skills Outlook 2013: First Results from the Survey of Adult Skills. Paris: OECD Publishing.

    Google Scholar 

  • Organisation for Economic Co-operation and Development (OECD). (2014). PISA 2012 Results: What Students Know and Can Do – Student Performance in Mathematics, Reading and Science (Volume I, Revised edition, February 2014). Paris: PISA, OECD Publishing. Retrieved from https://doi.org/10.1787/9789264201118-en [20.12.2019]

    Book  Google Scholar 

  • Parshall, C. G., Harmes, J. C., Davey, T. & Pashley, P. J. (2010). Innovative item types for computerized testing. In W. J. van der Linden & C. A. W. Glas (Eds.), Elements of adaptive testing (pp. 215–230). New York, NY: Springer.

    Google Scholar 

  • Parshall, C. G., Spray, J. A., Kalohn, J. C. & Davey, T. (2002). Practical considerations in computer-based testing. New York, NY: Springer.

    Google Scholar 

  • Richman-Hirsch, W. L., Olson-Buchanan, J. B. & Drasgow, F. (2000). Examining the impact of administration medium on examinee perceptions and attitudes. Journal of Applied Psychology, 85, 880–887.

    PubMed  Google Scholar 

  • Reiß, S. & Moosbrugger, H. (2008) Online Self Assessment Psychologie. Institut für Psychologie der Goethe-Universität Frankfurt am Main. https://www.psychologie.uni-frankfurt.de/49829947/20_self-Assessment [20.12.2019]

  • Reips, U.-D. (2010). Design and formatting in Internet-based research. In S. Gosling & J. Johnson (Eds.), Advanced methods for conducting online behavioral research (pp. 29–43). Washington, DC: American Psychological Association.

    Google Scholar 

  • Robitzsch, A., Lüdtke, O., Köller, O., Kröhne, U., Goldhammer, F. & Heine, J. H. (2017). Herausforderungen bei der Schätzung von Trends in Schulleistungsstudien. Diagnostica, 63, 148–165.

    Google Scholar 

  • Rölke, H. (2012a). Automata and Petri Net Models for visualizing and analyzing complex questionnaires – A case study. In S. Donatelli, J. Kleijn, R. J. Machado & J. M. Fernandes (Eds.) CEUR Workshop Proceedings (Vol. 827, p. 317–329). Braga: CEUR-WS.org. Retrieved from https://www.researchgate.net/publication/220852463_Automata_and_Petri_Net_Models_for_Visualizing_and_Analyzing_Complex_Questionnaires_A_Case_Study [20.12.2019]

    Google Scholar 

  • Rölke, H. (2012b). The Item Builder: A graphical authoring system for complex item development. In T. Bastiaens & G. Marks (Eds), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education (pp. 344–353). Chesapeake, VA: AACE. Retrieved from http://www.editlib.org/p/41614 [20.12.2019]

    Google Scholar 

  • Russel, M. (2010). Technology-aided formative assessment and learning: New developments and applications. In H. L. Andrade & G. J. Cizek (Eds.), Handbook of formative assessment (pp. 125–138). New York, NY: Routledge.

    Google Scholar 

  • Russell, M. (2011). Computerized tests sensitive to individual needs. In S. N. Elliott, R. J. Kettler, P. A. Beddow & A. Kurz (Eds.), Handbook of accessible achievement tests for all students. (pp. 255–273). New York, NY: Springer.

    Google Scholar 

  • Saris, W. E. (1991). Computer-assisted interviewing. Newbury Park, Calif: Sage Publications.

    Google Scholar 

  • Scalise, K. (2012). Creating innovative assessment items and test forms. In R. W. Lissitz & H. Jiao (Eds.), Computers and their impact on state assessment: Recent history and predications for the future (pp. 134–156). Charlotte, NC: Information Age Publisher.

    Google Scholar 

  • Scalise, K. & Gifford, B. R. (2006). Computer-based assessment in E-learning: A framework for constructing “intermediate constraint” questions and tasks for technology platforms. Journal of Teaching, Learning and Assessment, 4(6), 1–45.

    Google Scholar 

  • Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones, S. M., Agrawal, M., et al (2013). Personality, gender, and age in the language of social media: The open-vocabulary approach. PLoS ONE 8: e73791. https://doi.org/10.1371/journal.pone.0073791

    Article  PubMed  PubMed Central  Google Scholar 

  • Seidel, T., Blomberg, G., Stürmer, K. (2010). „Observer“ – Validierung eines videobasierten Instruments zur Erfassung der professionellen Wahrnehmung von Unterricht. Zeitschrift für Pädagogik, 56, 296–306.

    Google Scholar 

  • Shah, D. V., Cappella, J. N. & Neuman, W. R. (2015). Big data, digital media, and computational social science: Possibilities and perils. Annals of the American Academy of Political and Social Science, 659, 6–13.

    Google Scholar 

  • Shermis, M. D. & Burstein, J. (Eds.), (2003). Automated essay scoring: A cross-disciplinary perspective. Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Sinharay, S. & Johnson, M. S. (2013). Statistical modeling of automatically generated items. In M. J . Gierl & T. Haladyna (Eds.), Automatic item generation: Theory and practice (pp. 183–195). New York, NY: Routledge.

    Google Scholar 

  • Sireci, S. G., Li, S. & Scarpati, S. (2005). Test accommodations for students with disabilities: An analysis of the interaction hypothesis. Review of Educational Research, 75, 457–490.

    Google Scholar 

  • Sireci, S. G. & Zenisky, A. L. (2006). Innovative item formats in computer-based testing: In pursuit of improved construct representation. In S. M. Downing & Th. M. Haladyna (Eds.), Handbook of test development (pp. 329–347). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Steinwascher, M. A. & Meiser, T. (2016). How a high working memory capacity can increase proactive interference. Consciousness and Cognition, 44, 130–145.

    PubMed  Google Scholar 

  • Stout, W. (2002). Test Models for Traditional and Complex CBTs. In C. Mills, M. Potenza, J. Fremer & W. Ward (Eds.), Computer-Based Testing (pp. 103–118). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Souvignier, E., Förster, N. & Salaschek, M. (2014). quop: ein Ansatz internet-basierter Lernverlaufsdiagnostik und Testkonzepte für Mathematik und Lesen. In M. Hasselhorn, W. Schneider & U. Trautwein (Hrsg.), Lernverlaufsdiagnostik: Test und Trends N. F. (Bd. 12, S. 239–256). Göttingen: Hogrefe.

    Google Scholar 

  • Tien, J. M. (2013). Big data: Unleashing information. Journal of Systems Science and Systems Engineering, 22, 127–151.

    Google Scholar 

  • Upsing, B., Goldhammer, F., Schnitzler, M., Baumann, R., Johannes, R., Barkow, I., Rölke, H., Jars, I., Latour, T., Plichart P., Jadoul, R., Henry, C. &, Wagner, M. (2013). Chapter 5: Development of the Cognitive Items. In Organisation for Economic Co-Operation and Development (OECD) (Ed.), Technical Report of the Survey of Adult Skills (PIAAC) (p. 148–156). Paris: OECD.

    Google Scholar 

  • Vale, C. D. (2006). Computerized item banking. In S. M. Downing & T. M. Haladyna (Eds.), Handbook of test development (pp. 261–285). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • van der Linden, W. J. (1998). Optimal assembly of psychological and educational tests. Applied Psychological Measurement, 22, 195–211.

    Google Scholar 

  • van der Linden, W. J. (2002). On complexity in CBT. In C. Mills, M. Potenza, J. Fremer & W. Ward (Eds.), Computer-Based Testing (pp. 89–102). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • van der Linden, W. J. (2005). Linear models of optimal test design. New York, NY: Springer.

    Google Scholar 

  • VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46, 197–221.

    Google Scholar 

  • Visible Learning Lab (2010). Educator manual. E-asTTle fitness for national standards. Retrieved from http://e-asttle.tki.org.nz/User-manuals [20.12.2019]

  • Waalkens, M., Aleven, V. & Taatgen, N. (2013). Does supporting multiple student strategies lead to greater learning and motivation? Investigating a source of complexity in the architecture of intelligent tutoring systems. Computers and Education, 60, 159–171.

    Google Scholar 

  • Wenzel, S. F. C., Engelhardt, L., Hartig, K., Kuchta, K., Frey, A., Goldhammer, F., Naumann, J. & Horz, H. (2016). Computergestützte, adaptive und verhaltensnahe Erfassung Informations- und Kommunikationstechnologie-bezogener Fertigkeiten (ICT-Skills) (CavE-ICT). In BMBF (Hrsg.). Forschung in Ankopplung an Large-Scale Assessments (S. 161–180). Bonn, Berlin: BMBF.

    Google Scholar 

  • Williamson, D. M., Mislevy, R. J. & Bejar, I. I. (Eds.). (2006). Automated scoring of complex tasks in computer-based testing. Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Wise, S. L., Finney, S. J., Enders, C. K., Freeman, S. A. & Severance, D. D. (1999). Examinee Judgments of Changes in Item Difficulty: Implications for Item Review in Computerized Adaptive Testing. Applied Measurement in Education, 12, 185–198.

    Google Scholar 

  • Zehner, F., Sälzer, C. & Goldhammer, F. (2016). Automatic Coding of Short Text Responses via Clustering in Educational Assessment. Educational and Psychological Measurement, 76, 280–303.

    PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Arbeitsbereich: Technologiebasiertes Assessment, Abt. Bildungsqualität und Evaluation, Deutsches Institut für Internationale Pädagogische Forschung, Rostocker Straße 6, 60323, Frankfurt am Main, Deutschland

    Prof. Dr. Frank Goldhammer & Dr. Ulf Kröhne

Authors
  1. Prof. Dr. Frank Goldhammer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dr. Ulf Kröhne
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Frank Goldhammer .

Editor information

Editors and Affiliations

  1. Institut für Psychologie, Goethe-Universität Frankfurt am Main, Frankfurt am Main, Hessen, Germany

    Helfried Moosbrugger

  2. Methodenzentrum, Eberhard Karls Universität Tübingen, Tübingen, Baden-Württemberg, Germany

    Augustin Kelava

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer-Verlag GmbH Deutschland, ein Teil von Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Goldhammer, F., Kröhne, U. (2020). Computerbasiertes Assessment. In: Moosbrugger, H., Kelava, A. (eds) Testtheorie und Fragebogenkonstruktion. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-61532-4_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-61532-4_6

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-61531-7

  • Online ISBN: 978-3-662-61532-4

  • eBook Packages: Psychology (German Language)

Publish with us

Policies and ethics

Search

Navigation