Summary
Objective: To compare checklists against global ratings for student performance on each station in an OSCE without the confounder of the global rating scorer having first filled in the checklist. Method: Subjects were 96 medical students completing their preclinical studies, who took an 8 station clinical OSCE. 39 students were assessed with detailed performance checklists; 57 students went through the same stations but were assessed using only a single global rating per station. A subset of 39 students were assessed by two independent raters. Results: Inter-rater and inter-station reliability of the global rating was the same as for the checklist. Correlation with a concurrent multiple choice test was similar for both formats. Conclusion: The global rating was found to be as reliable as more traditional checklist scoring. A discussion of the validity of checklist and global scores suggests that global ratings may be superior.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Vleuten CPM van der, Norman GR, de Graaff ED. Pitfalls in the pursuit of objectivity: issues of reliability. Medical Education 1991;25:119–126.
Norcini JJ, Diserens D, Day SC, Cebul RC, Schwartz JS, Beck LH, et al. The scoring and reproducibility of an essay test of clinical judgement. Academic Medicine 1990;65:S41–42.
Luijk SJ van, Vleuten CPM van der. A comparison of checklists and rating scales in performance-based testing. In: Hart IR, Harden RM, Des Marchais J, editors. Current developments in assessing clinical competence. Montreal: Can-Heal Publications, 1992;357–362.
Cohen R, Rothman AI, Poldre P, Ross J. Validity and generaliza-bility of global ratings in an objective structured clinical examination. Academic Medicine 1991;66:545–548.
Cohen DS, Colliver JA, Marcy MS, Fried ED, Swartz MH. Psychometric properties of a standardized-patient checklist and rating-scale form used to assess interpersonal and communication skills. Academic Medicine 1996;71:S87–89.
Blake JM, Norman GR, Smith EKM. Report card from McMaster: Student evaluation at a problem based medical school. Lancet 1995;345:899–902.
Wainer H. Estimating coefficients in linear models: It don’t make no nevermind. Psycho logical Bulletin 1976;83:213–217.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1997 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Cunnington, J.P.W., Neville, A.J., Norman, G.R. (1997). The Risks of Thoroughness: Reliability and Validity of Global Ratings and Checklists in an OSCE. In: Scherpbier, A.J.J.A., van der Vleuten, C.P.M., Rethans, J.J., van der Steeg, A.F.W. (eds) Advances in Medical Education. Springer, Dordrecht. https://doi.org/10.1007/978-94-011-4886-3_41
Download citation
DOI: https://doi.org/10.1007/978-94-011-4886-3_41
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-010-6048-6
Online ISBN: 978-94-011-4886-3
eBook Packages: Springer Book Archive