Skip to main content
SpringerLink
Log in

Factors Predictive of Performance on USMLE Step 1: Do Commercial Study Aids Improve Scores?

  • Original Research
  • Published:
Medical Science Educator Aims and scope Submit manuscript

Abstract

Background

The United States Medical Licensing Examination (USMLE) Step 1 was the most cited factor in granting medical student applicants’ residency interviews in a 2016 NRMP survey. Medical students utilize numerous commercial resources to achieve the highest score possible. This study sought to examine preparatory resources and their association with USMLE Step 1 performance.

Methods

In 2016 and 2017, 170 medical students from a public, Midwestern medical school completed a voluntary survey after completing USMLE Step 1. The survey asked about study aids, anticipated performance, test preparation time, and practice exam performance. Actual scores and academic performance (overall percent for years 1 and 2 of medical school) were matched to survey responses. Data was analyzed using descriptive and inferential statistical analyses.

Results

USMLE Step 1 performance was associated with academic performance (r = 0.719, p < 0.01). Post hoc tests indicated a positive difference with the UWorld Qbank and exam performance (F4,134 = 10.29, p < 0.05). Amount of days dedicated to test preparation was negatively correlated with performance in the 2017 student cohort (r = − 0.304, p < 0.01). The use of other commercially available products and the number of practice tests completed were not correlated with performance.

Discussion

Cumulative pre-clinical grades had the strongest association with performance on the USMLE Step 1. Completing UWorld Qbank at least once was positively related with Step 1 scores. However, the use of other study aids, completing multiple practice exams, and delaying exam dates were not statistically significant. This study indicates performance in the pre-clinical curriculum and increased usage of UWorld are strongly associated with USMLE Step 1 performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. National Resident Matching Program, Data Release and Research Committee: Results of the 2016 NRMP program director survey. 2016. National Resident Matching Program: Washington, DC.

  2. National Resident Matching Program, Results and Data: 2017 Main residency match®. 2017. National Resident Matching Program: Washington, DC.

  3. Gauer JL, Jackson JB. The association of USMLE Step 1 and Step 2 CK scores with residency match specialty and location. Med Educ Online. 2017;22(1):1358579.

    Article  Google Scholar 

  4. Loh AR, Joseph D, Keenan JD, Lietman TM, Naseri A. Predictors of matching in an ophthalmology residency program. Ophthalmology. 2013;120(4):865–70.

    Article  Google Scholar 

  5. Baños JH, Pepin ME, Van Wagoner N. Class-wide access to a commercial Step 1 question bank during preclinical organ-based modules: a pilot project. Acad Med. 2018;93(3):486–90.

    Article  Google Scholar 

  6. Bonasso P, Lucke-Wold B III, Reed Z, Bozek J, Cottrell S. Investigating the impact of preparation strategies on USMLE Step 1 performance. MedEdPublish. 2015;4(1):5.

    Google Scholar 

  7. Giordano C, Hutchinson D, Peppler R. A predictive model for USMLE Step 1 scores. Cureus. 2016;8(9):e769.

    Google Scholar 

  8. Zhang C, Rauchwarger A, Toth C, O’Connell M. Student USMLE step 1 preparation and performance. Adv Health Sci Educ Theory Pract. 2004;9(4):291–7.

    Article  Google Scholar 

  9. Burk-Rafel J, Santen SA, Purkiss J. Study behaviors and USMLE Step 1 performance: implications of a student self-directed parallel curriculum. Acad Med. 2017;92(11S):S67–74.

    Article  Google Scholar 

  10. Kumar AD, Shah MK, Maley JH, Evron J, Gyftopoulos A, Miller C. Preparing to take the USMLE Step 1: a survey on medical students’ self-reported study habits. Postgrad Med J. 2015;91(1075):257–61.

    Article  Google Scholar 

  11. Schwartz LF, Lineberry M, Park YS, Kamin CS, Hyderi AA. Development and evaluation of a student-initiated test preparation program for the USMLE Step 1 examination. Teach Learn Med. 2018;30(2):193–201.

    Article  Google Scholar 

  12. Strowd RE, Lambros A. Impacting student anxiety for the USMLE Step 1 through process-oriented preparation. Med Educ Online. 2010;15.

  13. Lee MW, Johnson TR, Kibble J. Development of statistical models to predict medical student performance on the USMLE Step 1 as a catalyst for deployment of student services. Med Sci Educ. 2017;27:663–71.

    Article  Google Scholar 

  14. Lee HS, Ahn D. Testing prepares students to learn better: the forward effect of testing in category learning. J Educ Psychol. 2018;110(2):203–17.

    Article  Google Scholar 

  15. Grabovsky I, Johnson GR, Schmader KE, Levin MJ, Zhang JH, Loooney DJ, et al. The relationship between performance on the Infectious Diseases In-Training and Certification Examinations. Clin Infect Dis. 2015;60(5):677–83.

    Article  Google Scholar 

  16. Guffey RC, Rusin K, Chidiac EJ, Marsh M. The utility of pre-residency standardized tests for anesthesiology resident selection: the place of United States Medical Licensing Examination scores. Anesth Analg. 2011;112(1):201–6.

    Article  Google Scholar 

  17. Harmouche E, Goyal N, Pinawin A, Nagarwala J, Bhat R. USMLE scores predict success in ABEM initial certification: a multicenter study. West J Emerg Med. 2017;18(3):544–9.

    Article  Google Scholar 

  18. McCaskill QE, Kirk JJ, Barata DM, Wludyka PS, Zenni EA, Chiu TT. USMLE step 1 scores as a significant predictor of future board passage in pediatrics. Ambul Pediatr. 2007;7(2):192–5.

    Article  Google Scholar 

  19. Miller BJ, Sexson S, Shevitz S, Peeples D, Van Sant S, McCall WV. US Medical Licensing Exam scores and performance on the Psychiatry Resident In-Training Examination. Acad Psychiatry. 2014;38(5):627–31.

    Article  Google Scholar 

  20. Puscas L, Chang CWD, Lee HJ, Diaz R, Miller R. USMLE and otolaryngology: predicting board performance. Otolaryngol Head Neck Surg. 2017;156(6):1130–5.

    Article  Google Scholar 

  21. Spellacy WN, Downes KL. United States Medical Licensing Examination scores as a predictor of performance on the annual council of resident education in obstetrics and gynecology examinations. J Reprod Med. 2014;59(1–2):17–9.

    Google Scholar 

  22. Sutton E, Richardson JD, Ziegler C, Bond J, Burke-Poole M, McMasters KM. Is USMLE Step 1 score a valid predictor of success in surgical residency? Am J Surg. 2014;208(6):1029–34 discussion 1034.

    Article  Google Scholar 

  23. Welch TR, Olson BG, Nelsen E, Beck Dallaghan GL, Kennedy GA, Botash A. United States Medical Licensing Examination and American Board of Pediatrics Certification Examination results: does the residency program contribute to trainee achievement. J Pediatr. 2017;188:270–274 e3.

    Article  Google Scholar 

  24. Prober CG, Kolars JC, First LR, Melnick DE. A plea to reassess the role of United States Medical Licensing Examination Step 1 Scores in residency selection. Acad Med. 2016;91(1):12–5.

    Article  Google Scholar 

  25. Alterman DM, Jones TM, Heidel RE, Daley BJ, Goldman MH. The predictive value of general surgery application data for future resident performance. J Surg Educ. 2011;68(6):513–8.

    Article  Google Scholar 

  26. Yousem IJ, Liu L, Aygun N, Yousem DM. United States Medical Licensing Examination Step 1 and 2 scores predict neuroradiology fellowship success. J Am Coll Radiol. 2016;13(4):438–444 e2.

    Article  Google Scholar 

  27. Lee M, Vermillion M. Comparative values of medical school assessments in the prediction of internship performance. Med Teach. 2018;1:1–6.

    Google Scholar 

Download references

Acknowledgments

Drs. Pachunka and Parry completed this work as medical students and were co-leads on this project.

Author information

Authors and Affiliations

  1. Anesthesiology-Pediatrics Residency Program, University of Pittsburgh Medical Center, Pittsburgh, PA, USA

    Stephanie Parry

  2. Internal Medicine-Pediatrics Residency Program, University of Nebraska Medical Center, Omaha, NE, USA

    Joseph Pachunka

  3. UNC School of Medicine, 311 Berryhill Hall, Chapel Hill, NC, 27599, USA

    Gary L. Beck Dallaghan

Authors
  1. Stephanie Parry
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joseph Pachunka
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gary L. Beck Dallaghan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gary L. Beck Dallaghan.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflicts of interest.

Ethical Approval

Ethical approval was deemed unnecessary because the study was designed for quality improvement.

Informed Consent

This was a quality improvement project that did not require informed consent according to the institutional review board.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Previous Presentations

An oral presentation was made at the AAMC Central Group on Educational Affairs Annual Meeting on March 22, 2018, in Rochester, MN.

Appendix 1. USMLE Step I Follow Up Survey

Appendix 1. USMLE Step I Follow Up Survey

This survey was created in SurveyMonkey. Scaled items or text box notations indicate how data was collected.

Your responses to the following questions will be provided to future students as they prepare for USMLE Step I. Your responses will be helpful in identifying which resources you found useful and which may not be worth the investment.

  1. 1.

    When did you start reviewing specifically for Step I?

    • After the last core of the M2 year ended

    • During or after spring break (before the last core ended)

    • During the holiday break or piro to spring break M2 year

    • During the first semester of M2 year

  2. 2.

    Please indicate which item(s) you used that were most helpful and which were least useful: (Scale: Most Useful; Lease Useful; Did not use)

    [List of commercial products available to medical students.]

  3. 3.

    How did you decided on a date range to take the exam? Text box

  4. 4.

    Did you have difficulty getting your preferred examination date? Yes No

    If Yes, when did you submit your request?

  5. 5.

    Please estimate the amount of time (in hours/week) that you studied:

    Individually: Text box

    Small groups: Text box

  6. 6.

    What advice would you give students to best prepare for the exam? Text box

  7. 7.

    What was the range of scores you expected?

    • < 192

    • 192–200

    • 201–210

    • 211–220

    • 221–230

    • 231–240

    • 241–250

    • > 250

    • No expectation

  8. 8.

    What was your target score?

  9. 9.

    What was your raw percentage on UWorld?

  10. 10.

    What was your UWorld percentile listed based on questions you answered?

  11. 11.

    How many times did you go through UWorld?

    • Completed a few questions

    • Completed about half the questions

    • Completed almost all of the questions

    • Completed all of the questions one time

    • Completed most/all of the questions twice

  12. 12.

    How many times did you go through First Aid?

    • Some

    • Half

    • Most Once

    • More than once

  13. 13.

    How many times did you go through Pathoma?

    • Some

    • Half

    • Most

    • Once More than once

  1. 14.

    Please indicate the number of practice exams you took:

    NBME: Text box

    UWorld: Text box

    Other: Text box

For each practice exam you have taken, please answer the questions:

  1. 15.

    NBME Practice Exam

    Form Number: Text box

    Estimated Step I score for this test: Text box

    Number of weeks taken before actual exam: Text box

    UWorld Practice Exam.

    Form Number: Text box.

    Estimated Step I score for this test: Text box.

    Number of weeks taken before actual exam: Text box.

    Other Practice Exam

    Practice test name: Text box

    Form Number: Text box

    Estimated Step I score for this test: Text box

    Number of weeks taken before actual exam: Text box

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Parry, S., Pachunka, J. & Beck Dallaghan, G.L. Factors Predictive of Performance on USMLE Step 1: Do Commercial Study Aids Improve Scores?. Med.Sci.Educ. 29, 667–672 (2019). https://doi.org/10.1007/s40670-019-00722-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40670-019-00722-4

Keywords

Search

Navigation