AMPJ Journal

We substantially improved the exam scores of our anesthesiology residents over the past four years and are sharing how we did this. An improvement was necessary. During the decade preceding 2021, numerous residents scored below the 50th percentile on their American Board of Anesthesiology In-Training Examinations (ITE), and five failed their initial board certification exam. The West Virginia University (WVU) anesthesiology residency program graduates nine residents annually and supplies the state with most of its anesthesiologists.

Department leaders requested educational improvements in 2021 to raise resident ITE scores above the 50th percentile and for each resident to pass the board certification exam on their first try. Leaders focused on the ITE because of its unbiased evaluation of anesthesia knowledge. (https://www.theaba.org/trainingprograms/in-training-examinations/)Investigators have correlated higher ITE scores with improved clinical performance and higher board certification pass rates [1,2]. Improving resident test scores became a departmental goal.

We identified successful educational techniques from published studies, incorporated many into the WVU program, observed the ITE results, and made further changes. We are reporting the twenty changes we found most successful. The Figure 1 shows how the eight 2025 CA3 residents scored well above the 90^th percentile, with three achieving perfect ITE scores of 50. These results are the best in our program history and apparently in the United States.

Figure 1: 2025 ITE results. All residents are scoring above the 90th percentile by their CA2 year and several CA3 residents achieve perfect scores of 50.

The Changes:

1. Dedicate one day per week to resident lectures and workshops, with each resident spending half that workday attending them.
2. Create a “Question of the Day” program where residents receive 50 monthly questions from the Anesthesiology Question Bank, True Learn, and motivate resident participation with the philosophy that final success is the sum of small efforts repeated daily.
3. Incentivize residents to study for their ITE, with $50 rewarded for each percentile scored above the 50th.
4. Assess each resident with a 150-question mock written board exam two months before the ITE, and use the results to tailor a study plan that includes practice questions, flashcards, podcasts, and/or textbooks.
5. Schedule two months of board review led by faculty members who took their certification exams during the previous decade.
6. Appoint junior and senior residents as education liaisons to coordinate educational sessions and review resident experiences with faculty members.
7. Identify enthusiastic and effective faculty teachers for resident teaching cases. Reward these faculty members with nonclinical educational days.
8. Start lectures in the PGY1/Clinical Base Year that cover the American Board of Anesthesiology Content Outline.
9. Assign residents to surgeries to maximize their educational opportunities.
10. Provide regular anonymous feedback from residents to faculty members about their teaching styles and effectiveness.
11. Educate faculty members on how to provide effective practice oral board exams. Reward participating faculty with nonclinical time.
12. Provide biannual high-fidelity mock oral board exams and objective structured clinical exams for residents beginning in their first clinical year.
13. Recruit a department chair with a known educational focus. Add a vice chair for education with 40 percent non-clinical time.
14. Support resident attendance at education conferences, including the American Society of Anesthesiologists Annual, Legislative, and Advance meetings.
15. Schedule resident sessions in a state-of-the-art simulation center throughout their clinical training.
16. Support several faculty members as American Board of Anesthesiology examiners.
17. Hold participatory workshops on point-of-care ultrasound (POCUS) exams, perfused cadaver and animal dissections, and awake intubation techniques.
18. Encourage faculty members to achieve POCUS certification and to mentor residents on ultrasound examination techniques.
19. Convert one weekly grand-rounds session each month to a mentored resident case discussion.
20. Revise resident rotations based on resident evaluations.

We confirmed that individual residents prefer learning differently [3]. This meant including residents in decision-making and adding multiple educational techniques [4]. The iterative educational changes led eventually to the outstanding resident test scores.

We adapted our educational additions from known successful improvements at other programs. Incentives can improve ITE exam performance [5]. One anesthesiology training program, for instance, rewarded high-scoring residents with moonlighting privileges and payment for their written board examination, which decreased the number of low-scoring residents by 80 percent [6]. Numerous programs found that residents who studied from question banks improved their ITE scores [7]. Some programs found success with a half-day resident academic program [8], multimodality remediation programs [8], trainee mentoring [9], mock board exams [10], and analysis of residents’ exam performances to identify their strengths and weaknesses [11]. One study demonstrated that integrating perioperative POCUS into a cardiac anesthesia rotation improved resident cognitive and technical skills [12,13].

The outstanding results of our educational changes resulted from designating high ITE scores as a goal and making multiple changes over four years to achieve them. Other residency programs may want to use some of the listed techniques.

References

1. McClintock JC, Gravlee GP (2010) Predicting success on the certification examinations of the American Board of Anesthesiology. Anesthesiology 112: 212-219.
2. McCrary HC, Colbert-Getz JM, Poss WB, Smith BK (2021) A Systematic Review of the Relationship Between In-Training Examination Scores and Specialty Board Examination Scores. J Grad Med Educ 13: 43-57.
3. Rebel A, Dilorenzo A, Isaak R, Mcgrane S, Moran KR, et al. (2018) Replicating an Educational OSCE Project for Skill Assessment of Junior Anesthesiology Residents at Multiple Institutions: A Qualitative Description. J Educ Perioper Med 20: E622.
4. Wolpaw JT, Uhlig E, Isaac GR, Dwivedi P, Lekowski RW, et al. (2020) Anesthesia Learning in the Digital Age: Are Program Directors and Residents on the Same Page? J Educ Perioper Med 22: E643.
5. Fowler JG, VanEenenaam DP Jr, Johnson KN, Traunero JR, Reynolds JE (2020) Sticks or Carrots? How an Easy-to-Implement Incentive Plan Improved Our Performance on the In-training Exam. J Educ Perioper Med 22: E647.
6. Joseph JA, Terry CM, Waller EJ, Bortsov AV, Zvara DA, et al. (2014) Enhancement of anesthesiology in-training exam performance with institution of an academic improvement policy. J Educ Perioper Med 16: E072.
7. Flentje AO, Caturegli I, Kavic SM (2020) Practice Makes Perfect: Introducing a Question Bank for ABSITE Preparation Improves Program Performance. J Surg Educ 77: 54-60.
8. Latif A, Ahsan MJ, Jaffri A, Kapoor V, Tran A, et al. (2022) A new dawn in internal medicine education curriculum-a project by the residents, with the residents, for the residents. Proc (Bayl Univ Med Cent) 35: 798-801.
9. Cheun TJ, Davies MG (2020) Improving ABSITE scores - A metaanalysis of reported remediation models. Am J Surg 220: 1557-1565.
10. Ong J, Swift C, Magill N, Ong S, Day A, et al. (2018) The association between mentoring and training outcomes in junior doctors in medicine: an observational study. BMJ Open 8: e020721.
11. Scott NP, Martin TW, Schmidt AM, Shanks AL (2023) Impact of an Online Question Bank on Resident In-Training Exam Performance. J Med Educ Curric Dev 10: 23821205231206221.
12. Karimian M, Rahmatizadeh S, Kohzadi Z, Kohzadi Z, Madadi F, et al. (2024) Analyzing Patterns in Anesthesiology Residents’ Exam Performance Using Data Mining Techniques. Anesth Pain Med 14:e151686.
13. Sanders JA, Navas-Blanco JR, Yeldo NS, Han X, Guruswamy J, et al. (2019) Incorporating Perioperative Point-of-Care Ultrasound as Part of the Anesthesia Residency Curriculum. J Cardiothorac Vasc Anesth 33: 2414-2418.