Educational Research Applications (ISSN: 2575-7032)

research article

  PDF Download

Seeking Quality and Attracting Candidates: Residency Program Directors’ Perceptions

Maribeth Porter, John Malaty, Daniel Rubin, Denny Fe Agana, Robert Hatch, Dean Seehusen, Peter JCarek*

Department of Community Health and Family Medicine, College of Medicine, University of Florida, Gainesville, FL, USA

*Corresponding author:Peter J Carek, Department of Community Health and Family Medicine, College of Medicine, University of Florida, Gainesville, FL, USA. Tel: +13522735159; Fax: +13522735213; Email: carek@ufl.edu

Received Date: 17 October, 2017; Accepted Date: 02 November, 2017; Published Date: 09 November, 2017

1.      Abstract

Introduction: Certain factors or characteristics have been associated with a quality family medicine residency program. Also, certain characteristics of a program are attractive to potential candidates. Due to their experience and expertise, residency program directors should be familiar with the characteristics that are associated with a quality program and which traits are most attractive to potential residents. Understanding this information should assist program directors in ensuring a quality residency program that is also attractive to potential candidates. In this study, we examinedwhich characteristics program directors associated with a quality residencyprogram as well as the characteristics they believe to be most desirable to residency applicants.

Methods: Data were collected as part of the 2015 CERA Family Medicine Residency Program Director survey. A list of sixteen residency program features was provided and directors were asked to rank the top five (5) items in terms of importance in determining the quality of a residency program and the top five (5) characteristics that are most desirable to residency applicants.

Results:The response rate was 49.2%.Program directors use accreditation status, faculty reputation for teaching, board pass rate, curriculum innovation, and perceived reputation of the program to determine the quality of a program. Board pass rate, curriculum innovation, faculty reputation for teaching and reputation of program were also found to be highly ranked characteristics for desirability to applicants.

Conclusion: Potentially modifiable factors such as board pass rate, curriculum innovation, and faculty reputation for teaching were found to be highly ranked characteristics for both quality of a program and desirability to applicants. These modifiable factors represent opportunities for program directors to improve the perceived quality of their program and attractiveness to students.              

1.      Introduction 

Residency programs seek to have both a quality educational program for residents and to have traits attractive to medical students. Severalfactors or characteristicsare perceived by medical students, residents, and physician faculty members to be associated with a quality or successful family medicine residency program. Characteristics such as Accreditation Council for Graduate Medical Education (ACGME) accreditation status and meeting standards established by the Association for Family Medicine Residency Directors in their Residency Program Index (RPI)have been suggested as markers of a quality family medicine residency program [1]. One study noted that residency program directors perceived successful programs as having an outstanding reputation and being regarded positively by other departments within the institution [2] 

Aprogram’s initial match rateisoftenused as a marker of its desirabilityto medical students and other potential candidates. Interestingly, this rate is often dependent upon factors not routinely associated with quality of education. For instance, geographic locationhas repeatedly been found to be the most commonly cited reason for selection of a particular program[3-9].Other program characteristics such as the number of positions in the first year and the degree to which a program depends on international medical graduates may also affect initial match rates[2-10].Yet the American Board of Family Medicine (ABFM) board pass rate, Review Committee for Family Medicine (RC-FM) of the ACGME accreditation status and cycle length, complementary curricular foci, additional opportunities for education through international experience, extended residency training through the P4 Project, and training beyond the program’s accredited length were not found to be associated with higher initial match rates[11]. 

Per Accreditation Council for Graduate Medical Education (ACGME) Program Requirements for Graduate Medical Education in Family Medicine, program directors are required to annually assess and identify interventions that will improve the quality of their program[12].Additionally, as they seek to attract desirable residency candidates and based upon their expertise and experience, these directors should be familiar with the characteristics of programs that are attractive to potential residents. Additional information regarding the perceived characteristics of a quality residency program as well as program characteristics felt to be desirable by potential candidates should assist program directors in meeting both these aims.

To date, the opinions of residency program directors regarding characteristics associated with high quality programs and the attributes they feel are important to potential candidates have not been examined. The aim of this study was to identifycharacteristics that program directors believe are associated with the quality of family medicine residency programs as well as the characteristics they believe to be most desirable to residency applicants. The similarities and differences in characteristics that are associated with perceived quality of a program and desirability to candidates were also reviewed. 

2.      Methods 

Data were collected for this study as part of the 2015 CERA Family Medicine Program Director survey. CERA is a joint initiative of four major academic family medicine organizations (Society of Teachers of Family Medicine, North American Primary Care Research Group, Association of Departments of Family Medicine, and Association of Family Medicine Residency Directors).The survey methods and sampledemographics have previouslybeen presented[13]. 

The survey was conducted between December 14, 2015and February 1, 2016withthree reminders sent out during this period. The email invitation included a personalized greeting, a letter signed by the presidents of each of the four participating organizations urging participation, and a link to the survey that utilized the online program SurveyMonkey®. The study was approved by the American Academy of Family Physicians Institutional Review Board. 

3.      Survey Items 

Respondents were asked demographic information about their program, including type, approximate size of the community, proportion of current residents in program who are graduates of non-US medical schools, year program began training residents, program director tenure and age. Program location was reported as within Puerto Rico, Canada, or one of the nine geographic regions defined by the US Census Bureau. Based upon the information provided, programs were categorized into one of three periods based upon the year they began training residents: before 1980, between 1980 and 2000, and after 2000. These periods were selected to represent approximately the first 10 years of family medicine residency programs with rapid growth, a middle period of sustained growth, and a more recent period representing new programs. 

A list of 16 residency program features identified from the literature and the experience of the authors that were found or felt to affect residency choice was provided: accreditation status, attrition rate, board pass rate, curriculum innovation, diversity of faculty and residents, faculty participation in national medical organizations, faculty reputation for teaching, faculty/resident record of publications and presentations, location, Patient Center Medical Home (PCMH) of the Family Medicine practice site, patient visits to the Family Medicine practice seen by residents (number), post-residency fellowships affiliated with the program, prior year(s) match rate, procedures taught to residents, program director participation in the National Institute for Program Director Development (NIPDD) and/or tenure of the program director, and perceived reputation of the program. Program directors were asked to rank the top five (5) items in terms of importance in determining the quality of a residency program. Additionally, the program directors were also asked to rank the top five (5) items in terms of program characteristics that are most desirable to residency candidates. The program directors were asked whether their program was currently meeting the ACGME requirements for ABFM board pass rate, currently meeting the ACGME requirements for Family Medicine Practice (FMP) site numbers, presence of an associated fellowship program, and whether the clinic associated with their program had recognition as a PCMH. 

Descriptive statistics were used to characterize and summarize demographic data. Frequencies indicated the ranking of each program characteristic in terms of both importance in determining the quality of a residency program and desirability to residency applicants. 

To obtain an overall ranking score for each characteristic, a point system was developed. A characteristic receiving a top or 1 ranking was assigned 5 points, a 2 ranking was assigned 4 points, a 3 ranking was assigned 3 points, a 4 ranking was assigned 2 points, and a 5 ranking was assigned 1 point. The total number of points assigned determined the characteristic’s overall rank.

To compare the level of agreement between the characteristics used to assess the quality of a residency program, a measure of inter-rater agreement (the kappa (κ) statistic) was used. Values of κ >0.75 indicate excellent agreement; values of 0.4 through 0.74 indicate good agreement; and values <0.4 show marginal agreement. 

Finally, program directors’ ranking of characteristics were compared based upon their response to the following questions: Is your program currently meeting the ACGME requirements for ABFM board pass rate (greater than 90% first time pass rate over last 5 years)?, Is your program currently meeting the ACGME requirements for FM site numbers (at least 1650 encounters over 3 years)?, Does your program have an associated fellowship program?, and Does the clinic associated with your program have recognition as a Patient Centered Medical Home (PCMH)?.  The Kruskal-Wallis test was used to compare distributions.Significance was defined as p<0.05 level of confidence. 

4.      Results 

The overall response rate was 49.2% with 227 out of 461 program directors responding. The responding directors mostly represented community-based, university-affiliated programs located in communities with a population greaterthan 150,000 and had less than 24% of the current residents as graduates of non-US medical schools (Table 1). Most programs began training residents before 1980 and most directors are male (61.9%). 

Accreditation status, faculty reputation for teaching, board pass rate, curriculum innovation, and perceived reputation of the program were highly ranked based upon the directors’ responses of characteristics they use to determine the quality of a program (Table 2). The lowest ranked characteristics associated with the quality of a program were post-residency fellowship, location, program director participation in NIPDD and/or tenure, faculty participation in national medical organizations, and faculty/resident record of publication and presentations.

Location, perceived reputation of the program, curriculum innovation, faculty reputation of teaching, and board pass rate were highly ranked as characteristics program directors felt were desirable to candidates. The lowest ranked characteristics felt to be desirable to applicants were program director participation in NIPPD and/or tenure, faculty participation in national medical organizations, faculty/resident record of publication and presentations, PCMH status, and patient visits.

For characteristics that were ranked with a frequency as to allow for analysis, accreditation status, board pass rate, curriculum innovation, diversity of faculty and residents, faculty reputation for teaching, and reputation of program were characteristics that had marginal agreement with regards to being used as a marker of quality of the program as well as being desirable to applicants. Attrition rate, faculty participation in national medical organizations, faculty/resident record of publication and presentations, location, PCMH status, post-residency fellowship, procedures taught, and program director participation in NIPPD and/or tenure did not receive sufficient ranking to allow a meaningful comparison.

No statistically significantdifference was noted when comparing rankings based upon whether a program met ACGME requirements for ABFM board pass rate, met ACGME requirements for FMC numbers, had an associated fellowship program, or hadclinic recognition as a PCMH.

5.      Discussion

Program directors ranked several characteristics and factors in their assessment of the quality of a residency program. Board pass rate serves as a measurable characteristic of a program and can be easily used to compare programs, especially as these rates are publicly available on the ABFM website. While accreditation status is a characteristic highly associated with program quality, asignificant majority of family medicine residency programs receive full accreditation such that this characteristic may have little if any influence to assess overall quality of a program. Furthermore, ACGME program requirements for graduate medical education in Family Medicine are meant to ensure basic training in the specialty such that accreditation serves as a threshold for programs to meet and does not allow assessment of quality beyond a program simply meeting these standards.

While marginal agreement was noted between rankings of several characteristics for both program quality and desirability to applicants, several characteristics were ranked in the upper tertile for both quality and desirability: board pass rate, curriculum innovation, faculty reputation for teaching, and reputation of program. The difference in the high rankings consisted of accreditation status being ranked highest for program quality and location (geographic) being ranked highest for desirability to applicants. 

The results of this study should assist program directors and facultyin their assessment of their programs as they seek to modify and improve their educational activities and address issues as much as possible to enhance their ability to recruit highly qualified applicants. While understanding that location is the major factor in terms of desirability to applicants, the other characteristics associated with both program quality and desirability to applicants can be addressed by changing the activities and environment of a program. The specific methods used to change other characteristics such as curriculum innovation, faculty reputation for teaching and program reputation, require additional study. 

Scholarly and similar activities such as post-residency fellowship, program director participation in NIPDD and/or tenure, faculty participation in national medical organizations, and faculty/resident record of publication and presentations were not associated with the quality of a program. While these characteristics would appear to improve the reputation of faculty and program, their low rankings do not support this assumption.

Certain limitations of this study are noted. First, though the questionnaire was distributed to the entire population of program directors, we were not able to obtain a 100% response rate.  The effect of non-respondents upon the results would have been variable and probably would not have altered our findings. In addition, although program directors were asked to rank a fairly long list of characteristics associated with the quality of a program or felt to be desirable to candidates, some important factors may have been inadvertently omitted from the list. Finally, program directors may not know with a high degree of accuracy what attracts students to a particular program. Further study by surveying medical students regarding characteristics that attract them to specific residency programs would either validate or refute the opinions of the directors. 

In conclusion, potentially modifiable factors that include accreditation status, faculty reputation for teaching, board pass rate, curriculum innovation and reputation of a program were ranked highly by responding directors as characteristics associated with the quality of a residency program. While location was ranked highest in terms of a program’s desirability to applicants, several modifiable factors such as board pass rate, curriculum innovation, faculty reputation for teaching and reputation of program were found to be highly ranked characteristics for desirability to applicants. These modifiable characteristics and factors represent opportunities for program directors to improve the perceived quality of their program and attractiveness to students.



 

CHARACTERISTIC

 

n (%)

Type of Program

University-Based

40 (17.6)

Community-Based, University-Affiliated

149 (65.6)

Community-Based, Non-Affiliated

29 (12.8)

Military

9 (4.0)

Location (regional)

Puerto Rico

1 (0.4)

New England (NH, MA, ME, VT, RI, or CT)

11 (4.9)

Middle Atlantic (NY, PA, or NJ)

35 (15.6)

South Atlantic (FL, GA, SC, NC, VA, DC, WV, DE, or MD)

37 (16.4)

East South Central (KY, TN, MS, or AL)

13 (5.8)

East North Central (WI, MI, OH, IN, or IL)

29 (12.9)

West South Central (OK, AR, LA, or TX)

20 (8.9)

West North Central (ND, MN, SD, IA, NE, KS, or MO)

25 (11.1)

Mountain (MT, ID, WY, NV, UT, AZ, CO, or NM)

16 (7.1)

Pacific (WA, OR, CA, AK, or HI)

38 (16.9)

Community size

>30,000

12 (5.4)

30,000 to 74,999

40 (17.9)

75,000 to 149,000

41 (18.3)

150,000 to 499,999

62 (27.7)

500,000 to 1 million

35 (15.6)

>1 million

34 (15.2)

Current residents in program that are graduates of non-US medical schools

0-24%

122 (54.0)

25-49%

43 (19.0)

50-74%

28 (12.4)

75-100%

32 (14.2)

Don’t know

1 (0.4)

Year program began training residents

 

Before 1980

146 (65.5)

1980-2000

50 (22.4)

After 2000

27 (12.1)

Tenure as program director

<5 years

110 (48.9)

>5 years

115 (51.1)

Gender of program director

Female

86 (38.1)

Male

140 (61.9)

Other

 

ACGME ABFM board pass rate >90%

178 (86.0)

FMP site patient visits >1,650

199 (94.3)

Associated fellowship program

108 (50.9)

Clinic recognized as PCMH

141 (66.8)

 

Table 1: Program characteristics of responding directors.

 

Rank

QUALITY

(TOTAL SCORE)

DESIRABILITY TO APPLICANTS (TOTAL SCORE)

1

Accreditation Status (578)

Location (geographic) (706)

2

Faculty reputation for teaching (447)

Reputation of Program (569)

3

Board Pass Rate (444)

Curriculum Innovation (333)

4

Curriculum Innovation (393)

Faculty reputation for teaching (299)

5

Reputation of Program (276)

Board Pass Rate (290)

6

Diversity of Faculty and Residents (189)

Procedures taught to residents (277)

7

Patient visits (174)

Accreditation Status (251)

8

PCMH Status (160)

Diversity of Faculty and Residents (167)

9

Procedures taught to residents (118)

Prior year match rate (71)

10

Attrition Rate (81)

Post-residency fellowship (44)

11

Prior year match rate (68)

Attrition Rate (29)

12

Faculty participation in national medical organizations (43)

PCMH Status (28)

13

Faculty/resident record of publication and presentations (43)

Patient visits (28)

14

Program director participation in NIPDD and/or Tenure (40)

Faculty/resident record of publication and presentations (9)

15

Location (geographic) (25)

Faculty participation in national medical organizations (5)

16

Post-residency fellowship (11)

Program director participation in NIPDD and/or Tenure (0)

 

Table 2: Overall ranking of program characteristics associated with quality and desirability to applicants.

 

 

Rank

Quality

Desirable

Kappa*

Accreditation Status

 

1

77

27

0.26

 

2

31

14

 

3

13

9

 

4

7

9

 

5

16

15

Board Pass Rate

 

1

26

9

0.18

 

2

33

24

 

3

35

24

 

4

30

24

 

5

17

29

Curriculum Innovation

 

1

25

12

0.09

 

2

26

22

 

3

35

32

 

4

18

35

 

5

23

19

Diversity of Faculty and Residents

 

1

12

3

0.28

 

2

11

13

 

3

13

15

 

4

17

20

 

5

12

15

Faculty reputation for teaching

 

1

27

8

0.08

 

2

46

14

 

3

21

43

 

4

26

26

 

5

13

22

Reputation of Program

 

1

26

41

0.04

 

2

13

60

 

3

18

18

 

4

9

24

 

5

22

22

*Kappa statistic and Level of agreement: >0.75 - Excellent; between 0.4 and 0.75 - Good; <0.4 - Marginal

 

Table 3: Level of agreement between program directors ranking based upon characteristics associated with program quality versus desirability to applicants.

1.       Hoekzema GS, Maxwell L, Gravel JW, Mills WW, Geiger W (2014) The Residency Performance Index: An Effort at Residency Quality Assessment and Improvement in Family Medicine. J Grad Med Ed 6:756-759.

2.       Kikano GE, Galazka SS, Flocke SA, Saffran E, Zyzanski SJ(1994) Markers of successful recruitment of students to family practice residency programs. Fam Med 26:492-496.

3.       Simmonds AC, Robbins JM, Brinker MR, Rice JC, Kerstein MD (1990) Factors important to students in selecting a residency program. Acad Med65:640-643.

4.       Mayeaux EJ, Arnold J (1993) Why first-year family practice residents choose their residency programs. Fam Med 25:253-256.

5.       Nagler A, Andolsek K, Schlueter J, Weinerth J(2012) To match or not: factors influencing resident choice of graduate medical education program. J Grad Med Edu 4:159-164.

6.       Nuthalapaty FS, Jackson JR, Owen J (2004) The influence of quality-of-life, academic, and workplace factors on residency program selection. Acad Med 79:417-425.

7.       Flynn TC, Gerrity MS, Berkowitz LR (1993) What do applicants look for when selecting internal medicine residency programs? J Gen Intern Med 8: 249-254.

8.       Hitchcock MA, Kreis SR, Foster BM (1989) Factors influencing student selection of family practice residency programs in Texas. Fam Med 21:122-126.

9.       DeSantis M, Marco CA (2005) Emergency medicine residency selection: Factors influencing candidate decisions. AcadEmerg Med 12:559-561.

10.    Love JN, Howell JM, Hegarty CB, McLaughlin SA, Coates WC, et al. (2012) Factors that influence medical student selection of an emergency medicine residency program: implications for training programs. AcadEmerg Med 19:455-460.

11.    DeLisa JA, Jain SS, Campagnolo D, McCutcheon PH (1992) Selecting a physical medicine and rehabilitation residency. Am J Phys Med Rehab 71:72-76.

12.    Mims L, Mainous AG III, Chirina S, Carek PJ (2014) Do ABFM board pass rates or additional curricular activities affect match rates for family medicine residency programs? Fam Med 46:276-281.

13.    Accreditation Council for Graduate Medical Education.ACGME Program Requirements for Graduate Medical Education in Family Medicine.

14.    Mainous AG III, Seehusen D,Shokar N (2012) CAFMEducational Research Alliance (CERA) 2011Residency Director Survey: background, methods,and respondent characteristics. Fam Med 44:691-693.

Copyright and Licensing: This is an Open Access Journal Article Published Under Attribution-Share Alike CC BY-SA: Creative Commons Attribution-Share Alike 4.0 International License. With this license readers can share, distribute, download, even commercially, as long as the original source is properly cited. Read More.

   

share article