U.s. Dermatology Faculty Evaluation by Residents: Results of a National Survey

Although the Accreditation Council for Graduate Medical Education (ACGME) requires regular resident evaluation of faculty, little is known about the process. The program directors of 112 ACGME-accredited dermatology residency programs were mailed a twenty-item survey and asked to return a blank copy of any evaluation form(s) used to evaluate faculty at their program. Eighty-seven percent returned the completed survey (97/112) and 48% (54/112) returned copies of the faculty evaluation forms used. Approximately two-thirds of respondents were residency directors (63%) and one-third department chairs or section heads (33%). Ninety-six percent reported that residents evaluated dermatology faculty, and the vast majority (92%) had residents evaluate faculty anonymously. Evaluations were most often a paper form distributed to resident mailboxes (33%) or an electronic form sent to resident email accounts (29%). Only 4 programs used different forms to evaluate faculty in different dermatology subspecialties (e.g. dermatopathology vs pedi-atric dermatology). The most common components of the evaluations asked residents to rate faculty teaching (50/54), to fill in any additional comments about the faculty member (50/54), and to rate faculty availability/accessibility (40/54). Faculty evaluations by residents were most frequently regarded as somewhat important (67%) or very important (18%) for faculty promotion. Survey respondents' additional comments cited the increasing importance of faculty evaluations in influencing promotions and improving teaching, the critical necessity for assuring residents' anonymity, and concern regarding the possible negative impact of invalid assessments.


INTRODUCTION
Resident evaluations of faculty remain the most prevalent and consistent method used to assess teaching effectiveness [1,2].Because faculty evaluations enhance teaching and improve faculty performance [1,3] the Accreditation Council for Graduate Medical Education (ACGME) requires the following [4]: At least annually, the program must evaluate faculty performance, as it relates to the educational program.

•
These evaluations should include a review of the faculty's clinical teaching abilities, commitment to the educational program, clinical knowledge, professionalism, and scholarly activities.

•
This evaluation must include at least annual written confidential evaluations by the residents.
Very little is known regarding evaluation form content, specificity and usefulness.The aim of our study was to describe program resident faulty evaluation methods (e.g.frequency, distribution, and anonymity), communication of results, and the perceived importance of these evaluations for faculty promotion.

METHODOLOGY Survey Instruments
This study (protocol 06-0494) was approved by the Colorado Multiple Institutional Review Board.A twenty-item survey was created by investigators addressing the content of faculty evaluation forms (Fig. 1).A dermatoepidemiology lab group composed of 6 people created the survey.The group met in a focus session once per week for nine weeks.The survey was revised each time.The survey was piloted by the program director at the University of Colorado Health Sciences Center, and feedback was utilized to further modify the survey.The survey was voluntary and anonymous.Respondents were informed in an introductory letter that responses would not be individually identified.

Study Population
Dermatology programs and their directors within the United States were identified from online listings at www.acgme.org/adspublic/default.asp.Program directors of the 112 ACGME accredited dermatology residency programs were asked to complete the twenty-item survey by mail, and return it along with a blank copy of the evaluation form(s) used to evaluate faculty at their program.Surveys and follow-up letters were mailed up to three times to non-responding programs.Data collection began June 1, 2006 and ended October 18, 2006.

Statistical Analysis
Survey data were entered into a Microsoft Access database and statistical analyses were conducted using SAS version 9.1 (SAS Institute, Cary, NC) software.Evaluation components were compiled (EL and LU) independently, entered into Microsoft Excel spreadsheets and each compo-   nent's frequency was tabulated.Discrepancies were resolved by consensus.

Survey Responses
A high response rate was obtained: 97 of the 112 selected ACGME-accredited programs returned the completed survey (87%), Table 1 and Fig ( 1).Approximately two-thirds of respondents were residency directors (62%) and one third were chairs or section heads (33%).Only 2 programs reported that residents do not evaluate the dermatology faculty.Ninety-two percent of programs have residents complete the forms anonymously.The distribution of the evaluations to the residents was most often a paper form distributed to resident mailboxes (33%) followed by electronic forms sent to resident email accounts (29%).Four programs reported using different forms to evaluate faculty in different dermatology subspecialties (e.g.dermatopathology vs pediatric dermatology).Fifteen percent of respondents said that faculty evaluations by residents were not important for faculty promotion; 67% somewhat important; 18% very important.Twenty nine respondents filled in additional comments.These comments frequently discussed the importance of the evaluations and guaranteeing anonymity, and issues affecting evaluation validity (Table 2).

Evaluations
Fifty four respondents returned the faculty evaluation forms used by residents in their programs.The ten most common evaluation components are presented in Table 3.The most common components of the evaluations asked residents to rate faculty teaching (50/54), to fill in any additional comments about the faculty member (50/54), and to rate faculty availability/accessibility (40/54).

DISCUSSION
This study demonstrates that residents anonymously evaluate dermatology faculty in almost all dermatology programs without consideration of faculty subspecialty within dermatology.The majority of respondents identified these evaluations as somewhat or very important (85%) for determining faculty promotion.
Anonymity concerned many respondents.Some stated that residents might fear retribution for rating a teacher poorly, especially in programs with a smaller numbers of residents.Indeed, previous research has shown that faculty evaluation outcomes differ when performed openly vs anonymously, and that anonymous evaluations demonstrate more internal consistency [5].
A practical, task-oriented approach has been suggested for promoting program evaluation compliance with require- ments for faculty evaluation [6].Such an approach includes attention to evaluation needs, methods, focus, documentation, and result presentation to key constituents.Faculty and resident involvement in developing the evaluation process at each step will better assure successful outcomes.Dermatology residents cite a lack of mentoring, role models, and career guidance as reasons for losing interest in an academic dermatology career [7].Canadian dermatology residents have expressed the desire for more teaching and mentoring from faculty [8], and US dermatology residents reported greater satisfaction with more faculty involvement [9,10].
Faculty evaluations carefully exploring resident perceived deficiencies may foster improvement in these areas.
Internet portals may make evaluation more user-friendly and ACGME requirement compliance easier.Twenty nine percent distributed evaluation forms to residents using email while 26% used a web-based evaluation system like E*Value (https://www.e-value.net/index.cfm) or New Innovations (http://www.new-innov.com/) for the distribution and collection of the forms.Web-based evaluation systems can offer anonymity with greater convenience while preserving high response rates [11,12].Because of the emphasis on "outcomes" these evaluations are taken more seriously and more efforts are made to make sure evaluation processes are taking place. 2.
An additional faculty/program evaluation is conducted once a year by the GME office.Reports are issued to the program.

3.
Our evaluations are a numerical score of 1-5 and residents rank various criteria and may also list comments.I am concerned that some residents are less discriminatory and rank everyone highly across the board.Others are more critical.I believe the averages of the scores can however be a reflective measure of a faculty's performance.

4.
Dermatology residents do not evaluate dermatology faculty individually, however, residents make comments about specific faculty on the annual anonymous questionnaire they complete concerning the residency program's strengths and weaknesses.
While our study provides one of the most expansive descriptions of resident evaluation of faculty in the literature, it was nonetheless limited by 48% of respondents returning the actual forms residents use to evaluate dermatology faculty.Resident evaluations of faculty provide valuable feedback (that influences teaching and promotion) to faculty and residency program leaders.Current evaluations however might benefit from better tailoring to faculty subspecialization; to provide better feedback evaluations should be specific to faculty subspecialty (e.g.dermatopathology, medical dermatology, Mohs surgey, pediatric dermatology, dermatoepidemiology, cutaneous biology research, dermatoepidemiology, procedural dermatology, etc).Faculty evaluations would also benefit from more uniform assessment of faculty enthusiasm and professionalism, subjects addressed by a minority (48% and 39% respectively) of the evaluation forms received.
Our study was also limited to programs in the USA; further studies might compare these practices with resident evaluation of faculty in other countries.Further work should also examine evaluation practices from the perspective of the faculty and residents involved.

CONCLUSIONS
U.S. dermatology residents commonly evaluate their faculty members.These evaluations influence faculty promotion and affect teaching, and require resident anonymity to best assure valid assessments.

Please answer the following questions regarding overall dermatology faculty performance evaluations.
10. How is the faculty evaluation form(s) distributed to dermatology residents at your institution?97 responses, 104 answers __Paper form to residents' mailboxes 34 (33%) __Paper to the residents at a meeting 11 (11%) _ _ E m a i l 30 (29%) __Other__________________ 29 (28%) 11. How are the dermatology faculty evaluation form(s) collected from dermatology residents at your institution?Please provide any comments or concerns you have regarding the process of dermatology resident evaluation of dermatology faculty at your institution: 16. How are the results of the dermatology faculty evaluations discussed with the faculty at your program?(Check all that apply)

Table 1 . Summarized Results, Number of Responses/All Answers (%)*
*Denominators reflect multiple answers possible on select questions.