Does Self Evaluation Improve Surgical Skills? A Randomized Control Trial: A Pilot Study
Précis: This study investigated the effect of self-evaluation on the improvement of surgical skills by comparing OSATS scores over a 4-month period to determine if the addition of a self-evaluation improved surgical skills more than instructor evaluation alone.
Purpose: To determine if self-evaluation improves surgical performance more than instructor evaluation alone.
Background: The OSATS score is a validated rating scale for the evaluation of surgical skills. Self-evaluation using OSATS has been compared to instructor evaluation to determine the correlation of the scores. Self-evaluation in combination with instructor evaluation to improve surgical skills has not been studied.
Methods: A RCT was performed in which learners, PGY 1-4, were randomized into either the control group who only received instructor OSATS evaluations or the experimental group who received instructor evaluations as well as self-evaluations. Everyone received the same instructions and performed the same simulation which was videotaped for review. The instructor was blinded to each arm and the simulation was repeated after four months. The primary outcome was the change in instructor OSATS score over the study period.
Results: Fifteen residents were enrolled and completed the study. There was no difference in the change in instructor OSATS score (p=0.726). There was a correlation in the learner and instructor score for time and motion (p= 0.02) and instrument handling (p=0.008). All participants reported that self-evaluation was a useful educational tool. Only participants from the experimental group reported practicing on their own time.
Discussion: The current study attempted to demonstrate the utility of self-evaluation on surgical education. The correlation between learner and instructor scores are consistent with the literature. Self-evaluation did not improve overall change in score; however, all participants found it useful and it did increase practice at home.
SES, 2020, Resident, Faculty, Clerkship Director, Residency Director, Practice-Based Learning & Improvement, GME, Assessment, Simulation, Independent Study,
Read more »
Implementing Communications Training for OB/GYN Residents: Disclosure of Adverse Peri-operative Events
Precis: This study investigated the effect of self-evaluation on the improvement of surgical skills by comparing OSATS scores over a 4-month period to determine if the addition of a self-evaluation improved surgical skills more than instructor evaluation alone.
Purpose: To study the implementation of communications training for OB/GYN residents on the disclosure of adverse peri-operative events
Background: Communication skills are key components of the patient-physician relationship, however, these skills are not routinely taught during OB/GYN residency or fellowship. As OB/GYNs, disclosure of serious news happens routinely in various aspects of patient care.
Methods: OB/GYN residents at a single institution participated in a 4-hour communication didactics session using VitalTalk methodology. Participants were surveyed at baseline, immediately following training and 3 months post-training to measure changes in comfort and confidence in the performance of communication skills. A 5 point Likert scale was used to measure comfort/confidence: Novice (1), Advanced Beginner (2), Competent (3), Proficient (4) or Expert (5).
Results: 27 residents participated in training; 8(29.6%) interns, 7(25.9%) 2nd year, 7(25.9%) 3rd year and 5(18.5) 4th years. 11/27 (40.7%) reported prior communication training. Before training, residents reported the most competence: detecting emotion cues (median: 3.1), detecting patient/family sadness (3.0) and responding empathetically (2.8). Residents felt the least competent: assessing patient/family willingness to discuss perioperative complications (2.0), assessing how much a patient/family wants to know (2.1), confirming understanding of the adverse event (2.3). Baseline comfort and skill performing these conversations was 2.1 and 1.9, respectively. Immediately post-intervention, average comfort and skill level increased to 2.8 and 2.8, respectively. 3 month data is currently being collected.
Discussion: OB/GYN residents self-rate their communication confidence and skills at an advanced beginner level. Implementation of participatory communication training for OB/GYN residents is feasible and improves both trainee comfort and skills in the disclosure of adverse peri-operative events. Further work is ongoing to understand retention of these skills to inform more longitudinal curricula.
General Ob-Gyn, Standardized Patient, Simulation, GME, Interpersonal & Communication Skills, Professionalism, Residency Director, Clerkship Director, Faculty, Resident, Student, 2020, SES,
Read more »
A Needs Assessment for a National Ob/Gyn Residency Education Curriculum
Purpose: We sought to determine Ob/Gyn residents’, program directors’ (PDs), and assistant program directors’ (APDs) perceptions of the need for a national Ob/Gyn residency education curriculum. This study was funded by the CREOG Empower Award.
Background: Several specialties have implemented standardized, national residency education curricula. These curricula allow residents to have equal access to a centrally located, structured, common knowledge base. In Ob/Gyn, although we have outstanding educational resources, they are not integrated into a national curriculum nor are they co-located. Rather, each residency program develops its own didactic curriculum, which requires a significant amount of time and effort and leads to variability in curriculum quality and content.
Methods: A 12-question web-based survey was developed and distributed to residents, program directors, and assistant program directors nationwide using the Ob/Gyn program coordinator’s listserve and program contact information from the ACOG website.
Results: 780 participants (651 residents, 83 PDs, and 46 APDs) completed the survey, with a diverse range of regions, program types, and program sizes represented. Almost all (97.2%) participants agreed or strongly agreed that residents nationwide should have equal access to high quality Ob/Gyn education resources. Further, 91.9% agreed or strongly agreed that core Ob/Gyn educational resources should be centrally located. A majority, 77.8% (residents 78%, PDs 75%, and APDs 78%), agreed or strongly agreed that there is a need for a national Ob/Gyn residency education curriculum.
Discussions: Our results demonstrate a perceived need for a centrally located, structured national Ob/Gyn residency education curriculum.
General Ob-Gyn, Independent Study, Lecture, GME, Medical Knowledge, Residency Coordinator, Residency Director, Osteopathic Faculty, Clerkship Coordinator, Clerkship Director, Faculty, Resident, Student, 2020, CREOG & APGO Annual Meeting,
Read more »
Text vs Email for a Question of the Day: Which do Today’s Medical Students Prefer?
Purpose: Compare completion rates and satisfaction of
question of the day(QOD) received via text vs email.
Background: Fast, hands on learning e-tools are a popular
effective technique with current generation of medical learners.
Methods: Twenty-eight questions of the day(QOD) developed
focusing on commonly missed NBME exam concepts. A prospective cohort study of
ob/gyn clerkship students conducted. Students self-selected to receive QOD by
text or email from 1/19-7/19. Students received QOD nightly on weekdays via
text or email, completion was optional.
Primary outcome was overall response rate(RR). Secondary outcomes include
RR/question and average weekly RR. Qualitative feedback regarding satisfaction,
time spent, and utility also obtained.
Results: Eighty-one medical students participated, 40 selecting
text and 41 email. In sum, 863 texts and 1269 emails sent during study
period. Overall text RR was significantly lower than
email(55.2%vs78.6%,p< 0.001). For no question was text RR greater than email
RR. Average RR did not vary by week(wk1: text 55% vs 83% email, wk6: text 64%
vs email 77%).
43/81(53%) students gave qualitative feedback, 28/41 from email cohort, and
15/40 text. 86% found the QOD helpful/very helpful(no difference by mode of
receipt). However, email recipients more likely to find it very
helpful(54%vs13%,p=0.02). 77% found the level of difficulty to be “just
right”,(no difference by mode of receipt). The QOD took all students less than
5 minutes. Nearly all(93%) students found the QOD worthwhile and 98% would
recommend to future students.
Discussions: Surprisingly, QOD completion rates were higher with
email than text, though satisfaction and perceived utility high in both
CREOG & APGO Annual Meeting, 2020, Student, Faculty, Clerkship Director, Medical Knowledge, Practice-Based Learning & Improvement, UME, Assessment, Independent Study, General Ob-Gyn,
Read more »
Residents as Preceptors: Developing a Teaching Curriculum for Obstetrics and Gynecology Residents
Background: Residents serve a critical role in medical student
education as near-peer educators. Positive experiences with residents and
perceived quality of teaching have been associated with improved clerkship
satisfaction and career choices in that discipline. ACOG and ACGME milestones
further emphasize the importance of residents as educators. While the role of
residents as preceptors is widely acknowledged, preparation for that role is
Methods: A single arm interventional study with pre and post
intervention analyses was used. A four part lecture series was designed and
implemented during obstetrics and gynecology resident didactics at the
University of Minnesota. Residents completed blinded and paired online
surveys pre and post lecture. 5 point Likert scale responses
were summarized and compared using Wilcoxon Signed Rank and Nominal Symmetry
Results: There was a statistically significant
difference in the distribution of rankings towards the positive between pre and
post surveys for questions regarding evaluation and teaching
. There were no statistically significant differences in the distribution of
rankings for questions regarding feedback. Barriers to teaching were also
Discussions: Implementation of a teaching curriculum significantly
improved residents\' perceived preparedness and understanding of medical student
teaching and preceptorship but did not improve attitudes towards and frequency
of feedback. Strengths of this study include description of a novel, resident
led program. Limitations include a small sample size at a single institution
within a single specialty.
CREOG & APGO Annual Meeting, 2020, Resident, Faculty, Clerkship Director, Residency Director, Professionalism, Interpersonal & Communication Skills, GME, Assessment, Lecture, General Ob-Gyn,
Read more »
Resident Perceptions of Power-Based Violence and Impact of Bystander Intervention Training
Purpose: To evaluate resident perceptions of power-based
violence (PBV) and assess initial impact of bystander intervention (BI)
Background: Creating a safe, welcoming workplace is important in
medical education. PBV in the form of overt aggression, microaggression,
or bullying can threaten the educational community. BI training offers
residents a skill set to intervene to counteract PBV and foster a safer,
welcoming clinical environment for their teams.
Methods: Pre-training and post-training surveys
were completed by OB-Gyn residents at Vanderbilt prior to formal Green
Dots BI training. The survey assessed frequency of experienced or
witnessed PBV, sexism, and racism. Residents self-assessed their
ability to recognize overt versus subtle PBV, and whether they would intervene
in instances of PBV based on the role of the perpetrator. Residents
completing BI training were re-surveyed with the same instrument 3 months
later. Exempt IRB approval was obtained.
Results: 22 active residents were surveyed pre-training. 19
residents completed BI training and the post-training survey. Active
residents reported at least sometimes being the target of PBV (22.7%), sexism (45.5%),
or racism (9.0%). Even more residents reported at least sometimes
witnessing PBV (50.0%), sexism (59.1%), or racism (45.5%). Residents’
willingness to intervene varied by perpetrator: fellow resident (59.0%),
attending (18.2%), nursing staff (40.9%). 100% of active residents felt
obligated to foster a safe work environment free of PBV.
Among residents completing BI training,
the percentage reporting recognition of overt PBV increased
after training from 84.2% to 94.7% (p=0.29), and the percentage reporting
recognition of subtle PBV increased after training from 78.9% to 89.5% (p=0.37).
There was a modest increase after training in the percentage reporting they
would intervene for PBV perpetrated by a resident (52.6% to 57.9%, p=0.74) or
nursing staff (31.6% to 36.8%, p=0.73), but no change for faculty-perpetrated
Discussions: PBV is experienced and/or witnessed by residents not
infrequently. While residents are more likely to intervene for PBV
perpetrated by a fellow resident, they are less likely to intervene when
perpetrated by attendings or nursing staff. BI training appears to offer
modest improvements in recognition of PBV and willingness to intervene, but its
effect in this study was limited by sample size.
CREOG & APGO Annual Meeting, 2020, Student, Resident, Faculty, Clerkship Director, Residency Director, Professionalism, GME, UME, Team-Based Learning, Advocacy,
Read more »
Proficiency Perception and Relevance of Gynecological Procedures Among Obstetrics and Gynecology Graduates; Evaluating the Residency Surgical Curriculum
Purpose: To determine graduates’ perception of their
proficiency in gynecologic operations and the relevance of these procedures to
their current practice
Background: Program directors must certify that graduates are competent
to practice without supervision after residency. Achieving proficiency in
gynecologic surgery after 4 years is challenging. Evaluating the current
surgical curriculum is critical
Methods: A survey of 40 graduates from a single academic
instituition was conducted. 36 gynecologic procedures were listed. Each
procedure corresponded to two four-point scales (strongly agree to strongly
disagree). One scale was headed with preparation to independently perform the
operation, and the other questioned the relevance of the procedure in their
current practice. A third scale reviewed the need for additional training, and
the case/volume per year. A subgroup analysis was done to evaluate the
preferred route of hysterectomy and the rationale if vaginal hysterectomy was
Results: The response rate was 67.5%. All respondents felt
prepared to perform 25% (9/36) of procedures. More than 50% did not feel
competent performing laparoscopic single site hysterectomy, laparoscopic
myomectomy, sling, utero-sacral ligament suspension, vesico-vaginal fistula
repair, colpocleisis, and office hysteroscopy. Vaginal hysterectomy was
relevant for 78% of graduates, however, 89% performed fewer than 10 cases per
year, and 55% would benefit from additional training. More than 50% of
graduates would choose a non-vaginal route of hysterectomy, and interestingly,
93% of those based their decision on lack of comfort with the procedure
Discussions: While several gynecologic surgeries are relevant to
recent graduates’ practices, some may not feel competent performing these
procedures after residency. Vaginal hysterectomy is not commonly performed
among graduates, largely due to inexperience with the procedure
CREOG & APGO Annual Meeting, 2020, Resident, Faculty, Clerkship Director, Clerkship Coordinator, Residency Director, Residency Coordinator, Patient Care, Systems-Based Practice & Improvement, Practice-Based Learning & Improvement, GME,
Read more »
OBGYN Trainees Perception of Departmental Climate with Regards to Sexual Harassment
Purpose: To assess Obstetrics and Gynecology(OBGYN) trainees’
perception of departmental-cultural-climate(DCC) regarding sexual
Background: Positive DCC results in better psychological health
and workplace environment. Climates perceived to be permissive of SH by the
employees has lower overall work satisfaction.
Methods: Using REDCap data collection tool, we conducted an
anonymous survey of OBGYN trainees in ACGME-accredited residency/fellowship programs
in the U.S. The survey included demographics; the validated Sexual Experience
Questionnaire(SEQ) short form; and assessment of DCC via questions regarding
institutional reporting systems and trainees’ perceptions of how complaints
would be handled. Descriptive statistics were utilized.
Results: The survey was distributed to 1,473 residents/fellows
in the U.S. with 26.8%(395/1,473) response rate. The prevalence of SH was
69.1%(253/366). 90.3%(331/366) of trainees felt safe in their institution, while
only 3.0%(11/366) felt unsafe. 72.7%(8/11) of trainees who felt unsafe had
experienced SH. 88.5%(337/381) believed their SH report would be taken
seriously. 75.1%(286/381) believed privacy would be maintained. 80.3%(305/380)
thought their department would support them, and 76.7%(289/377) that the report
would be handled fairly. 15.6%(59/379) thought they would be labeled as a
troublemaker, and 10%(38/379) thought they would be punished. Only
3.3%(4/121) made an official report. 67.4%(149/221) did not tell anyone; and
39.4%(95/241) did not know of a reporting system. No action was taken for
47.5%(29/61) of the reports; the perpetrator was spoken to about their
behavior[14.8%(9/61)]; concerns were taken seriously[8.2%(5/61)]; and, concerns
were found to be unsubstantiated[1.6%(1/61)].
Discussions: OBGYN trainees are often reluctant to report SH, and
OBGYN departments may to need improvement in DCC based on our results.
GME, Systems-Based Practice & Improvement, Professionalism, Residency Coordinator, Residency Director, Osteopathic Faculty, Clerkship Director, Faculty, Resident, Student, 2020, CREOG & APGO Annual Meeting,
Read more »
Novel Realistic Bladder Model Teaches Anatomy & Builds Resident Confidence in Cystotomy Recognition/Repair
Purpose: To develop and pilot an inexpensive, reusable,
water-tight moderate-fidelity bladder model that can be used to teach Ob-Gyn
residents cystotomy repair.
Background: Cystotomy repair is a required Ob-Gyn milestone
procedure; however, low surgical volume for cystotomy may impact resident
ability/confidence to perform the procedure. Available simulation models are
either expensive or low-fidelity and not water-tight. Our goal is to create a
cost-effective, realistic, functional bladder model (visible ureteral jets,
water-tight after repair) for simulation of cystotomy repair.
Methods: Phase 1: Design and creation of the model using molds
developed with 3D design software and 3D printing. The silicone model includes
ureters, urethra, trigone, ureteral orifices, rugae, muscularis, and mucosa.
Phase 2: Pilot of the model with Ob-Gyn residents as part of their simulation
curriculum. Residents were given a pre- and post-test to identify the model’s
anatomic landmarks and rate their confidence in 1) identifying cystotomy and 2)
performing open cystotomy repair. Residents evaluated the model’s fidelity
Results: Eight bladder models were constructed for $206. The
model was water-tight post-repair and cystoscopy demonstrated bilateral
ureteral jets. All residents, n=16, had no prior cystotomy simulation and
limited experience with cystotomy repair. The model improved ability to
identify bladder anatomy (4.3/6 items pre vs 5.8/6 items post, p=0.04) and
confidence in both identifying cystotomy (2.14 vs 3.4 of 5, p= 0.006) and
performing cystotomy (1.4 vs 3.3 of 5, p=0.014). Learners strongly agreed the
model was realistic, mean 4.6/5.
Discussions: Our novel bladder model is cost-effective, realistic
and useful for teaching anatomy and simulation of cystotomy repair.
CREOG & APGO Annual Meeting, 2020, Student, Resident, Faculty, Clerkship Director, Clerkship Coordinator, Osteopathic Faculty, Residency Director, Residency Coordinator, Patient Care, Medical Knowledge, Practice-Based Learning & Improvement, GME, Simulation, General Ob-Gyn,
Read more »
Improving the Basic Science Curriculum by Integrating the Clinical Sciences
Purpose: To evaluate medical student perceptions on whether
curriculum innovations through the integration of clinical medicine with basic
science improves learning in the pre-clinical curriculum.
Background: Medical schools are transitioning to competency-based
curricula that emphasize an early introduction to clinical training. Students
who learn basic science explanations for clinical correlates have improved
retention. By integrating clinical sciences into the pre-clinical curriculum,
learning is optimized for students.
Methods: At the end of the Reproduction pre-clinical sequence
at the University of Michigan, students were asked to evaluate the overall
quality of the course, whether objectives were met, and whether the course was
well-organized, with a one-to-five level scale. These parameters were compared
for the sequence between 2018 and 2019, after curricula changes were
Results: For the Reproduction sequence in 2019, course
assessment parameters were improved in comparison to 2018 values. Overall,
students rated the quality of the course as “very good/excellent” (4.21 ±
0.91), compared to “fair/good” (2.88 ± 1.01). Students “strongly agreed/agreed”
that objectives for the course were met (4.44 ± 0.82) versus “agreed” (3.74 ±
0.88), and “agreed” that the course was well-organized (4.21 ± 1.07) compared
to “disagreed” (2.66 ± 1.19).
Discussions: The restructuring of the Reproduction sequence allowed
for purposeful integration of clinical medicine into the basic science
pre-clinical curriculum, resulting in improved quality of learning for
students. Recruiting clinical faculty for lectures, and implementing case-based
instruction with the APGO videos for supplementation, provided an effective and
efficient method of integrating clinical medicine content within the sequence.
CREOG & APGO Annual Meeting, 2020, Clerkship Director, Medical Knowledge, Practice-Based Learning & Improvement, UME, Lecture, Problem-Based Learning,
Read more »
Impact of TrueLearn Question Bank on Resident Performance on the CREOG In-Service Training Exam
Purpose: To determine how implementation of the TrueLearn (TL)
question bank and mock in-service training exam (ITE) impacts performance on
the CREOG ITE.
Background: The CREOG ITE is an annual exam on which, success has
been correlated to success on the American Board of Obstetrics and Gynecology
Qualifying Exam. Question banks are quickly becoming a popular study tool, but their
utility on improving CREOG ITE performance is unknown.
Methods: We performed a quasi-experimental retrospective review
of the CREOG ITE score reports from 2017 to 2019 at our institution. Starting
in 2018, TL was made available and a mock ITE was administered for the first
time. A two-sided paired t-test compared average annual improvement in CREOG
ITE scores before and after TL implementation. Correlation coefficients between
TL usage and improvement on the 2019 ITE were calculated using Pearson correlation
testing and a linear regression was used to determine their association.
Results: After providing TL, mean improvement in percentage
points on the CREOG ITE was 6.6, 9.33 and 6.75 as residents matriculated from
PGY1-2, PGY2-3 and PGY3-4 respectively. This was a significant improvement compared
to the previous year. As a program there was a moderate correlation with
TL usage and percentage point improvement on the CREOG ITE; r=0.38, p= 0.038.
Discussions: After providing universal access to TL question bank
and administering a mock ITE, a statistically significant improvement on the
CREOG ITE was seen. TL’s question bank and mock ITE are valuable resources for
improving CREOG ITE performance.
CREOG & APGO Annual Meeting, 2020, Resident, Faculty, Clerkship Director, Clerkship Coordinator, Osteopathic Faculty, Residency Director, Residency Coordinator, Medical Knowledge, Systems-Based Practice & Improvement, Practice-Based Learning & Improvement, GME, Assessment, Independent Study, Problem-Based Learning, General Ob-Gyn,
Read more »
Gender and Satisfaction with Mentorship In Medical School: A National Study
Purpose: Using the American Association of Medical Colleges
(AAMC) Graduation Questionnaire (GQ), we assess if males and females at all
U.S. medical schools report differing experiences with satisfaction with
faculty mentorship. Secondarily, we assess the role of men\'s and women\'s
scholarly projects and career plans as they relate to satisfaction with mentoring.
Background: Female physicians experience gender-based professional
differences, such as lower compensation, slower rates of promotion, and
decreased representation in leadership positions as compared to their male
colleagues. The way this gender-based professional inequity affects the
experience of medical student mentees has yet to be elucidated in the
Methods: Data were obtained from the AAMC GQ years
2016-2018. Student satisfaction with faculty mentoring was analyzed by
chi-squared and logistic regression.
Results: With an 82% response rate we analyzed data for 47,063
students; 51% were male and 49% female. When asked about satisfaction with
faculty mentoring, 81% of males verses 79% of females reported being
\"satisfied\" or \"very satisfied\" (p< 0.001). A higher
proportion of males reported faculty were helpful to students with academic
matters (60% versus 55%; p< 0.001) and with non-academic matters (58% versus
55%; p< 0.001) \"very often\" or \"always.\" Females were
less likely to be satisfied with faculty mentoring even after controlling for
participation in research with a faculty member and future research plans.
Discussions: This work serves as a needs assessment to encourage
individual medical schools to investigate their own gender-based cultures
regarding faculty representation and leadership to allow gender-equal
mentorship of all students.
CREOG & APGO Annual Meeting, 2020, Faculty, Clerkship Director, Residency Director, Professionalism, Interpersonal & Communication Skills, GME, CME, UME, Advocacy,
Read more »
Gaining Resident Buy-In: Three Year Follow Up of a Resident-Led Resident as Teacher Program
Purpose: To evaluate teaching culture in an OB/GYN residency
three years after the initiation of a Resident-Led Resident as Teacher (RLRT)
Background: In October 2016, a RLRT program was created to foster
the growth of clinician-educators by providing opportunities for direct instruction
of medical students, educational leadership and professional development.
Residents completing 25 hours of educational work are recognized as
Distinguished Resident Educators at graduation.
Methods: We reviewed end of clerkship evaluations of resident
teaching from July 2016 - June 2019. We also surveyed residents annually
regarding teaching skills and views of the RLRT program in October 2016-2018.
Chi-square and Kruskal-wallis testing were used for statistical analysis.
Results: For 2018-2019 academic year, 48% of medical students
strongly agreed that residents provided effective teaching compared to 30% in
2016-2017 (p< 0.01). Of 48 residents, 42 (88%), 34 (71%), and 30 (63%)
residents completed the survey in 2016, 2017, and 2018, respectively. In
2018, 93% of PGY1s and PGY2s intended to be recognized as Distinguished
Resident Educators, compared to 56% of PGY3s and PGY4s (p=0.02). Incoming
PGY1s in 2018 rated themselves higher on average in teaching skills than PGY1s
in 2016, specifically in making mini-lectures and guiding students to
evidenced-based medicine resources (p< 0.01). All of the PGY1s (100%)
in 2018 felt that the RLRT positively influenced their decision to join the
A RLRT program can strengthen the teaching culture by
increasing the effectiveness of resident teachers and aiding in recruitment of
residents interested in resident teaching.
CREOG & APGO Annual Meeting, 2020, Student, Resident, Faculty, Clerkship Director, Residency Director, Professionalism, Interpersonal & Communication Skills, GME, UME, Team-Based Learning,
Read more »
Evaluation Disparity and Implicit Bias in Medical Student Grading: Findings from Evaluator Focus Groups
Purpose: To perform a needs assessment of medical student
evaluators to uncover reasons for evaluation disparity in order to work towards
improving the quality, consistency, and accuracy of OB/GYN clerkship
Background: Accurate evaluation of medical student clinical
clerkship performance is challenging. Scale-based student ratings often do not
correlate to comments provided. Additionally, there can be a wide variance of
scores for the same student by different evaluators. Information regarding
differing grading systems is available, but there is a lack of guidance around
inter-rater reliability and implicit bias.
Methods: From May - July 2019, we performed three resident and
faculty focus groups. We began the session discussing evaluation form
reliability and current behaviorally anchored rating scales, same student
evaluation inconsistencies demonstrated between evaluators, accuracy, and
potential evaluator bias. Qualitative analysis utilizing two reviewers was used
to identify themes.
Results: Overall, participants expressed a need for more formal
student evaluation information and instruction. Other themes were requests for
education and information regarding the evaluation forms and student grading,
an admission of variation between ‘hard’ and ‘easy’ graders, and implicit bias
in student evaluation, particularly regarding perceived specialty choice.
Discussions: Evaluators felt a need to better understand how
evaluations contributed to student rotation grades and were interested in
improving the quality, consistency, and accuracy of evaluations. Time with
students directly influences the accuracy of objective evaluations. Implicit
bias around one’s own ‘grading scale’, and around student specialty selection,
can influence student evaluations.
CREOG & APGO Annual Meeting, 2020, Student, Resident, Faculty, Clerkship Director, Clerkship Coordinator, Professionalism, Interpersonal & Communication Skills, UME, Assessment,
Read more »
Empathy Training Using Improvisational Comedy: Impact on Multidisciplinary Education among Health Professions Students
Purpose: Determine the impact of improvisation training on
self-reported and observed measures of empathy among a multidisciplinary group
of health professions students
Background: Provider empathy improves patient outcomes. The most
effective technique for promoting empathy remains unknown. Improvisation
training may improve cognitive and affective empathy by enhancing one’s ability
to react to the observed experience of others. Improvisation workshops have
been shown to improve empathy in OB/GYN residents. We aimed to evaluate the
impact this training on self-reported and observed measures of empathy within a
multidisciplinary group of health professions students.
Methods: This is a prospective cohort study of health
professions students who participated in a mixed-methods empathy training
activity (improvisation exercises and case-based scenarios administered by
standardized patient actors before, during, and after the educational component).
Students completed pre- and post-intervention self-reported empathy
questionnaires. Trained patient actors observed student interactions and
completed validated empathy questionnaires following each case-based interaction.
Paired t-tests were used for within-person comparisons over time and ANOVA was
used to compare changes across groups.
Results: Twenty-eight students participated (4 medical; 8
social work; 11 nursing; 5 physical therapy). Self-reported empathy scores
improved significantly from pre- to post-intervention (34.3 vs 38.2, p<
0.0001) although observed empathy scores did not change (43.6 vs 42.9, p=0.65).
When comparing across disciplines, there was no difference in observed empathy
Discussions: Improvisation training improves self-reported but not
observed empathy among a multidisciplinary group of health professions
students. There was no difference in the efficacy of the training workshop
CREOG & APGO Annual Meeting, 2020, Resident, Faculty, Clerkship Director, Clerkship Coordinator, Osteopathic Faculty, Interpersonal & Communication Skills, GME, Assessment, Simulation, Team-Based Learning,
Read more »
Duty Hours and Medical Student Performance: Is There a Correlation?
Purpose: To describe the association between duty hours (DH)
and shelf exam score, and between DH and overall grade.
Background: Medical students are required to comply with the same
DH regulations as residents. However, variation exists between the total DH
worked due to diversity of clinical sites, surgical rotations, and student
engagement. The impact of DH on exam scores and clinical grade is poorly
Methods: Retrospective cohort study of students in OB/GYN
clerkship at a single academic center from 8/18-6/19. Weekly duty hours
tabulated and averages obtained. Students working more than 75%ile for DH
weekly (long hours) compared to those working less. Impact of long hours in
last two weeks of clerkship also examined. Outcomes of interest were high
overall grade ( >95%), high shelf score ( >90%), and low shelf score
(< 60%) adjusted for quartile of year. Simple statistics used to analyze data.
Results: The average weekly DH was 43.1hr/wk (SD 7.32hr). This
didn’t vary by gender. Students in the latter half of the year worked longer
(45.2 hrs/wk vs 41.9 hrs/wk, p=0.03). The 75%ile of weekly DH was 47.1 hrs.
Students working long hours were not more likely to have low shelf (21.7% vs
30.4%,p=0.6), high shelf (28.3% vs 28.6%,p >0.99), or high overall grade
(25% vs 28.9%,p >0.99). Long hours in the last 2 weeks of the clerkship was
not associated with high overall grade (33.3% vs 31.1%,p >0.99) or low shelf
(21.7% vs 34.2%,p=0.32). A trend toward high shelf scores was seen (41.3% vs
Discussions: Long duty hours, even late in the clerkship, were not
associated with high or low shelf scores, nor high overall grade.
CREOG & APGO Annual Meeting, 2020, Student, Resident, Faculty, Clerkship Director, Clerkship Coordinator, Medical Knowledge, UME, Assessment, General Ob-Gyn,
Read more »
Curriculum Development in the OBGYN Clerkship: Effect on NBME Scores
Purpose: Investigate whether implementation of APGO Medical
Student Educational Objectives into an OBGYN clerkship didactic curriculum
results in higher NBME exam scores.
Background: Medical student curriculum reform is at the forefront
of OBGYN educational research. Active, adult learning techniques are highly
touted; data are promising for student satisfaction, but more studies about
effect on objective assessments are needed.
Methods: We implemented 21 of 58 APGO Medical Student
Educational Objectives into the OBGYN Clerkship didactic curriculum during
academic year (AY) 2018-2019. Students were assigned pre-reading material and
completed weekly, graded individual readiness assessment tests, followed by
team readiness assessments, before each didactic session. NBME scores were
compared pre- and post-intervention (AY 2017-2018 to AY 2018-2019, respectively)
by yearly cumulative, as well as by individual cohort within each year.
Analysis was completed with independent samples t-test with two-sided P <
0.05 statistically significant.
Results: Overall, AY 2018-2019 had higher average NBME scores
than AY 2017-2018, although not statistically significant (79.5 vs 78.6,
P=0.29). Cohorts 4 and 6 showed significantly higher scores in AY 2018-2019
relative to AY 2017-2018, 80.0 vs 75.4 and 82.1 vs 77.7 respectively, (P<
0.05). Cohorts 3 and 5 scores were slightly higher on average and cohorts 1 and
2 scores were lower on average in AY 2018-2019 relative to AY 2017-2018, but
these differences were not found to be statistically significant.
Discussions: Implementation of the APGO Medical Student Education
Objectives into our OBGYN didactic curriculum showed a trend toward higher NBME
scores, suggesting benefit, however this difference was not statistically
CREOG & APGO Annual Meeting, 2020, Student, Faculty, Clerkship Director, Clerkship Coordinator, Medical Knowledge, GME, Assessment, Lecture, Team-Based Learning, General Ob-Gyn,
Read more »
CREOGs Over Coffee—A Feasibility Study on Medical Education Podcast Creation and Social Media Outreach
Purpose: To describe the development and implementation of a
medical education podcast series for obstetrics and gynecology (Ob/Gyn)
resident learning and demonstrate feasibility, sustainability, and acceptance
of this series.
Background: Podcasts and other digital resources have become
popular among medical learners. Demand by learners for these types of resources
compared to traditional methods of learning may increase as we move further
into the digital age. Medical educators may wish to incorporate podcast
learning into their curriculum and disseminate their work to a larger audience.
Methods: Council on Resident Education in Obstetrics and
Gynecology (CREOG) educational guidelines were used to create a study podcast
for Ob/Gyn residents over 9 months. Feasibility markers include production of
weekly episodes and ability to offset costs. Sustainability markers included
download markers over time. Acceptance was measured by number of reviews on
Apple iTunes and followers on Twitter.
Results: 42 episodes were released from September 30, 2018-June
30, 2019 (39 weeks). Initial costs included $3150 startup and $29 monthly.
Online donations through Patreon amounted to $200/month, which covered 50% of
startup costs at 6 months and are projected to cover full costs by 1.5 years.
The podcast had 137,500 downloads as recorded through Podbean (39/month 9/2018,
increased to 25,000/month 6/2019). It obtained 609 followers on Twitter and 131
ratings on Apple iTunes, with an average of 5.0 stars.
Discussions: Medical education podcasts are a feasible way of
disseminating educational materials to learners. We also demonstrate sustainability
and acceptance of this platform.
CREOG & APGO Annual Meeting, 2020, Student, Resident, Faculty, Clerkship Director, Residency Director, Patient Care, Medical Knowledge, GME, UME, Lecture, Independent Study,
Read more »
Clerkship Directors’ Reports and Attitudes on Pelvic Exams Performed by Medical Students
Purpose: To examine current practices of obtaining consent for
pelvic exams performed by medical students as reported by OBGYN Clerkship
Background: Recently, ethical concerns regarding student
involvement in pelvic exams performed while a woman is under anesthesia have
been raised in the scientific, professional, and popular literature.
Methods: An online survey about the educational pelvic exam was
administered to OBGYN CDs as part of the APGO annual clerkship directors’
survey. CDs were asked if their institution had policies on the
performance of pelvic exams by students, if explicit consent was obtained, and
if patients, students, and providers were educated on the policy. These were
evaluated by descriptive statistics of self-reported policies. We also compared
the median values of the perceived importance of telling a patient about the
involvement of a medical student across seven different procedures.
Results: The survey was sent to 230 clerkship directors, 140
responses were returned, with a response rate of 61%. 57 CDs reported they
follow all 4 recommendations (40.7%) as published by APGO’s statement on pelvic
exams under anesthesia. 9 CDs did not believe the exam had to be related
to the procedure performed. Consent for pelvic exam was seen equivalent to
consent for rectal exam, intubation, and making the incision. Consent was
perceived less important to watch the procedure, suture the incision, and
Discussions: While many CDs report institutional policies on
students performing pelvic exams, it is clear that education is still needed to
ensure proper consent is obtained.
CREOG & APGO Annual Meeting, 2020, Student, Faculty, Clerkship Director, Clerkship Coordinator, Patient Care, Professionalism, Systems-Based Practice & Improvement, Interpersonal & Communication Skills, UME, Quality & Safety, General Ob-Gyn,
Read more »
Care Across the Life Cycle: Results of an Integrated Pediatric, Obstetrics and Gynecology, and Geriatric Medical Student Clerkship
Purpose: The authors explore medical knowledge attainment and
course evaluation data in Pediatrics and Obstetrics and Gynecology after the
implementation of a novel, three-phase undergraduate medical school education
curriculum called ForWard. In ForWard, the two clerkships were shortened from
six to four weeks and combined with geriatrics to form an integrated 12-week
Background: The traditional structure for clinical clerkships are
specialty specific rotations. Due to evolving needs of both patients and
learners, there has been interest in integrating the clinical fields during
delivery of undergraduate medical education. With any curricular change, it is
important to evaluate the learning outcomes.
Methods: Pediatric and Obstetrics and Gynecology NBME subject
exam scores were compared prior to (n=180) and after (n=162) the implementation
of the ForWard curriculum using an analysis of variance with covariate
(ANCOVA). Student satisfaction with the traditional and ForWard
curriculum were determined based on post-clerkship evaluation forms and
compared using unpaired t-tests.
Results: The unadjusted mean NBME Pediatrics and Obstetrics and
Gynecology subject examination scores were 77.1 and 80.0 in the traditional
curriculum and 76.9 and 79.3 in the ForWard curriculum (p=0.83) and (p=0.37).
The mean Pediatric and Obstetrics and Gynecology clerkship evaluation scores in
the traditional curriculum were 77.1 (SD 7.6) and 80.0 (SD 6.9) compared
to 76.9 (SD 7.9) (95%CI -1.84-1.47) and 79.3 (SD 7.7) (95%CI -2.26-0.85) in the
Despite shortened Pediatric and Obstetrics and Gynecology
clerkship experiences, performance on the NBME subject examinations in both
subjects was unchanged and student satisfaction was maintained.
CREOG & APGO Annual Meeting, 2020, Student, Resident, Faculty, Clerkship Director, Clerkship Coordinator, Osteopathic Faculty, Medical Knowledge, UME, Assessment, General Ob-Gyn,
Read more »