Learner-Initiated Feedback Framework Trial (LIFFT): a Formal Framework for Perioperative Feedback
Purpose: To evaluate the effect of a learner-initiated framework for perioperative feedback on the frequency and satisfaction of resident feedback.
Background: Procedural feedback is an important aspect of resident education and surgical performance improvement. Satisfaction with feedback has been low in our program (64% reported somewhat, very or extremely satisfied) compared to the ACGME national average (72%).
Methods: This is a prospective cohort study of OBGYN residents at Los Angeles County+USC Medical Center. A learner-initiated framework was implemented for 3 months of gynecologic cases. After a pre-survey was performed the framework was introduced via didactic. Residents completed immediate postoperative surveys following each case. Residents and faculty completed a post-intervention survey. Primary outcome was proportion of cases in which feedback was given. To detect a 25% difference in feedback frequency, a total of 99 cases were required with 80% power.
Results: Before intervention, residents reported feedback in 25 of 42 cases (59.5%), compared to 82 of 122 cases (67.2%) during the study period (NS). Resident satisfaction (reported sometimes, very, or extremely satisfied) with feedback increased from 67.9% to 90.3% (p=0.003). Residents also reported both more frequent review of case goals (p<0.01) and receipt of feedback (p=0.007).
Discussion: Learner-initiated framework objectively raised frequency of feedback received from faculty, albeit non-significantly. Subjectively, residents indicated they received more frequent feedback and were more satisfied with that feedback. This trial demonstrated that a formal framework for perioperative feedback significantly improved resident satisfaction with feedback and should be considered for routine use at USC.
Assessment, GME, Interpersonal & Communication Skills, Residency Director, Faculty, Resident, 2020, SES,
Read more »
Does Self Evaluation Improve Surgical Skills? A Randomized Control Trial: A Pilot Study
Précis: This study investigated the effect of self-evaluation on the improvement of surgical skills by comparing OSATS scores over a 4-month period to determine if the addition of a self-evaluation improved surgical skills more than instructor evaluation alone.
Purpose: To determine if self-evaluation improves surgical performance more than instructor evaluation alone.
Background: The OSATS score is a validated rating scale for the evaluation of surgical skills. Self-evaluation using OSATS has been compared to instructor evaluation to determine the correlation of the scores. Self-evaluation in combination with instructor evaluation to improve surgical skills has not been studied.
Methods: A RCT was performed in which learners, PGY 1-4, were randomized into either the control group who only received instructor OSATS evaluations or the experimental group who received instructor evaluations as well as self-evaluations. Everyone received the same instructions and performed the same simulation which was videotaped for review. The instructor was blinded to each arm and the simulation was repeated after four months. The primary outcome was the change in instructor OSATS score over the study period.
Results: Fifteen residents were enrolled and completed the study. There was no difference in the change in instructor OSATS score (p=0.726). There was a correlation in the learner and instructor score for time and motion (p= 0.02) and instrument handling (p=0.008). All participants reported that self-evaluation was a useful educational tool. Only participants from the experimental group reported practicing on their own time.
Discussion: The current study attempted to demonstrate the utility of self-evaluation on surgical education. The correlation between learner and instructor scores are consistent with the literature. Self-evaluation did not improve overall change in score; however, all participants found it useful and it did increase practice at home.
SES, 2020, Resident, Faculty, Clerkship Director, Residency Director, Practice-Based Learning & Improvement, GME, Assessment, Simulation, Independent Study,
Read more »
Simulation Increases Anticipated Usage of Trigger Point Injections in Treatment of Myofascial Abdominal Pain
PRECIS: Evaluation of a simulation model and didactic training session for the education of ACGME residents in trigger point injections for myofascial abdominal pain in chronic pelvic pain patients.
PURPOSE: To test a training module for the education of abdominal trigger point injections
BACKGROUND: For the treatment of chronic abdominal and pelvic pain, training in interventions is lacking among trainees at the residency level. One very effective and simple intervention is abdominal trigger point injections for the treatment of abdominal myofascial pain syndrome, present in 74% of women in chronic pelvic pain practices.
METHODS: This study evaluates an abdominal trigger point teaching model for the training of USMLE OBGYN residency level physicians, containing a multimedia didactic presentation and a gelatin-based abdominal wall injection model. Participants completed a 10-item knowledge pre- test and an 8-item participant experience questionnaire gauging prior knowledge and experience with myofascial pain syndrome and abdominal trigger point injections. After 5 minutes of unsupervised time with the gelatin model, a 30-minute scripted didactic session was given, participants interacted post-training with the simulation model, and a post-test was completed.
RESULTS: Trainees improved from pre-test (48%) to post-test scores (90%) and reported increasing confidence levels on a 5-point Likert scale from 1.67 pre-test to 3.7 post-test. Ninety percent of participants agreed or strongly agreed that this exercise would result in them using abdominal trigger points in their own practice.
DISCUSSION: A simple educational tool containing a short didactic educational module and gelatin simulation model increases knowledge, confidence level and the likelihood of USMLE OBGYN residents to use trigger point injections in the treatment of chronic pelvic pain.
SES, 2020, Resident, Faculty, Patient Care, Medical Knowledge, Practice-Based Learning & Improvement, GME, CME, Assessment, Simulation, Lecture, Gynecologic Oncology, Pediatric & Adolescent Gynecology, Minimally Invasive Surgery, Reproductive Endocrinology & Infertility, General Ob-Gyn,
Read more »
Validation of the Animal Tissue Model for Laparoscopic Salpingectomy Training
Purpose: To gather validity evidence for a laparoscopic salpingectomy
model developed for resident training.
Background: Laparoscopic salpingectomy is a milestone procedure
for gynecologic trainees and is commonly performed in the management of ectopic
pregnancies. Opportunities to practice the use of electrosurgical devices prior
to the operating room can be limited. An animal tissue model was developed and
previously piloted at our institution for this purpose, and was modified to
include a pelvic sidewall and ovaries.
Methods: 25 simulations of laparoscopic salpingectomy were
performed by 14 trainees (5 PGY-1, 4 PGY-2, 1 PGY-3, 4 PGY-4). For
analysis purposes the PGY-3 and PGY-4 trainees were combined. Each
trainee received scripted instructions and reviewed a video with proper
technique beforehand. These recordings were evaluated by 2 blinded gynecologic
surgeons using the Global Operative Assessment of Laparoscopic Skills (GOALS)
and Objective Structured Assessment of Laparoscopic Salpingectomy (OSA-LS).
Results: There was good agreement between the graders with
intraclass correlation coefficients of 0.63 and 0.79, respectively, for GOALS
and OSA-LS. The mean GOALS score increased with experience (10,
13.2, and 15.5 for PGY-1, PGY-2, and PGY-3/4, respectively) and was
significantly different between PGY-1 and PGY-3/4 (two-sample t-test, p=0.004).
The mean OSA-LS score increased with experience (8.9, 14.2, and 16.2 for PGY-1,
PGY-2, and PGY-3/4) and was significantly different between PGY-1 and PGY-2
(p=0.006) and between PGY-1 and PGY-3/4 (p< 0.001).
Discussions: This training model and rubrics differentiated
experienced and novice trainees and could be used to establish a preoperative
CREOG & APGO Annual Meeting, 2020, Student, Resident, Faculty, Residency Director, Practice-Based Learning & Improvement, GME, Assessment, Simulation, Minimally Invasive Surgery, General Ob-Gyn,
Read more »
Pilot Study of Interactive Video-assisted Laparoscopic Teaching for OB/GYN Surgical Residents
Purpose: It is common knowledge that athletes review tapes as
part of training. Although differences between a sports game and surgery are
obvious, both require technical skill and performing under pressure. Surgeons
may benefit from reviewing surgical tapes in training.
Background: Prior studies have explored using surgical videos to
increase exposure to operative scenarios or evaluate technical skill. However,
no standardized approach exists to incorporate video review into surgical
training, particularly in gynecology. This interactive teaching model uses
review of laparoscopic videos to enhance surgical knowledge and technique.
Methods: A gynecologic surgeon led participants through review
of laparoscopic gynecologic surgery videos. The surgeon highlighted aspects of
the video, pausing to enforce teaching points and lead discussion. Participants
completed a 25-question multiple-choice survey prior to, immediately after and
two months after the session. Individual changes in scores were analyzed before
and after aforementioned intervention, stratified by residency year.
Results: Of participants who underwent the
intervention, 100% completed the pre-survey, 60% completed the immediate
post-survey. Response rate was 40% for the 2-month post-session survey. Of
those who underwent intervention and completed the post-session survey, 71%
improved their score while 29% had no change. When stratified by year, junior
residents showed score improvement, senior residents had no change.
Discussions: Video-assisted teaching can
enhance gynecologic surgical training. It seems most beneficial
earlier in training as junior residents have yet to develop ways of doing
things. While video sessions do not replace OR time, they provide forums for
reviewing surgical approach and technique with visual aids to reinforce
CREOG & APGO Annual Meeting, 2020, Resident, Faculty, Residency Director, Systems-Based Practice & Improvement, Practice-Based Learning & Improvement, GME, Assessment, Team-Based Learning,
Read more »
Text vs Email for a Question of the Day: Which do Today’s Medical Students Prefer?
Purpose: Compare completion rates and satisfaction of
question of the day(QOD) received via text vs email.
Background: Fast, hands on learning e-tools are a popular
effective technique with current generation of medical learners.
Methods: Twenty-eight questions of the day(QOD) developed
focusing on commonly missed NBME exam concepts. A prospective cohort study of
ob/gyn clerkship students conducted. Students self-selected to receive QOD by
text or email from 1/19-7/19. Students received QOD nightly on weekdays via
text or email, completion was optional.
Primary outcome was overall response rate(RR). Secondary outcomes include
RR/question and average weekly RR. Qualitative feedback regarding satisfaction,
time spent, and utility also obtained.
Results: Eighty-one medical students participated, 40 selecting
text and 41 email. In sum, 863 texts and 1269 emails sent during study
period. Overall text RR was significantly lower than
email(55.2%vs78.6%,p< 0.001). For no question was text RR greater than email
RR. Average RR did not vary by week(wk1: text 55% vs 83% email, wk6: text 64%
vs email 77%).
43/81(53%) students gave qualitative feedback, 28/41 from email cohort, and
15/40 text. 86% found the QOD helpful/very helpful(no difference by mode of
receipt). However, email recipients more likely to find it very
helpful(54%vs13%,p=0.02). 77% found the level of difficulty to be “just
right”,(no difference by mode of receipt). The QOD took all students less than
5 minutes. Nearly all(93%) students found the QOD worthwhile and 98% would
recommend to future students.
Discussions: Surprisingly, QOD completion rates were higher with
email than text, though satisfaction and perceived utility high in both
CREOG & APGO Annual Meeting, 2020, Student, Faculty, Clerkship Director, Medical Knowledge, Practice-Based Learning & Improvement, UME, Assessment, Independent Study, General Ob-Gyn,
Read more »
Spellbound: Impact of a Professionalism Workshop on Abortion Attitudes for Residents at Catholic Programs
Purpose: To determine if a professionalism workshop that
explores attitudes about abortion care influences ob-gyn resident attitudes at
Catholic residency programs.
Background: Catholic residency programs face institutional
barriers to abortion training.
Methods: Between 2018-2019, we provided professionalism
workshops at five Catholic programs that do not provide abortion training.
Participants received a pre and post survey related to abortion training and
attitudes. We compared Likert responses using Wilcoxon matched pair
Results: 41 residents (87% of eligible residents) completed
both surveys; 29.3% reported Catholic religion, 14.6% reported their personal
views on reproductive care are in line with their institution, and 12.2% reported
selecting a Catholic program based on its religious affiliation. With
respect to prior abortion experiences, 75.6% provided options counselling,
41.5% provided abortion referral, and 9.8% provided first-trimester abortion
for nonmedical reasons. Views on whether abortion or related
circumstances is justifiable did not change for a patient presenting with an
undesired pregnancy (p=0.32), but demonstrated increasing acceptance for the
following circumstances: (1) patient presenting for abortion at 19 weeks
(p=0.001), (2) patient presenting for second abortion (p=0.01), and (3) patient
declining post-abortal contraception (p< 0.001). Emotional reactions
to scenarios were unchanged. When asked about when abortion is morally
acceptable, views became more accepting for patients presenting with financial
inability (p< 0.01) and for disruption to career or education (p< 0.01).
Discussions: A professionalism workshop can be a useful tool for
residents at Catholic training programs to explore their abortion attitudes and
results in more accepting attitudes toward women who choose abortion in certain
CREOG & APGO Annual Meeting, 2020, Resident, Faculty, Residency Director, Professionalism, Systems-Based Practice & Improvement, Interpersonal & Communication Skills, GME, Assessment, Public Health, Contraception or Family Planning,
Read more »
Residents as Preceptors: Developing a Teaching Curriculum for Obstetrics and Gynecology Residents
Background: Residents serve a critical role in medical student
education as near-peer educators. Positive experiences with residents and
perceived quality of teaching have been associated with improved clerkship
satisfaction and career choices in that discipline. ACOG and ACGME milestones
further emphasize the importance of residents as educators. While the role of
residents as preceptors is widely acknowledged, preparation for that role is
Methods: A single arm interventional study with pre and post
intervention analyses was used. A four part lecture series was designed and
implemented during obstetrics and gynecology resident didactics at the
University of Minnesota. Residents completed blinded and paired online
surveys pre and post lecture. 5 point Likert scale responses
were summarized and compared using Wilcoxon Signed Rank and Nominal Symmetry
Results: There was a statistically significant
difference in the distribution of rankings towards the positive between pre and
post surveys for questions regarding evaluation and teaching
. There were no statistically significant differences in the distribution of
rankings for questions regarding feedback. Barriers to teaching were also
Discussions: Implementation of a teaching curriculum significantly
improved residents\' perceived preparedness and understanding of medical student
teaching and preceptorship but did not improve attitudes towards and frequency
of feedback. Strengths of this study include description of a novel, resident
led program. Limitations include a small sample size at a single institution
within a single specialty.
CREOG & APGO Annual Meeting, 2020, Resident, Faculty, Clerkship Director, Residency Director, Professionalism, Interpersonal & Communication Skills, GME, Assessment, Lecture, General Ob-Gyn,
Read more »
Resident Surgical Evaluations: Feasibility of Integration into the Electronic Medical Record (EMR)
Purpose: Describe the development and implementation of a
surgical evaluation tool within the Epic EMR to facilitate real-time feedback
for resident learners.
Background: Evaluation of a resident’s surgical performance is
critical to the developing surgeon’s education but challenging for both
learners and teachers in a busy clinical environment. This study aims to describe
how a surgical evaluation tool can be integrated into the EMR and the
acceptability of the tool to both faculty and residents.
Methods: A novel tool was developed within the EMR of a single
academic hospital that triggers an automated in-basket message to the attending
surgeon upon case completion. This message directs the attending surgeon to
complete a 4 question evaluation consisting of 2 validated feedback questions
and 2 open ended questions. Upon completion, an email is sent providing residents
with immediate feedback. At the end of the study period, a survey was sent to
attending surgeons and residents to assess the acceptability of the tool.
Results: Between February 2019 and June 2019 719 cases were
performed and 553 surgical evaluations were completed (77%). The acceptability
survey was completed by 26 of 27 (96%) of residents and 14 of 16 (88%) faculty.
Among residents, 96% reported that they received more feedback because of the
tool and 85% liked receiving feedback in this way. Among faculty, 21% reported
that they felt they gave more feedback because of the tool and 93% found the
tool acceptable to use.
Discussions: Implementation of an automated surgical evaluation
tool within the EMR is feasible, acceptable to use, and well received by
CREOG & APGO Annual Meeting, 2020, Resident, Faculty, Osteopathic Faculty, Residency Director, Residency Coordinator, Patient Care, Medical Knowledge, Interpersonal & Communication Skills, Practice-Based Learning & Improvement, GME, Assessment, General Ob-Gyn,
Read more »
Resident Confidence with Obstetric Ultrasound – Are We Meeting CREOG Objectives and ACGME Milestones?
Purpose: The objective of this study assess resident confidence
in obstetric ultrasound as per ACGME milestones and CREOG
objectives, and factors associated with confidence performing US independently.
Background: Performing obstetric ultrasound(US) is an ACGME
milestone, a CREOG objective, and a CREOG core procedure, though many programs
lack formal US curricula.
Methods: CREOG coordinator listserve used to distribute
27-question anonymous e-survey in 3/2018 to OBGYN residents in US and Puerto
Rico. Questions included experience performed growth US(gUS) and anatomy
US(aUS) as primary sonographer, performing US independently, and for 4th year
residents only, comfort performing US post-residency. Bivariate statistics
compared residents comfortable performing US independently to those who were
Results: 417 residents completed the survey: 88% were female,
75% from academic programs, and nearly 50% were post-graduate year(PGY)
3&4. While 89.1% had been primary sonographer for gUS, only 63% felt
comfortable performing independently. Of chief residents, 14% didn’t feel
comfortable performing gUS post-residency. In contrast, 60.9% had been primary
sonographer for aUS, only 20.1% felt comfortable performing aUS independently,
and 68.8% didn’t feel comfortable performing aUS post-residency.
Residents were more likely to feel comfortable performing gUS independently in
PGY3&4 (p< 0.01) and after longer MFM rotations(p=0.02), and less likely
when MFM fellows were present (p=0.02). Residents not comfortable performing
aUS were twice as likely to have MFM fellows (p< 0.01), and less likely to
be PGY3&4(p< 0.01).
Discussions: significant percentage of residents are not
comfortable with their ultrasound skills. This data further emphasizes the need
for structured US curricula and simulation.
CREOG & APGO Annual Meeting, 2020, Resident, Faculty, Residency Director, Patient Care, Practice-Based Learning & Improvement, GME, Assessment, Simulation, Maternal-Fetal Medicine, General Ob-Gyn,
Read more »
Impact of TrueLearn Question Bank on Resident Performance on the CREOG In-Service Training Exam
Purpose: To determine how implementation of the TrueLearn (TL)
question bank and mock in-service training exam (ITE) impacts performance on
the CREOG ITE.
Background: The CREOG ITE is an annual exam on which, success has
been correlated to success on the American Board of Obstetrics and Gynecology
Qualifying Exam. Question banks are quickly becoming a popular study tool, but their
utility on improving CREOG ITE performance is unknown.
Methods: We performed a quasi-experimental retrospective review
of the CREOG ITE score reports from 2017 to 2019 at our institution. Starting
in 2018, TL was made available and a mock ITE was administered for the first
time. A two-sided paired t-test compared average annual improvement in CREOG
ITE scores before and after TL implementation. Correlation coefficients between
TL usage and improvement on the 2019 ITE were calculated using Pearson correlation
testing and a linear regression was used to determine their association.
Results: After providing TL, mean improvement in percentage
points on the CREOG ITE was 6.6, 9.33 and 6.75 as residents matriculated from
PGY1-2, PGY2-3 and PGY3-4 respectively. This was a significant improvement compared
to the previous year. As a program there was a moderate correlation with
TL usage and percentage point improvement on the CREOG ITE; r=0.38, p= 0.038.
Discussions: After providing universal access to TL question bank
and administering a mock ITE, a statistically significant improvement on the
CREOG ITE was seen. TL’s question bank and mock ITE are valuable resources for
improving CREOG ITE performance.
CREOG & APGO Annual Meeting, 2020, Resident, Faculty, Clerkship Director, Clerkship Coordinator, Osteopathic Faculty, Residency Director, Residency Coordinator, Medical Knowledge, Systems-Based Practice & Improvement, Practice-Based Learning & Improvement, GME, Assessment, Independent Study, Problem-Based Learning, General Ob-Gyn,
Read more »
Impact of a Pediatric Adolescent Gynecology (PAG) Learning Module: A Multi-Institutional Randomized Controlled Study
Purpose: To determine if an online PAG learning module improves
resident knowledge and clinical performance.
Background: CREOG recommends education in PAG, yet resident
training is often inadequate secondary to insufficient specialists and clinical
Methods: We conducted a multi-institutional single-blinded
randomized controlled trial across four university programs; three had PAG
rotations and two had PAG fellowship-trained faculty. Our intervention
was a previously validated 20-minute electronic module. We applied block
randomization across institutions in a 1:1 ratio. All residents
subsequently completed a PAG-related knowledge assessment (maximum score=5)
that queried understanding of pre-pubertal bleeding and an objective structured
clinical examination (OSCE, maximum score=16) that assessed history collection
and performance of a genital exam, vaginal culture, and vaginoscopy for a
pediatric patient. OSCEs were videotaped and reviewed by two blinded
faculty; inter-rater reliability score was 96.5%. We calculated
frequencies and compared composite knowledge assessment and OSCE scores using
Results: Amongst 115 invited residents, 97 (83.4%)
participated; 45 were randomized to the module with equal representation across
training levels. Majorities were female (90.7%) and reported no
prior PAG didactic (77%) or clinical experiences (77%-87.6%); randomization
groups were similar. The intervention group scored significantly higher
on both the knowledge assessment (4 versus 2, p < 0.001) and the OSCE (13
versus 7, p < 0.001). Notably, the intervention group was more likely
to appropriately avoid using a speculum in a pediatric patient (95.6% vs.
57.7%, p < 0.001).
Discussions: Our self-study PAG module resulted in improved
knowledge and clinical approaches and may be of benefit to other training
CREOG & APGO Annual Meeting, 2020, Student, Resident, Faculty, Residency Director, Residency Coordinator, Patient Care, Medical Knowledge, Systems-Based Practice & Improvement, Practice-Based Learning & Improvement, GME, Assessment, Problem-Based Learning, Standardized Patient, Pediatric & Adolescent Gynecology,
Read more »
Forceps Simulation Training Improves Knowledge, Skills and Attitudes of OB/GYN Residents in an Academic Medical Center
Purpose: To improve resident “readiness” for clinical training
opportunities in forceps delivery.
Background: Exposure to forceps-assisted delivery is declining
nationwide. In the clinical learning environment, forceps training
opportunities are often acute/emergent. Simulation training may help residents
feel more prepared for such opportunities.
Methods: Eleven residents (PGY1-4) underwent 8 hours training
on forceps delivery over 6 months. The curriculum included a 2-hour didactic
session (covering case selection, clinical evaluation, safety checks and global
second stage awareness) followed by three 2-hour simulation training sessions
utilizing “Lucy and Lucy’s Mum” (MODEL-med, Australia) at 3-monthly intervals.
Knowledge/confidence scores (10-point VAS) were assessed at baseline and 3/6
months later. A 20-point skills checklist was used to objectively score
residents’ procedural competency before each simulation session as well as 2
weeks after the first.
Results: Knowledge scores increased from 53±5.4% pre-training
to 64±5.3% at 3 months and 77±6.8% at 6 months (p=0.007). Confidence scores
increased from 22±8.3% pre-training to 33±9.2% at 3 months and 49±9.6% at 6
months (p=0.046). Skills assessment scores were 47±7.2% at baseline and
increased to 76±5.0% two weeks after the first simulation (p< 0.01). Scores
remained higher at 3 months (79±2.3%, p< 0.001) and 6 months (93±0.9%, p<
0.001). Pre-training there was an effect of seniority (p=0.002) with lowest
scores for PGY1 (12±0.9%) and highest for PGY4 (59±9.3%). Post-training scores
were independent of PGY level (p=0.605).
Discussions: Structured simulation training increases residence
knowledge, skills and confidence levels with respect to forceps delivery and
may help supplement clinical learning opportunities to protect against loss of
this valuable skill.
CREOG & APGO Annual Meeting, 2020, Resident, Faculty, Osteopathic Faculty, Residency Director, Patient Care, Medical Knowledge, GME, Assessment, Simulation, Quality & Safety,
Read more »
Evaluation Disparity and Implicit Bias in Medical Student Grading: Findings from Evaluator Focus Groups
Purpose: To perform a needs assessment of medical student
evaluators to uncover reasons for evaluation disparity in order to work towards
improving the quality, consistency, and accuracy of OB/GYN clerkship
Background: Accurate evaluation of medical student clinical
clerkship performance is challenging. Scale-based student ratings often do not
correlate to comments provided. Additionally, there can be a wide variance of
scores for the same student by different evaluators. Information regarding
differing grading systems is available, but there is a lack of guidance around
inter-rater reliability and implicit bias.
Methods: From May - July 2019, we performed three resident and
faculty focus groups. We began the session discussing evaluation form
reliability and current behaviorally anchored rating scales, same student
evaluation inconsistencies demonstrated between evaluators, accuracy, and
potential evaluator bias. Qualitative analysis utilizing two reviewers was used
to identify themes.
Results: Overall, participants expressed a need for more formal
student evaluation information and instruction. Other themes were requests for
education and information regarding the evaluation forms and student grading,
an admission of variation between ‘hard’ and ‘easy’ graders, and implicit bias
in student evaluation, particularly regarding perceived specialty choice.
Discussions: Evaluators felt a need to better understand how
evaluations contributed to student rotation grades and were interested in
improving the quality, consistency, and accuracy of evaluations. Time with
students directly influences the accuracy of objective evaluations. Implicit
bias around one’s own ‘grading scale’, and around student specialty selection,
can influence student evaluations.
CREOG & APGO Annual Meeting, 2020, Student, Resident, Faculty, Clerkship Director, Clerkship Coordinator, Professionalism, Interpersonal & Communication Skills, UME, Assessment,
Read more »
Empathy Training Using Improvisational Comedy: Impact on Multidisciplinary Education among Health Professions Students
Purpose: Determine the impact of improvisation training on
self-reported and observed measures of empathy among a multidisciplinary group
of health professions students
Background: Provider empathy improves patient outcomes. The most
effective technique for promoting empathy remains unknown. Improvisation
training may improve cognitive and affective empathy by enhancing one’s ability
to react to the observed experience of others. Improvisation workshops have
been shown to improve empathy in OB/GYN residents. We aimed to evaluate the
impact this training on self-reported and observed measures of empathy within a
multidisciplinary group of health professions students.
Methods: This is a prospective cohort study of health
professions students who participated in a mixed-methods empathy training
activity (improvisation exercises and case-based scenarios administered by
standardized patient actors before, during, and after the educational component).
Students completed pre- and post-intervention self-reported empathy
questionnaires. Trained patient actors observed student interactions and
completed validated empathy questionnaires following each case-based interaction.
Paired t-tests were used for within-person comparisons over time and ANOVA was
used to compare changes across groups.
Results: Twenty-eight students participated (4 medical; 8
social work; 11 nursing; 5 physical therapy). Self-reported empathy scores
improved significantly from pre- to post-intervention (34.3 vs 38.2, p<
0.0001) although observed empathy scores did not change (43.6 vs 42.9, p=0.65).
When comparing across disciplines, there was no difference in observed empathy
Discussions: Improvisation training improves self-reported but not
observed empathy among a multidisciplinary group of health professions
students. There was no difference in the efficacy of the training workshop
CREOG & APGO Annual Meeting, 2020, Resident, Faculty, Clerkship Director, Clerkship Coordinator, Osteopathic Faculty, Interpersonal & Communication Skills, GME, Assessment, Simulation, Team-Based Learning,
Read more »
Duty Hours and Medical Student Performance: Is There a Correlation?
Purpose: To describe the association between duty hours (DH)
and shelf exam score, and between DH and overall grade.
Background: Medical students are required to comply with the same
DH regulations as residents. However, variation exists between the total DH
worked due to diversity of clinical sites, surgical rotations, and student
engagement. The impact of DH on exam scores and clinical grade is poorly
Methods: Retrospective cohort study of students in OB/GYN
clerkship at a single academic center from 8/18-6/19. Weekly duty hours
tabulated and averages obtained. Students working more than 75%ile for DH
weekly (long hours) compared to those working less. Impact of long hours in
last two weeks of clerkship also examined. Outcomes of interest were high
overall grade ( >95%), high shelf score ( >90%), and low shelf score
(< 60%) adjusted for quartile of year. Simple statistics used to analyze data.
Results: The average weekly DH was 43.1hr/wk (SD 7.32hr). This
didn’t vary by gender. Students in the latter half of the year worked longer
(45.2 hrs/wk vs 41.9 hrs/wk, p=0.03). The 75%ile of weekly DH was 47.1 hrs.
Students working long hours were not more likely to have low shelf (21.7% vs
30.4%,p=0.6), high shelf (28.3% vs 28.6%,p >0.99), or high overall grade
(25% vs 28.9%,p >0.99). Long hours in the last 2 weeks of the clerkship was
not associated with high overall grade (33.3% vs 31.1%,p >0.99) or low shelf
(21.7% vs 34.2%,p=0.32). A trend toward high shelf scores was seen (41.3% vs
Discussions: Long duty hours, even late in the clerkship, were not
associated with high or low shelf scores, nor high overall grade.
CREOG & APGO Annual Meeting, 2020, Student, Resident, Faculty, Clerkship Director, Clerkship Coordinator, Medical Knowledge, UME, Assessment, General Ob-Gyn,
Read more »
Developing Four Year Surgical Skill Learning Curves: Modeling a Process in OB/GYN Residency Using myTIPreport
Purpose: To develop a process for procedural learning curve
construction in a national cohort of OB/GYN residents from PGY1 through PGY4
Background: Descriptions of resident surgical proficiency have
been limited by subjective experience and surrogates such as operative time or
numbers of completed cases. Objective data describing number of procedures
required to achieve proficiency for OB/GYN procedures are limited.
Methods: This descriptive study was a secondary analysis of the
myTIPreport database from July, 2015 – June, 2019 for the graduating PGY-class
of 2019. Feedback included in this database was characterized by residents
receiving procedure-specific surgical proficiency scores, rated 1-5. Surgical
proficiency was defined as Level 4 (“Supervision Only”). Using database time
stamps, chronologic case numbers were generated. Fitted Linear Modeling
generated an approximate number of cases to achieve proficiency. To reduce type
I error, specific procedures were examined only when 100 or more feedback
encounters were reported. Procedures were subsequently excluded when the
model-predicted number of cases needed to achieve proficiency was not achieved
or when a negative modeling slope was noted.
Results: Of the 3599 feedback encounters reported, 5 procedures
met inclusion criteria. Using the above modeling for these 5 procedures, the
number of cases needed to achieve proficiency, a Level 4 rating, was: D&C
-9, operative hysteroscopy-11, diagnostic laparoscopy-9, laparoscopic BSO-11,
Discussions: These data demonstrate a process using myTIPreport to
begin the innovative work of procedural learning curve construction. Such a
process could ultimately be used to build program-specific learning curves and
potentially help identify at-risk learners.
CREOG & APGO Annual Meeting, 2020, Resident, Faculty, Residency Director, Patient Care, Systems-Based Practice & Improvement, Interpersonal & Communication Skills, GME, Assessment,
Read more »
Developing Committed Resident Leaders: A Survey of the Obstetrics and Gynecology Administrative Chief Resident Experience
Purpose: To define the challenges and experiences of Obstetrics and Gynecology (OBGYN)
administrative chief residents (ACRs) and identify support needed to make them
Background: ACRs are integral to OBGYN residency programs, however little research has been
done on the experiences, training, and support of ACRs.
Methods: A Qualtrics survey was emailed to program coordinators at all 292 ACGME OBGYN
residency programs. Response styles included MCQs, yes/no, likert scales, and
short answer. Question topics included demographics, chief duties, support, and
experiences. Study participants were defined as current / past ACRs (2019-2020,
2018-2019) and program directors (PDs).
Results: Ninety two ACRs and 24 PDs received the survey, as confirmed by program
coordinators. Forty five surveys were completed (response rate 39%), including
38 (84%) ACRs and 7 (16%) PDs.
The majority (67%) were from academic programs with a median of six residents
and two ACRs per year. Forty-nine percent of respondents reported ACRs spend 4-7
hours per week on administrative duties. Eighty seven percent reported no
protected time. On a 10-point likert scale, mean perceived support from
faculty, PD, administration, and other residents was 6.8, 8.6, 7.1, and 7.5
respectively; however, mean stress level was 7.1.
Common challenges reported by ACR’s are conflict resolution and workload
management. Many ACRs felt they would benefit from formal leadership training
and protected time.
Discussions: These results summarize the experience of ACRs. Best practices and formal
training in identified challenge areas should be added to residency curriculum
and used to develop toolkits to support ACRs nationwide.
CREOG & APGO Annual Meeting, 2020, Resident, Faculty, Residency Director, Residency Coordinator, Professionalism, Interpersonal & Communication Skills, GME, Assessment, Team-Based Learning,
Read more »
Curriculum Development in the OBGYN Clerkship: Effect on NBME Scores
Purpose: Investigate whether implementation of APGO Medical
Student Educational Objectives into an OBGYN clerkship didactic curriculum
results in higher NBME exam scores.
Background: Medical student curriculum reform is at the forefront
of OBGYN educational research. Active, adult learning techniques are highly
touted; data are promising for student satisfaction, but more studies about
effect on objective assessments are needed.
Methods: We implemented 21 of 58 APGO Medical Student
Educational Objectives into the OBGYN Clerkship didactic curriculum during
academic year (AY) 2018-2019. Students were assigned pre-reading material and
completed weekly, graded individual readiness assessment tests, followed by
team readiness assessments, before each didactic session. NBME scores were
compared pre- and post-intervention (AY 2017-2018 to AY 2018-2019, respectively)
by yearly cumulative, as well as by individual cohort within each year.
Analysis was completed with independent samples t-test with two-sided P <
0.05 statistically significant.
Results: Overall, AY 2018-2019 had higher average NBME scores
than AY 2017-2018, although not statistically significant (79.5 vs 78.6,
P=0.29). Cohorts 4 and 6 showed significantly higher scores in AY 2018-2019
relative to AY 2017-2018, 80.0 vs 75.4 and 82.1 vs 77.7 respectively, (P<
0.05). Cohorts 3 and 5 scores were slightly higher on average and cohorts 1 and
2 scores were lower on average in AY 2018-2019 relative to AY 2017-2018, but
these differences were not found to be statistically significant.
Discussions: Implementation of the APGO Medical Student Education
Objectives into our OBGYN didactic curriculum showed a trend toward higher NBME
scores, suggesting benefit, however this difference was not statistically
CREOG & APGO Annual Meeting, 2020, Student, Faculty, Clerkship Director, Clerkship Coordinator, Medical Knowledge, GME, Assessment, Lecture, Team-Based Learning, General Ob-Gyn,
Read more »
Care Across the Life Cycle: Results of an Integrated Pediatric, Obstetrics and Gynecology, and Geriatric Medical Student Clerkship
Purpose: The authors explore medical knowledge attainment and
course evaluation data in Pediatrics and Obstetrics and Gynecology after the
implementation of a novel, three-phase undergraduate medical school education
curriculum called ForWard. In ForWard, the two clerkships were shortened from
six to four weeks and combined with geriatrics to form an integrated 12-week
Background: The traditional structure for clinical clerkships are
specialty specific rotations. Due to evolving needs of both patients and
learners, there has been interest in integrating the clinical fields during
delivery of undergraduate medical education. With any curricular change, it is
important to evaluate the learning outcomes.
Methods: Pediatric and Obstetrics and Gynecology NBME subject
exam scores were compared prior to (n=180) and after (n=162) the implementation
of the ForWard curriculum using an analysis of variance with covariate
(ANCOVA). Student satisfaction with the traditional and ForWard
curriculum were determined based on post-clerkship evaluation forms and
compared using unpaired t-tests.
Results: The unadjusted mean NBME Pediatrics and Obstetrics and
Gynecology subject examination scores were 77.1 and 80.0 in the traditional
curriculum and 76.9 and 79.3 in the ForWard curriculum (p=0.83) and (p=0.37).
The mean Pediatric and Obstetrics and Gynecology clerkship evaluation scores in
the traditional curriculum were 77.1 (SD 7.6) and 80.0 (SD 6.9) compared
to 76.9 (SD 7.9) (95%CI -1.84-1.47) and 79.3 (SD 7.7) (95%CI -2.26-0.85) in the
Despite shortened Pediatric and Obstetrics and Gynecology
clerkship experiences, performance on the NBME subject examinations in both
subjects was unchanged and student satisfaction was maintained.
CREOG & APGO Annual Meeting, 2020, Student, Resident, Faculty, Clerkship Director, Clerkship Coordinator, Osteopathic Faculty, Medical Knowledge, UME, Assessment, General Ob-Gyn,
Read more »