Improving Continuing Education Outcomes

Improving Continuing Education Outcomes

Test-Enhanced CE – A Randomized Controlled Trial

Enhancing the retention and application of knowledge is a challenge for Continuing Professional Development (CPD). Evidence from educational psychology suggests that test-enhanced learning (TEL) is a potential learning strategy that can increase durability of CPD learning. TEL is consistent with annual recertification of medical specialists, which requires documentation of CPD that includes assessment and feedback. How best to deploy testing and its impact on knowledge is still poorly understood. Mark Feldman MD, FRCPC, Oshan Fernando PhD, Michelle Wan MA, Tina Martimianakis PhD, and Mahan Kulasegaram PhD investigated the use of a pre and post MCQ TEL package for CPD and the impact on knowledge as well as self-reported learning behaviours.

For the study, the researchers used an randomized controlled-trial comparing knowledge retention among learners who registered for an annual CPD update conference at the University of Toronto. Participants were randomized to receive pre-workshop multiple choice question (MCQ) testing (no feedback) followed up by post workshop MCQ tests (with feedback) or randomized to the control group, to receive no testing. Neither the pre- nor post-test MCQs were used as outcome measures; they were the essence of the educational intervention.

In June 2016 a clinical vignette-based retention and application test was delivered to the control group and the TEL group. Retention testing results was the primary outcome of interest. Secondary outcomes examine efficiency of the TEL package, measuring cost, satisfaction, self-reported changes in learning behaviour and completion rates.

300 physicians from across Canada registered for the four-day conference. 186 physicians consented to participate in the study and were randomized to receive pre & post-testing for some workshops while randomized to the control group for others. As few as 12 learners participated in smaller workshops while as many as 80 participated in larger workshops. Retention test results for each workshop were analyzed as an individual ‘study’. A meta-analysis was performed to determine the overall effect of the TEL package. Analyzed as a group, the TEL package recipients scored significantly higher than controls on retention testing with a moderate effect size. Analysis of secondary outcomes is ongoing. Results are displayed in figure 1.

Testing can be leveraged across the faculty to efficiently and effectively improve the outcomes of CE. These results will fill a gap in the CPD literature internationally and improve practice for CPD in Canada. Testing remains an under-utilized education intervention in CPD and the use of formative assessment to enhance professional development should be a key target for research.