We conduct an evaluation of the cross-cultural ‘export’ of the Objective Structured Clinical Examination (OSCE), a well-established Western medical education assessment tool that is in keeping with Competency-Based Medical Education (CBME) principles, into the new socio-economic setting of Rwanda. The evaluation framework of ‘assessment utility’ is applied, where the utility of an assessment is described conceptually as the multiplicative function of its validity (V), reliability (R), educational impact (E), cost/feasibility (C) and acceptability (A). A mixed-methods approach of both quantitative and qualitative data analysis is used.
The quantitative findings support high content and face validity, high reliability, high acceptability and achievable cost and feasibility of the OSCE, all of which would suggest high utility. The analysis of qualitative data identifies some important threats to validity, namely perceived significant gaps in training in the internship program that were thought to likely be the underlying reason for the low mean assessment scores. This threat to the validity of the results appears to influence and limit the acceptability of the assessment in this context. While it is believed that it would be suitable as a formative assessment, primarily for the purpose of ‘assessment for learning’, it was not felt that it was currently acceptable as a summative or high-stakes ‘assessment of learning’, until and unless training deficits are addressed. Currently, the OSCE is seen to have greatest value in its potential for educational impact by acting as both a driver and a marker for change both at the individual and programmatic levels. Many principles of CBME and the concept of ‘entrustability’ as a criterion-referenced assessment standard were well-received cross-culturally, when training and assessment were viewed in tandem. Our study highlights the importance of using a comprehensive evaluation framework that includes both quantitative and qualitative methods to accurately characterize the utility of an assessment. / Thesis / Master of Science (MSc) / This is a case-study where the Objective Structured Clinical Examination (OSCE), a well-established, performance-based and resource-intensive Western medical education assessment tool, was introduced to the culturally different, resource-limited setting of Rwanda. What we wanted to evaluate is how the OSCE was received in the Rwandan medical training system.
What we found is that generally, the OSCE was received in a positive way. Both examiners and participants thought it was a relevant, realistic, feasible, valuable test for doctors in training. However, examiners also felt that the candidates did not do as well as they could have on the test not because they were fundamentally bad doctors, but because there were major gaps in their training. The OSCE therefore demonstrated its usefulness by identifying these deficiencies in training. Examiners felt that addressing these gaps in training was most important and should be done before any institutional body uses the OSCE results to decide who should get a medical license or not.
| Identifer | oai:union.ndltd.org:mcmaster.ca/oai:macsphere.mcmaster.ca:11375/25513 |
| Date | January 2020 |
| Creators | Misir, Amita |
| Contributors | Monteiro, Sandra, Health Research Methodology |
| Source Sets | McMaster University |
| Language | English |
| Detected Language | English |
| Type | Thesis |
Page generated in 0.2008 seconds