Back to search

McMaster University

An Evaluation of the Utility of a Hybrid Objective Structured Clinical Examination for the use of Assessing Residents Enrolled in McMaster University's Orthopaedic Surgery Residency Program

Abstract

dc:description.abstract

<p><strong>Introduction: </strong>McMaster University’s orthopaedic surgery residency program implemented the OSCE as an assessment tool in 2010; this study evaluates the first four OSCEs administered to residents. The OSCEs were composed of knowledge-testing stations, which are normally not included in this testing format, and performance-testing stations. Recruiting enough faculty evaluators challenged the ability to feasibly implement this examination format. Knowledge-testing stations were incorporated since they do not require evaluators to be present. Reliability was assessed, and the correlation between knowledge-testing station scores and performance-testing station scores was determined. The ability of the OSCE to discriminate between residents in different post-graduate years (PGYs) was assessed. Residents’ acceptability of the OSCE was also assessed. <strong>Methods: </strong>Reliability was assessed using generalizability theory. The correlation of knowledge-testing and performance-testing station scores was measured with Pearson’s r. A two-way ANOVA was used to analyze whether the OSCE can discriminate between residents in different PGYs. An exit survey was administrated after each OSCE to assess acceptability. <strong>Results: </strong>The generalizability estimates of each OSCE ranged from 0.71 to 0.87. The disattenuated correlation between knowledge- and performance-testing stations for senior residents was 1.00, and 0.89 for junior residents. A significant effect of year of residency was found for the October 2010 OSCE in the ANOVA (F(1,30) = 11.027, p = 0.005), but the remaining OSCEs did not replicate this finding. In general, residents felt that they were able to present an accurate portrayal of themselves in the OSCEs and that the examination covered appropriate topics. <strong>Discussion: </strong>The OSCEs were reliable and acceptable to residents. The high correlations between knowledge- and performance-testing station scores suggest that the examination can be made more feasible by including knowledge-testing stations. The small sample sizes made significant differences difficult to detect between levels of training, resulting in inconclusive evidence for this construct validation measure.</p>

Degree

thesis:*
Department dc:contributor.department
Health Research Methodology
Year dc:date.issued
2014

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Gavranic, Vanja
Advisor dc:contributor.advisor
  • Dore, Kelly

Subjects

dc:subject × 6

Identifiers

dc:identifier.*
Dc Identifier Other
opendissertations/8694
9780
4955857

Chain of custody

source
Harvested from
McMaster University
Base URL
macsphere.mcmaster.ca/server/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Gavranic, Vanja. An Evaluation of the Utility of a Hybrid Objective Structured Clinical Examination for the use of Assessing Residents Enrolled in McMaster University's Orthopaedic Surgery Residency Program. 2014. http://hdl.handle.net/11375/13859