Background: Self-assessment is widely used across the health professions for a variety of purposes, including appraisal, CPD and revalidation. Despite numerous reported short-comings, the use of self-assessment is increasing, frequently on the requirements of regulatory bodies. Traditionally it has been a paper exercise, but in recent years self-assessment has appeared in electronic portfolios – a medium often used to collate assessments and other educational requirements. This thesis evaluates the effectiveness of self-assessment, in particular delivered via an e-portfolio, to determine if it: ? Improves the accuracy of perception of learning needs ? Promotes appropriate change in learner activity ? Improves clinical practice Methods: This thesis is comprised of two systematic reviews and a case study. The first of two systematic reviews examines the evidence for effectiveness of self-assessment in the three research questions. The second evaluates the effectiveness of portfolios as a medium for postgraduate healthcare education. Both reviews are notable in that they employ systematic review methodology on non-clinical questions and amalgamate quantitative and qualitative data. The final research component is an exploratory case study that tests the questions against a large data set (an entire training year of Scottish Foundation doctors) collated by the NHS ePortfolio. The case study provided the opportunity to separate groups of self-assessors identified by the literature, and compare the groups’ self-scores against those of their supervisors and peers in the first and final post rotations; additionally, the groups’ behaviour was matched against the literature for related educational activities recorded by the ePortfolio such as personal development planning. The case study also allowed the medium of e-portfolios to be itself evaluated in practice as an educational infrastructure. Through the comprehensive and iterative examination of the large dataset it became apparent that quantitative analysis was of limited value and qualitative analysis of elicited the richness on the data in context. Results: With both reviews, the original research questions were unable to be fully answered due to the paucity of evidence of sufficient quality; however, both did discover relevant related evidence. The self-assessment review found competent practitioners are the best able to self-assess whilst the least competent are the least able to self-assess. Peer assessment was found to be more accurate than self and better aligns with faculty/supervisor assessment. Feedback and benchmarking can improve self-assessment accuracy, especially for the most competent, and video can be seen to enhance this. There is no conclusive evidence that gender or culture effect self-assessment ability. Practical skills are better self-assessed than knowledge-based or “soft” skills. The portfolio review found summative assessment reliability improved with multiple raters and discussion between the raters. Evidence on whether portfolio use aided reflection was mixed, possibly because it was dependent on individual conditions. The engagement and support of supervisors is key to portfolios being used properly, and there is some evidence portfolio learners are less passive then non-users. The time required to effectively use a portfolio is rarely considered. Although many of the literature’s findings were born out by the case study, the data also revealed (often by omission) many flaws in the use of self-assessment and related activities, many of which can be ascribed to the training year examined Much of the qualitative examination of text corresponded with the wider literature with low self-raters being over-critical of their often superior skills and high self-raters being over confident. However, there was some dissonance with the literature in the final component in that supervisor scoring conflicted with expectations whilst there text comments continued to match the literature. Conclusions: Assessment in post-graduate health care is high stakes and resource-intensive. Self-assessment, and its use within an electronic portfolio, is demonstrated to have enormous potential if properly implemented.
| Identifer | oai:union.ndltd.org:bl.uk/oai:ethos.bl.uk:613578 |
| Date | January 2013 |
| Creators | Haig, D. Alexander J. |
| Contributors | Clarkson, Janet |
| Publisher | University of Dundee |
| Source Sets | Ethos UK |
| Detected Language | English |
| Type | Electronic Thesis or Dissertation |
| Source | https://discovery.dundee.ac.uk/en/studentTheses/aae612a3-774c-4546-9c8d-66bd0769c385 |
Page generated in 0.0021 seconds