Judging the quality of reflective portfolios is assuming critical importance with their increasing use in accreditation and revalidation. The experience of this work has shown that despite explicit instructions to compilers, considerable investment in assessor training, and the negotiation, agreement and publication of overt criteria, individual assessments are consistent but show only fair inter-rater reliability and are untrustworthy in high stakes assessment. This study has also shown that reliability is not improved through the process of trying to 'shape' portfolios to introduce structural consistency, although it can be improved by using discussant pairs of assessors, with consequent issues of cost. While this method should be further researched as a means by which portfolios can be judged, consideration of the nature of professional practice and the processes that underpin judgements made by professionals moves the debate towards rejecting the traditional first hurdle of reliability before examining the validity of assessment tools. This thesis describes a methodology that begins the process by justifying the general direction such enquiries might be taking. Based on an exploration of the issues, it offers a model for a system for the assessment of professionals that is both theoretically supported and practically possible.
| Identifer | oai:union.ndltd.org:bl.uk/oai:ethos.bl.uk:431212 |
| Date | January 2003 |
| Creators | Pitts, John |
| Contributors | Williams, Ann ; Coles, Colin |
| Publisher | University of Winchester |
| Source Sets | Ethos UK |
| Detected Language | English |
| Type | Electronic Thesis or Dissertation |
Page generated in 0.0026 seconds