In the UK examiners assign marks to A level examination scripts using extensive mark-schemes. Examiners work under strict time constraints, and must consider various sources, from script to mark-scheme to exemplar marked scripts. Essay subjects, such as History and English (two subjects which are associated with difficulty of marking), are likely to form a particular cognitive challenge for examiners, and their marking has not been extensively researched. Most examiners mark within accepted variations of reliability, as determined by Awarding Bodies’ monitoring systems. The question is, then, how they make these decisions, given the amount of information and the limited time available. The training process which is intended to bring the examiners’ decisions into line with the standard of the Principal Examiner also represents a lacuna in the literature. This study therefore sought to investigate examiners’ decision-making processes and the process of the training meeting. Five day-long standardising meetings (four examiners’ meetings and one senior examiners’ pre-standardising meeting), split between English and History, were recorded, transcribed and subjected to discourse analysis; three examiners, spread between the four units, provided additional Verbal Protocol Analysis data while undertaking live marking. A survey, which presented preliminary conclusions from that data and some extracts from examiners’ discourse, was used to collect further data from a larger sample of History and English A level examiners. The data are considered in relation to the theory of heuristics, which has been used to consider examining at other levels, or in other subjects, and with other question types. The data are also considered in the light of other theories, including those of expertise and construct-referenced assessment. The data demonstrate that decisions were not usually made in the rule-based way which is suggested as the ideal by the regulations, and which would be assumed from the mark-schemes and rubric issued by the exam boards. The mark-scheme did provide a guide to the foci which should, and can be seen to, attract examiners’ attention. However, a great deal of ‘professional judgement’ was also exercised, and examiners used a number of informal heuristics, and made relative judgements to reach a mark; comparison is established as a major mechanism of their decision-making. These behaviours do not necessarily lead to bias, however, and many were actually suggested during the training process. Some were suggested consciously by senior examiners, but some appeared to be unconsciously modelled during the training meeting. The theory of heuristics is seen to be widely applicable to the data; the choice of material and training mitigated the potential bias which heuristics could cause. A wide range of cognitive processes are demonstrated in the data, which were used to varying degrees by different examiners, at different times within and between scripts.
| Identifer | oai:union.ndltd.org:bl.uk/oai:ethos.bl.uk:595899 |
| Date | January 2011 |
| Creators | Elliott, Victoria Faith |
| Contributors | Lunt, Ingrid; Stanley, Gordon |
| Publisher | University of Oxford |
| Source Sets | Ethos UK |
| Detected Language | English |
| Type | Electronic Thesis or Dissertation |
| Source | http://ora.ox.ac.uk/objects/uuid:50e2acc6-88cc-445b-94e9-d4837ae80dce |
Page generated in 0.0022 seconds