This thesis presents a model of evaluator behaviour in usability evaluations and describes an evaluator centred approach to analytical usability evaluation assessment. Usability evaluations involving multiple evaluators often produce inconsistent results, even when using the same usability evaluation method. This would suggest that individual evaluator judgement is influenced by resources other than those provided by the usability evaluation method. This research sought to discover factors external to usability evaluation methods that influence evaluators in their decision making during usability evaluations, and thus, explaining such inconsistencies between evaluators. The research involved conducting two large analytical evaluations using multiple evaluators and subjecting the results to falsification testing to validate evaluation predictions. Extended Structured Problem Report Formats were specifically designed for usability problem reporting to enable the confident coding of all possible prediction types. The problem reports required explanation for decisions made in problem discovery and analysis thus aiding the identification of factors influencing evaluator judgement. The results show evaluation methods provide little support in usability problem discovery and analysis; hence evaluators rely on a variety of individual knowledge resources such as technical, design, domain and user knowledge for example. Such knowledge resources were not common to all evaluators. The absence, or inappropriate perception of different knowledge resources resulted in missed or inappropriately analysed candidate usability problems. The results provide a model of evaluator behaviour that explains why some evaluators discover usability problems that others fail to discover, and how evaluators make appropriate and inappropriate decisions about candidate usability problems that result in true and false positive, and true and false negative predictions.
| Identifer | oai:union.ndltd.org:bl.uk/oai:ethos.bl.uk:568775 |
| Date | January 2012 |
| Creators | Woolrych, Alan |
| Publisher | University of Sunderland |
| Source Sets | Ethos UK |
| Detected Language | English |
| Type | Electronic Thesis or Dissertation |
Page generated in 0.0023 seconds