The objectives of this paper are to raise awareness of the methodological standards that can affect the quality of radiographer plain-film reading performance studies and to determine the frequency with which these standards are fulfilled. Multiple search methods identified 30 such studies from between 1971 and the end of June 1999. The percentage of studies that fulfilled criteria for the 10 methodological standards were as follows. (1) Performance of a sample size calculation, 3%; (2) definition of a normal and abnormal report, 97%; (3) description of the sequence of events through which films passed before reporting, 94%; (4) analysis of individual groups of observers within a combination of groups, 50% (5) appropriate choice of reference standard, 80%; (6) appropriate choice of arbiter, 57%; (7) appropriate use of a control, 22%; (8) analysis of pertinent clinical subgroups, e.g. body areas, patient type, 44%; (9) availability of data for re-calculation, 59%; and (10) presentation of indeterminate results, 69%. These findings indicate variation in the application of the methodological standards to studies of radiographer's film reading performance. Careful consideration of these standards is an essential component of study quality and hence the validity of the evidence base used to underpin radiographic reporting policy.

Purpose To raise awareness of the frequency of bias that can affect the quality of radiographer plain film reading performance studies. Methods Studies that assessed radiographer(s) plain film reading performance were located by searching electronic databases and grey literature, hand-searching journals, personal communication and scanning reference lists. Thirty studies were judged eligible from all data sources. Results A one-way analysis of variance (ANOVA) demonstrates no statistically significant difference (P=0.25) in the mean proportion of biases present from diagnostic accuracy (0.37), performance (0.42) and outcome (0.44) study designs. Pearson¿s correlation coefficient showed no statistically significant linear association between the proportion of biases present for the three different study designs and the year that the study was performed. The frequency of biases in film and observer selection and application of the reference standard was quite low. In contrast, many biases were present concerning independence of film reporting and comparison of reports for concordance. Conclusions The findings indicate variation in the presence of bias in radiographer plain film reading performance studies. The careful consideration of bias is an essential component of study quality and hence the validity of the evidence-base used to underpin radiographic reporting policy.