Traditional methods of evaluating quality in computer end user documentation have been subjective in nature, and have not been widely used in practice. Attempts to quantify quality and more narrowly define the essential features of quality have been limited -- leaving the issue of quality largely up to the writer of the user manual. Quantifiable measures from the literature, especially Velotta (1992) and Brockman (1990), have been assembled into a set of uniformly weighted metrics for the measurement of document quality. This measure has been applied to the end user documentation of eighty-two personal computer packages. End user documentation is defined in terms of paper documents only. The research examined only those manuals that were titled “user guide,” “training manual,” “tutorial,” or similar title. The research examined six categories of software: applications, graphics, utilities, spreadsheets, databases, and word processing. Following the recommendation of Duffy (1985), a panel of experts was assembled and asked to evaluate several of the 82 end user manuals in order to determine what correlation exists between the set of metrics and the subjective opinion of experts. The eighty-two documents in the sample were scored by the metrics using a convenient random sampling technique. This technique was selected based the consistency of the material in commercial software manuals and the methods of Velotta (1992). Data from the metrics suggest that there is little correlation between quality, category, price, page length, version number, and experience. On a scale of 0.0 to 1.0, the minimum total score from the metrics was .2; the maximum score .83; the mean total score was. 70; the median .697 with a standard deviation of .093. The distribution is slightly skewed and leptokurtic (steeper than a normal curve). The metrics further suggest a declining score as the integration of sentences into chapters and chapters into the document progresses. Of the metrics two consistently had lower scores: those relating to the transition between sections of the document; and the reference tools provided. Though not conclusive, the analysis of data from the panel of experts compared with the model results suggests only a moderate correlation. However, by varying the weighting scheme, it is possible to improve model performance - essentially by "tuning" the model to match the sample data from the panelists. Further research would be required to verify if these weights have more global application.
| Identifer | oai:union.ndltd.org:pdx.edu/oai:pdxscholar.library.pdx.edu:open_access_etds-2325 |
| Date | 01 January 1993 |
| Creators | Morrison, Ronald |
| Publisher | PDXScholar |
| Source Sets | Portland State University |
| Detected Language | English |
| Type | text |
| Format | application/pdf |
| Source | Dissertations and Theses |
Page generated in 0.002 seconds