To incorporate users' opinions into the evaluation phase early in the software documentation development stage, the critical incident technique was used to identify usability problems in a fault free software user document. The critical incidents were used to modify the original document to improve its usability. To test whether the modified document was better in usability than the original document, an experiment was conducted to compare objective measures and subjective ratings. Four objective measures included number of errors, task completion time, document reading time, and number of personal helps requested. In addition, subjective ratings on ease of use, accuracy of information, inconsistencies, ease of learning, completeness, helpfulness of figures, and ease of understanding were compared between the two documents. The analyses showed that for 3 of 4 objective measures and 7 out of 9 subjective ratings, the new document was better and easier to use. In some cases, this difference was task specific. Generally, easier tasks accounted for better objective measures and more favorable subjective ratings. / Master of Science

Since humans first put pen to paper, people have critically assessed written work; thus, the assessment of documents per se is not new. Only recently, however, has the issue of formalized document quality assessment become feasible. Enabled by the rapid progress in computing technology, the prospect of an automated, formalized system of quality assessment, based on the presence of certain attributes deemed essential to the quality of a document, is feasible. The existing Objectives/Principles/Attributes Framework, previously applied to code assessment, is modified to allow application to documentation quality assessment. An automated procedure for the assessment of software documentation quality assessment and the development of a prototype documentation analyzer are described. A major shortcoming of the many quality metrics that are proposed in computer science is their lack of empirical validation. In pursuit of such necessary validation for the measures proposed within this thesis, a study is performed to determine the agreement of the measures rendered by Docalyze with those of human evaluators. This thesis demonstrates the applicability of a quality assessment framework to the documentation component of a software product. Further, the validity of a subset of the proposed metrics is demonstrated. / Master of Science