The need for cost-effective usability evaluation has led to the development of methodologies to support the usability practitioner in finding usability problems during formative evaluation. Even though various methods exist for performing usability evaluation, practitioners seldom have the information needed to decide which method is appropriate for their specific purpose. In addition, most methods do not have an integrated relationship with a theoretical foundation for applying the method in a reliable and efficient manner. Practitioners often have to apply their own judgment and techniques, leading to inconsistencies in how the method is applied in the field. Usability practitioners need validated information to determine if a given usability evaluation method is effective and why it should be used instead of some other method. Such a desire motivates the need for formal, empirical comparison studies to evaluate and compare usability evaluation methods. In reality, the current data for comparing usability evaluation methods suffers from a lack of consistent measures, standards, and criteria for identifying effective methods. The work described here addresses three important research activities. First, the User Action Framework was developed to help organize usability concepts and issues into a knowledge base that supports usability methods and tools. From the User Action Framework, a mapping was made to the Usability Problem Inspector; a tool to help practitioners conduct a highly focused inspection of an interface design. Second, the reliability of the User Action Framework was evaluated to determine if usability practitioners could use the framework in a consistent manner when classifying a set of usability problems. Third, a comprehensive comparison study was conducted to determine if the Usability Problem Inspector, based on the User Action Framework, could produce results just as effective as two other inspection methods (i.e., the heuristic evaluation and the cognitive walkthrough). The comparison study used a new comparison approach with standards, measures, and criteria to prove the effectiveness of methods. Results from the User Action Framework reliability study showed higher agreement scores at all classification levels than was found in previous work with a similar classification tool. In addition, agreement using the User Action Framework was stronger than the results obtained from the same experts using the heuristic evaluation. From the inspection method comparison study, results showed the Usability Problem Inspector to be more effective than the heuristic evaluation and consistent with effectiveness scores from the cognitive walkthrough. / Ph. D.

Made available in DSpace on 2014-12-17T13:53:36Z (GMT). No. of bitstreams: 1 MarioJFMF_DISSERT.pdf: 1237604 bytes, checksum: 133d301f889ce67c91bbd146c035e0d1 (MD5) Previous issue date: 2013-05-31 / On the aggravation of social problems and the shortage of resources, the improvement of evaluation methods and control of its application, requiring more efficiency, efficacy, effectiveness and participation in its management, has been growing. As a result, emerges the importance of studying and developing such methodologies. The overall goal of this dissertation is to know what are the difficults to incorporate the point of view of executers and beneficiaries in evaluation process. To do so, has been done a research characterized as qualitative, with a field strategy using the case study of two social projects called Petrobras Child Program, situated in the metropolitan region of Natal, and Content Analysis technique for analyze the data. The conclusions of this work can assist in improving the process of projects evaluation financed by Petrobras, contributing with its social role, besides the possibility of encouraging a greater participation of other society actors, such as beneficiaries, in the evaluation process / Diante do agravamento dos problemas sociais e da escassez de recursos, o aprimoramento dos m?todos de avalia??o e controle da sua aplica??o, exigindo-se mais efici?ncia, efic?cia, efetividade e participa??o na sua gest?o, tem sido crescente. Da? resulta a import?ncia de se estudar e desenvolver tais m?todos. O objetivo geral da presente Disserta??o ? compreender como incorporar a vis?o de executores e benefici?rios no processo avaliativo. Para tanto, realizou-se uma pesquisa caracterizada como qualitativa, utilizando como estrat?gia de campo o estudo de caso de dois projetos sociais denominados Programa de Crian?a Petrobras, situados na regi?o metropolitana de Natal, e a an?lise de conte?do como t?cnica anal?tica dos dados. As conclus?es deste trabalho poder?o auxiliar no aperfei?oamento do processo de avalia??o dos projetos financiados pela Petrobras, contribuindo assim com o cumprimento do seu papel social, al?m da possibilidade de estimular uma maior participa??o de outros atores da sociedade, tais como os benefici?rios desses projetos, no processo de avalia??o