Back to search

University of British Columbia

A critical examination of the functions and methods of evaluation research

Abstract

dc:description

This thesis examines the aims and methods of evaluation research, and, ultimately, considers what functions evaluation may serve in innovative programs. It addresses the controversy with respect to the inadequacy of the scientific experimental method in evaluating social action. Analysis of a case study indicates that applying the experimental method is problematic when programs have vague concepts and uncertain targets, which is almost certain in innovatory programs. Even if that method were successful in generating reliable facts about program effects, it would have overlooked some important subjective aspects of the program. Our general conclusion suggests that it is important to go beyond scientific experimental approaches by exploring alternative modes of evaluation. Critical theory, it is suggested, is useful for studying the normative and latent aspects of programs and program effects. A main presupposition of this study is that an epistemological investigation, which examines the theories of knowledge underlying the aims and methods of evaluation, can make a worthwhile contribution to program evaluations. Inherent in evaluation are fundamental epistemological questions such as "What may be considered knowledge?", "What is the factual and value content of that knowledge?", and "For what purposes do we use enquiries?". Numerous figures in evaluation suggest that not enough attention is given to the conceptual analysis of evaluation purposes and methods. The literature on evaluation is the source of our analysis of the functions and methods proposed by various practitioners. We examine the views of two prominent camps: those who support the scientific experimental methods of testing, measuring and generating facts; and those who use normative social theories and attend to value issues and subjective aspects of programs. Relying upon the literature related to social enquiry, the thesis examines diverse theories of knowledge: scientism, positivism, pragmatism, and finally, critical theory. The critical school advocates a flexible approach to enquiry which extends beyond scientific experimental methods. Following Kant, it emphasizes the pervasive influence of a priori cognitive models in our generation of knowledge. It recognizes, too, the importance of the Hegelian view that the history of abstract thoughts and values is essential for understanding social affairs. As well, its critique incorporates the Marxian stance of focusing on the conflicts of interest among social classes. Accordingly, it goes beyond the positivistic notion that bare facts devoid of value content can be generated. The critical approach, by considering multiple perspectives and by investigating latent aspects such as value systems, conflicting interests, perceived social needs and human consciousness, is appropriate for studying the complex nature of social action. Following upon these concerns, the following evaluation functions are proposed: to clarify theoretical assumptions of programs, to reveal and revise value assumptions and relate them to a history of ideas, to identify the social impacts of programs and the perceived needs of program participants, and ultimately, to foster the exercise of critical self-reflection among participants by encouraging decentralized decision-making and evaluation. These criteria are applied to a case study, an evaluation of a community energy program. The conclusions drawn from the case study indicate that the controlled experiment should be supplemented with subjective evaluation approaches. The experiment may be useful at times, but at least with respect to social innovative projects, its application is problematic. With the presence of program features including vague program objectives, uncertain criteria of program success, and unvalidated causal assumptions (attitude →behaviour → actual energy savings) , it is impossible to "test" (in the scientific sense) hypotheses such as Program A →Causal Process B → Objective C. Our critical examination suggests that, by concentrating on concrete program effects, the case study evaluation had overlooked important intrinsic factors affecting the program and plan. These factors are the program's theoretical and value assumptions, the perceived needs of recipients, and the conflicts of interest between the sponsors and the program designer. The critical school of thought reminds us that we have to persistently question the factual content, value basis and causal assumptions of our programs. Critical theory, by identifying the limits of traditional enquiries and exploring alternative epistemologies, provides invaluable insights for developing guidelines for evaluation research.

Degree

thesis:*
Name thesis:degree_name
Master of Arts in Planning - MA (Plan)
Level thesis:degree_level
master's
Discipline thesis:degree_discipline
Planning
Grantor dc:publisher
University of British Columbia
Year dc:date
1984

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Auyeung, Poyin

Rights

dc:rights
Statement dc:rights
  • For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/24972

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Auyeung, Poyin. A critical examination of the functions and methods of evaluation research. master's thesis, University of British Columbia, 1984. http://hdl.handle.net/2429/24972