Thesis (M.P.H.)--University of Michigan, 1946.

Evaluation use is a complex and multifaceted construct comprising domains of findings and process use and sub-categories of instrumental use, conceptual use, and persuasive (legitimative/symbolic) use. It is an important aspect of evaluation practice which has received much attention from evaluation scholars and practitioners. However, evaluation and evaluation use in countries outside North America, Europe and Australasia is under-investigated. This study proposes to make a modest contribution to understanding evaluation practices in an Asian context through investigating the value and utility of program evaluation as perceived by officials in selected Singapore social sector ministries. Singapore is a democratic country but has retained the characteristics of a highly centralized state. In addition, Confucian values are very important for the political leaders and Singapore society. A modified explanatory sequential mixed methods design was used to address the research questions. Both quantitative and qualitative data were collected through a questionnaire with both closed-ended and open-ended questions. The questionnaire was followed up with interviews to provide additional depth and explanations related to the results from the questionnaire. The questionnaire, based on current theories and the applicability of these evaluation theories to Singapore, was sent through email to officers in two social sector ministries in Singapore - Ministry of Culture, Community and Youth (MCCY) and Ministry of Education (MOE), and 120 valid responses were obtained. Results were analyzed by examining the internal structure and bifactor model underlying the questionnaire. The hypothesized bifactor model comprised a general factor, Perceptions of Evaluation, and two sub-factors – Value of Program Evaluation and Purposes of Program Evaluation. The score on Perceptions of Program Evaluation was regressed on respondents’ background variables to investigate factors that affected their perceptions. Qualitative findings from the open-ended questions and the interviews were analyzed separately and integrated with the quantitative results. The questionnaire functioned well for the respondents, indicating the applicability of existing scholarship on program evaluation to the Singapore context. However, there were a few items that were not respond to as hypothesized. Unique aspects of the context in Singapore social sector ministries might have been the reason for the responses to these items. In addition, the results suggested that a modified bifactor model (with purposes split into internal and external to the program) would fit the data better. On the whole, respondents perceived program evaluation positively (Mean = 1.58, SD = 0.52 on a scale from minus three to three with zero as the mid-point) and emphasized the value of program evaluation for learning and improvement. However, respondents seemed to have reservations with going through the evaluation process. Factors influencing the respondents’ perceptions were layered, and included such factors as officers’ characteristics and experiences, bosses, evaluators, corporate values, organizational structure, societal values and political climate. Officers’ experiences with program evaluation emerged as an important factor, particularly experiences related to benefitting from program evaluation. Given the Confucianism based, highly centralized Singapore state, there might be gaps in current literature to inform and understand the evaluation practices within this and similar contexts. Further studies would be needed to investigate the extent of influence and the mechanisms through which the factors affect perceptions of program evaluation in the Singapore context.

To provide guidance to evaluators and stakeholders, evaluation scholars (i.e., those conducting research on program evaluation) have conducted numerous studies on the feasibility and effectiveness of using participatory and collaborative evaluation approaches in various contexts. While some participatory and collaborative evaluation approaches may involve youth in the evaluation of programs and interventions, few evaluations in this area have been formally documented and/or widely published. As a result, there remains a dearth of empirical research on participatory and collaborative evaluations involving youth. One such collaborative evaluation approach, empowerment evaluation (EE), appears to be well suited for engaging youth in program evaluation, as participants are co-evaluators. Using qualitative, quantitative, and mixed methods, EE aims to teach program stakeholders, including beneficiaries, how to conduct their own evaluations. In this two-part mixed methods research project I sought to investigate and formally document: (a) the use of EE for programs targeting youth; and/or (b) the involvement of youth in EE of such programs. By investigating and documenting these areas, this study builds on the very limited body of empirical research on EE. As such, it provides important information to evaluators who are embarking on evaluations of programs targeting youth, so that they can make informed decisions about the use of EE and the involvement of youth in their evaluation activities. To address these goals, this study used a mixed methods case study approach and included two parts and multiple phases. Part 1 Phase 1 involved a survey of evaluators associated with particular Targeted Interest Groups (TIGs) of the American Evaluation Association (AEA) who are involved in evaluating programs that target youth. It determined the extent to which: (a) evaluators report using EE to evaluate youth programs; and (b) how evaluators report involving youth in EE of youth programs. Part 1 Phase 2 involved interviews with a select group of these evaluators and explored what factor(s) facilitate and hinder: (a) the use of EE to evaluate programs involving youth; and (b) the involvement of youth in EE of programs targeting youth. Part 2 then used observations from an EE with youth of their science, technology, engineering and math (STEM) focused educational outreach program to explore: (a) what an EE of a youth program might look like in practice; (b) how youth can be involved in an EE. Youth also took part in follow-up interviews to allow an examination of: (c) the strengths and limitations of using an EE to evaluate a program targeting youth; and (d) the strengths and limitations of involving youth in an EE of a program targeting youth. Overall, the findings show that the use of EE to evaluate programs involving youth may be limited, however, there are factors that can facilitate and hinder the use of EE and the involvement of youth in EE. The findings also demonstrate that an EE can be carried out in practice with youth acting as co-evaluators and that through EE youth may experience both positive and negative outcomes of using EE and of being involved in EE. In light of these findings, ways to improve the involvement of youth in the evaluation of programs that target youth using EE are discussed.

No description available.

In regions of the world that experience a deficit of surgical care, educational initiatives can foster the development of a skilled surgical workforce. Implicit in training these health workers is the mandate to retain them in the country in order build capacity. Eyal’s framework presents ways in which locally relevant training can improve retention and outlines the ethical and pragmatic concerns of such initiatives. In 2006, Guyana established it’s first surgical training program, an example of locally relevant training. The University of Guyana Diploma of Surgery (UGDS) program was selected for this case study research. Consistent data collection, supported by a systemic procedure to analyze that data, is paramount to increase the effectiveness of the UGDS program. The purposes of this dissertation research were two-fold. Firstly, it sought to understand how the UGDS program influences retention and the ways in which the UGDS members contribute to capacity building and the program’s sustainability. Secondly, this program evaluation provides a useful context to inform Eyal’s framework. 8 graduates, 2 trainees, 4 faculty members and 2 persons identified as policy makers were interviewed. Interviews were conducted face to face, and then transcribed. Surveys were administered to graduates and trainees and reflective reports and presentations were coded and analyzed. Overall, the data mapped fairly well onto Eyal’s framework. The results of the study suggest that the benefits and concerns Eyal outlines would be better represented along a continuum rather than being classified as either advantageous or disadvantageous with respect to retention. While Eyal’s claims generalize across settings, he should acknowledge this limitation and consider the important role that context plays. Overall, the results suggest that the UGDS program has positively influenced retention and capacity building. Key recommendations were made to the UGDS program that aim to improve retention and capacity building. As regions continue to face challenges associated with providing adequate surgical care, fostering retention and capacity building is recommended so that a sustainable surgical workforce can meet surgical needs. / Thesis / Candidate in Philosophy

Microcredit programs are of great interest to economists and policymakers because of their potential for reducing poverty, particularly among women. The first chapter mainly investigates the effectiveness aspect of microcredit programs. Using program evaluation methods, we find significant improvement in women's economic condition after participating in these programs. This study also corrects for the self-selection bias that might arise due to the fact that women decide on whether to participate in the programs or not. The second chapter studies the determinants of women's economic performance in microcredit programs. These determinants are in the form of different types of characteristics of women: their own characteristics, such as age or schooling or the characteristics of the household or village they live in. One obstacle to measure the effect of observed characteristics is the problem of omitted variable bias, typically caused by unavailability of data on unobserved ability of individuals. In the absence of suitable instruments, this study finds information about unobserved ability from the marriage market. It is found that incorporating estimates of women's unobserved characteristics significantly changes the estimated effect of women's observed characteristics and substantially removes the omitted variable bias. Microcredit programs originated from Bangladesh and now three major microcredit programs are operating: Grameen Bank, BRAC and RD-12. The third chapter investigates how these different microcredit programs have been performing relative to each other. Using similar program evaluation technique as in chapter 1, we measure program impact on women's economic welfare for these programs separately. We find that BRAC outperforms Grameen Bank and RD-12 significantly. This result is interesting since it contradicts the popular notion that Grameen Bank is the most successful microcredit program. This study also tries to find the determinants of economic success of women participating in these programs, separately for each program. These results provide more insights into different aspects of microcredit program. / Ph. D.

Many public administration scholars and administrators look to the concept of organizational learning as a means of coping with shrinking resources and increasing responsibilities and accountability. At the theoretical level, the notion of organizational learning is appealing, since it emphasizes achieving organizational goals through optimization of internal resources, i.e., assimilation of staff members' fullest potential and organizational visions through the synergistic effects of knowledge creation and continuous learning. There has been a fascination without foundation about organizational learning among scholars of public sector organizations because the validity and utility of organizational learning theory has not been tested in public sector organizations. From the perspective of organizational learning, this dissertation evaluates an intervention program–the penetration rate enhancement project (PREP)–to determine whether and to what extent organization learning has taken place in selected localities. The penetration rate is a foster care funding ratio of federal to state and local dollars. The Commonwealth of Virginia's Department of Social Services through collaborative relationships sought to impart policy knowledge (cooptation) in the local departments of social services (LDSS). This study measured the presence (or absence) of a learning environment that was hypothesized to influence the degree of organizational learning and tested whether it correlates with learning variations as approximated by the penetration rate in LDSSs over a period of more than four years. The main research focus is whether an effective learning environment was related to the extent of learning and thereby may help explain learning variation in public sector organizations. This study uses mixed methods to examine the research questions. The study found evidence of some differential, dispersed, and intermittent learning in various localities. The localities are classified as exhibiting conscious learning, facade learning, unaware learning, and absent learning. Those localities that learned and exhibited evidence of a learning environment are referred to as conscious learners; those localities that saw the presence of learning environment but had not been able to learn are referred to as having experienced facade learning; those localities that have learned but did not see a presence of a leaning environment are referred as to being marked by unaware learning; and those localities that did not learn and did not have a presence of a learning environment are referred to as exhibiting absent learning. The anecdotes of special difficulties experienced by public sector organizations to learn have been reaffirmed by the differential perceptions about learning environments held by the senior and junior level staffers in LDSSs. It is also apparent that problematic organizational structures, an economic (dis)incentive system, and the omission of financial component all contributed to the constraints on organizational learning in LDSSs. Despite the constraints, however, the PREP was largely successful in cultivating organizational learning at the LDSSs, and the organizational learning lens for evaluating intervention programs in public sector organizations at the local level was valid. / Ph. D.

Men and masculine qualities stereotypically define leadership. Role Congruity Theory explains that women leaders may experience a double discrimination. First, because leadership roles are stereotypically male, a woman may not have the necessary qualities. Second, because leadership qualities are stereotypically masculine, when women do exhibit agentic qualities, they are perceived negatively and not feminine (Eagly and Karau, 2002). Further, adopting a masculine style is perceived as an inauthentic leadership style. However, an androgynous style can blend feminine expectations with corporate needs, creating advantage and perceptions of authenticity (Tzinerr and Barsheshet-Picke, 2014). The leadership development training described in this thesis was held for female volunteer counselors of Girls Rock! Roanoke, an empowerment camp for young girls. The appreciative pedagogy included individual reflections, group discussions, and working through simulations. The evaluation of the training sought to identify: if discussion of leadership increased awareness of feminine and masculine leadership; if discussion of personal leadership experiences increased empowerment to self-identify as leaders; and what impact the experience had on participants' ability to improve and employ leadership. After participants completed the training, awareness of feminine and androgynous leadership increased; whereas, it is unclear if awareness of masculine leadership increased. Challenging participants to reflect on their leadership increased their self-identification as leaders, while those who already identified did so in androgynous terms. The training impacted participants' knowledge of themselves as leaders, how they planned to use leadership in camp and in real life, but it is unclear to what degree it impacted their ability to improve and employ leadership beyond the training. / Master of Science in Life Sciences

Program evaluation is becoming increasingly important in pediatric rehabilitation settings that adhere to FCS philosophy. This philosophy recognizes that each family is unique, that parents know their children best, and that optimal child functioning occurs within a family context. However, researchers know little about the specific evaluation activities occurring in these settings or the extent to which evaluators uphold FCS philosophy in their activities. The primary goal of this study is to examine the strengths, limitations, and consequences of current evaluation practice, including its compatibility with FCS philosophy. As a secondary goal, the study aims to understand the promise and prospects of alternative evaluation approaches that, in theory, are compatible with FCS philosophy. To address these goals, this study uses a mixed-methods approach and includes three phases. Phase 1 involves a survey of staff members involved with program evaluation at 15 Canadian pediatric rehabilitation centres. It determines the level of program evaluation occurring in these settings, verifies on the motivation for evaluation, and describes the degree to which evaluation activities are consistent with the FCS philosophy. Phase 2 involves interviews with staff members and explores the values, factors, and conditions that support and inhibit the evaluation of family-centred programs in pediatric rehabilitation settings as well as the benefits and limitations of using mainstream practices for evaluating these programs. Phase 3 then uses focus groups with staff members and interviews with parents to explore how the evaluation of family-centred programs can be improved as well as to identify the compatibility and practicality of using alternative evaluation approaches within these settings. Overall, the findings show that the amount of evaluation activities occurring within these centres is variable; that the majority of individuals working in program evaluation do not have formal training in it; and that the centres have limited resources for evaluation. The findings also demonstrate that participatory evaluation approaches are more compatible with FCS philosophy but that it might be difficult to implement such approaches given the limited resources and diverse characteristics of rehabilitation settings. In light of these circumstances, the study notes ways for improving program evaluation activities.

Program evaluation is becoming increasingly important in pediatric rehabilitation settings that adhere to FCS philosophy. This philosophy recognizes that each family is unique, that parents know their children best, and that optimal child functioning occurs within a family context. However, researchers know little about the specific evaluation activities occurring in these settings or the extent to which evaluators uphold FCS philosophy in their activities. The primary goal of this study is to examine the strengths, limitations, and consequences of current evaluation practice, including its compatibility with FCS philosophy. As a secondary goal, the study aims to understand the promise and prospects of alternative evaluation approaches that, in theory, are compatible with FCS philosophy. To address these goals, this study uses a mixed-methods approach and includes three phases. Phase 1 involves a survey of staff members involved with program evaluation at 15 Canadian pediatric rehabilitation centres. It determines the level of program evaluation occurring in these settings, verifies on the motivation for evaluation, and describes the degree to which evaluation activities are consistent with the FCS philosophy. Phase 2 involves interviews with staff members and explores the values, factors, and conditions that support and inhibit the evaluation of family-centred programs in pediatric rehabilitation settings as well as the benefits and limitations of using mainstream practices for evaluating these programs. Phase 3 then uses focus groups with staff members and interviews with parents to explore how the evaluation of family-centred programs can be improved as well as to identify the compatibility and practicality of using alternative evaluation approaches within these settings. Overall, the findings show that the amount of evaluation activities occurring within these centres is variable; that the majority of individuals working in program evaluation do not have formal training in it; and that the centres have limited resources for evaluation. The findings also demonstrate that participatory evaluation approaches are more compatible with FCS philosophy but that it might be difficult to implement such approaches given the limited resources and diverse characteristics of rehabilitation settings. In light of these circumstances, the study notes ways for improving program evaluation activities.