Evaluation is a burgeoning field and remains fairly young by most standards. Within Cooperative Extension, evaluation practices have been implemented at a variety of levels given that evaluation is mandatory for much of the funding Cooperative Extension receives. With evaluation in high demand, it is expected that most Extension educators are performing some levels of evaluation as a routine part of their jobs. In order to perform the required evaluations, an Extension educator must exhibit some level of knowledge and skill regarding evaluation. While much research to date has been done on the level of evaluation within the organization, there is a lack of understanding regarding the evaluation competencies that Extension educators must possess and the culture of evaluation within the organization. This study set out to examine the evaluation competencies, culture, and evaluation behaviors within Cooperative Extension. Utilizing an online survey format and quantitative methodology, a widely accepted set of evaluation competencies were examined for their importance within Cooperative Extension. A panel of 13 experts was selected to examine the competencies and it was determined than all competencies in the list were necessary for Extension educators to exhibit in their jobs. The list of competencies was then combined with a subscale regarding culture and a subscale based on the Theory of Planned Behavior (Ajzen, 1991). A total of 419 Extension educators in four Extension systems participated in the study, with 222 generating usable data for a response rate of 13%. The highest and lowest skill level for the competencies were determined by Extension educators self-reporting. Perception of importance of each competency was examined and the highest and lowest importance rankings were determined. These were compared to the rankings of importance by the expert panel. A path analysis was conducted by modifying the Theory of Planned Behavior model and multiple regression analysis. Mean weighted discrepancy scores were calculated to determine the differences in skill level and perception for each of the competencies. The subscale of culture was examined for potential areas of Evaluation Capacity Building (ECB) within the organization. Results show that while there was much agreement between the expert panel and Extension educators regarding the importance of competencies, experts ranked all competencies as important while Extension educators did not. The results of the path analysis determined intention and perceived behavioral control explained 3.9% of the variance in the evaluation behavior exhibited by skill. Subjective norm and attitude explained 11.8% of the variance within intention. Perceived behavioral control, attitude and culture accounted for 13.1% of the variance in subjective norm. Culture and perception accounted for 7.1% of the variance in attitude. Perception, program area, college major, location, training in evaluation, degree level and years of experience explained 28% of the variance within evaluation culture. Finally, recommendations for practice and future research were made based on these findings. / Ph. D.

Family support programs serve vulnerable families by providing various forms of support, such as education, health services, financial assistance, and referrals to community resources. A major feature of evaluation involves assessing program effectiveness and learning from evaluation findings (Mertens and Wilson, 2012). Collaboration and cultural responsiveness are important topics in evaluation which remain largely distinct in the literature. However, evaluation capacity building provides a context for exploring possible intersections. Data about seven programs were collected via semi-structured interviews and document analysis. This study revealed that the program leaders feel that their programs are unique, complex, and misunderstood. The findings also suggest that program leaders believe that evaluation is important for program improvement and funding. Although participants did not anticipate evaluation capacity building and did not readily express a desire to develop their own evaluation skills, participants from all seven programs enthusiastically expressed interest in evaluation capacity building once explained. Although participants did not discuss cultural responsiveness as it relates to race, they expressed a need to overcome a community culture of reluctance to participate in programs and aversion to educational pursuits. Given the programs' shared population of interest, similar outcomes, and common challenges, evaluation capacity building in a group setting may give Roanoke family support program leaders the evaluation knowledge, skills, and peer support to engage in program evaluation that is both collaborative and culturally responsive. / Master of Science in Life Sciences

Program evaluation has become an increasingly urgent task for organizations, agencies, and initiatives that have the obligation or motivation to measure program outcomes, demonstrate impact, improve programming, tell their program story, and justify new or continued funding. Evaluation capacity building (ECB) is an important endeavor not only to empower program staff to understand, describe, and improve their programs, but also to enable programs to effectively manage limited resources. Accountability is important as public funds for social programs continue to dwindle and program administrators must do their best to fulfill their program missions in ethical, sustainable ways despite insufficient resources. While ECB on its own valuable, as it can promote evaluative thinking and help build staff's evaluation literacy and competency, ECB presents a ripe opportunity for program staff to understand the principles of equity and inclusivity and to see themselves as change agents for societal transformation. In the present study, I developed, tested, and evaluated the concept of transformative ECB (TECB), a social justice-oriented approach, rooted in culturally responsive evaluation, critical adult education, and the transformative paradigm, which promotes not only critical and evaluative thinking, but also kaleidoscopic thinking. Kaleidoscopic thinking (KT) is thinking that centers social justice and human dignity through intentional consideration (turning of the kaleidoscope) of multiple perspectives and contexts while attending to the intersectional planes of diversity, such as culture, race, gender identity, age, belief system, and socioeconomic status. KT involves reflexivity, creativity, respect for diversity, compassion and hope on the part of the thinker when examining issues and making decisions. / Doctor of Philosophy / Program evaluation has become increasingly important for organizations seeking to measure program outcomes, demonstrate impact, improve programming, tell their program story, and make the case for new or continued funding. Evaluation capacity building (ECB) includes training that is important not only to help program staff to understand, describe, and improve their programs, but also to allow programs to successfully "do more" with less. While ECB on its own is valuable, as it can help program staff become more evaluation-minded and skilled, ECB presents a ripe opportunity for program staff to understand the principles of equity and inclusivity and to see themselves as drivers of social change. In this study, I developed, tested, and evaluated the idea of transformative ECB (TECB), a social justice-oriented approach, rooted in culturally responsive evaluation, critical adult education, and the transformative (social justice-related) framework. The TECB approach promotes not only critical thinking and evaluative thinking, but also kaleidoscopic thinking, which focuses on social justice and human dignity. KT involves reflexivity, creativity, respect for diversity, compassion, and hope on the part of the thinker when examining issues and making decisions.

No description available.