Evaluation is a process of investigating a project's implementation and results methodically and objectively. The evaluation problems in industry are extremely complicated and call for the examination of numerous qualitative and quantitative variables. Evaluators from different disciplines examine these variables. In order for the evaluators to collaborate during a broad evaluation process, there is a requirement for a collaborative evaluation methodology. In this research, a methodology for evaluation of any general application was implemented which is initially developed as part of a research effort supported by the Office of Naval Research. This technique is based on identifying the indicators to be applied in the evaluation of complex projects. In order to decrease complexity, these indicators are organized hierarchically which is modeled after the Analytic Hierarchy Process method. While at the top layer of this hierarchy is the subject we want to evaluate, towards the lower layers it is divided into more and more simple indicators. Therefore, the evaluators are responsible for assessing the project in terms of only the most simplified indicators. As a result, a precise evaluation which is not subject to human bias is accomplished. We have created a web-based application called Evaluator which allows the users to execute this methodology step by step. It simplifies the process by providing a user-friendly and easily accessible cloud-based platform in accordance with today's user experience principles. Although we have used this method and the software to evaluate projects, it is a tool that provides general usage for the assessment of a broad range of systems from various contexts. / Master of Science / Evaluation of a project is a process of systematic and objective assessment using a certain methodology. The evaluation problems we encounter in industry are quite complicated. This is due to the fact that a project must be evaluated in light of the numerous indicators it contains. Therefore, the project being assessed must be divided into these indicators as the first stage in the evaluation process. This layered form of the problem creates a hierarchical structure. Because the people who take part in evaluation are only capable of handling problems up to a certain level of complexity due to human nature. Therefore, it is essential to simplify problems to the point that those evaluating them can come to informed determinations. The strategy employed in this study makes it feasible to assess the primary subject of the evaluation effectively by limiting the evaluators to assessing only the most simplified indicators at the bottom of this hierarchy. This is the only task that the people in the evaluation process are responsible for. As a result, an accurate assessment not subject to human bias of error is achieved. We have created a web-based application called Evaluator which allows users to carry out this method's procedures. The user-friendly user interface of Evaluator makes it straightforward for people to take part in this complex process. Though we have used this approach and the software to assess projects, it is a flexible software which can be applied broadly for the assessment of a wide variety of systems from different contexts.

This instrumental case study (Stake, 1995) provides an in-depth examination of a collaborative approach to evaluation within a school district’s central office. This study analyzes the implementation of collaborative approaches to evaluation (CAE) within a school district’s central office with an emphasis on evaluation use from the evaluators’ perspectives. It also explores the role and evolution of the evaluators throughout the evaluation process. Furthermore, this study adds to educational research on how to support the implementation and use of improvement processes in school district central offices. The analysis focuses on the evaluation activities that my colleagues and I led as internal evaluators for a school district’s school improvement initiative from September 2015 to October 2016. The conceptual framework that I used to guide my case study are the Principles to Guide CAE (Shulha, Whitmore, Cousins, Gilbert, & al Hudib, 2016), which were recently developed in response to the need for clarification on collaborative approaches to evaluation. Findings suggest that: (1) the organizational context and the evaluators’ skills in using CAE impacted evaluation use, including enhancing the capacity for evaluative thinking for involved stakeholders; (2) the quality of the evaluation process, especially the formation of the findings and recommendations, enhanced the credibility of the evaluation and stakeholders’ willingness to implement the recommendations; and (3) evaluator involvement over the long-term is needed to support all aspects of the continuous improvement cycle. Findings regarding the evaluator’s role within CAE include the need for evaluators: (4) to fully understanding their role and positioning themselves as critical friends and facilitators; and (5) to create a safe space for learning and inquiry to occur. The findings from this study have implications for evaluation researchers interested in evaluation use and the implementation of CAE, as well as for educational researchers focused on incorporating evidence-based practices and improvement processes in school district central offices.