Evaluation is a process of investigating a project's implementation and results methodically and objectively. The evaluation problems in industry are extremely complicated and call for the examination of numerous qualitative and quantitative variables. Evaluators from different disciplines examine these variables. In order for the evaluators to collaborate during a broad evaluation process, there is a requirement for a collaborative evaluation methodology. In this research, a methodology for evaluation of any general application was implemented which is initially developed as part of a research effort supported by the Office of Naval Research. This technique is based on identifying the indicators to be applied in the evaluation of complex projects. In order to decrease complexity, these indicators are organized hierarchically which is modeled after the Analytic Hierarchy Process method. While at the top layer of this hierarchy is the subject we want to evaluate, towards the lower layers it is divided into more and more simple indicators. Therefore, the evaluators are responsible for assessing the project in terms of only the most simplified indicators. As a result, a precise evaluation which is not subject to human bias is accomplished. We have created a web-based application called Evaluator which allows the users to execute this methodology step by step. It simplifies the process by providing a user-friendly and easily accessible cloud-based platform in accordance with today's user experience principles. Although we have used this method and the software to evaluate projects, it is a tool that provides general usage for the assessment of a broad range of systems from various contexts. / Master of Science / Evaluation of a project is a process of systematic and objective assessment using a certain methodology. The evaluation problems we encounter in industry are quite complicated. This is due to the fact that a project must be evaluated in light of the numerous indicators it contains. Therefore, the project being assessed must be divided into these indicators as the first stage in the evaluation process. This layered form of the problem creates a hierarchical structure. Because the people who take part in evaluation are only capable of handling problems up to a certain level of complexity due to human nature. Therefore, it is essential to simplify problems to the point that those evaluating them can come to informed determinations. The strategy employed in this study makes it feasible to assess the primary subject of the evaluation effectively by limiting the evaluators to assessing only the most simplified indicators at the bottom of this hierarchy. This is the only task that the people in the evaluation process are responsible for. As a result, an accurate assessment not subject to human bias of error is achieved. We have created a web-based application called Evaluator which allows users to carry out this method's procedures. The user-friendly user interface of Evaluator makes it straightforward for people to take part in this complex process. Though we have used this approach and the software to assess projects, it is a flexible software which can be applied broadly for the assessment of a wide variety of systems from different contexts.

Online learning has grown exponentially within higher education in the past decade, especially at community colleges. As online course offerings expand community colleges need to assess student learning in order to ensure quality learning experiences for students and for accreditation purposes. The purpose of this study was to compare the assessment methods of community college faculty who teach the same course in both face-to-face and online formats, in order to determine the similarities and differences that exist in assessment practices. This study stemmed from the constructionist and pragmatic perspectives of the researcher. Additionally this study asked how community college faculty perceive the effectiveness of different assessment methods on student learning. The method for this research was a qualitative multiple case study in which community college accounting faculty were interviewed to explore how they assess student learning in both the online and face-to-face formats. Homework problems, exams, quizzes were the most commonly used methods of assessment of student learning in both formats by the study participants. Furthermore, exams and quizzes were believed to be most effective in assessing student learning, among the community college faculty members interviewed. Another discovery from this study included the prevalent use of publisher's software,called My Accounting Lab, among community college accounting faculty in administering assessments to students. Additional concerns around assessing online students effectively, managing online student group work, and mitigating online student cheating were also revealed during interviews with study participants. / Graduation date: 2013

Thesis (PhD)--Stellenbosch University, 2015. / ENGLISH ABSTRACT: This thesis presents a failure evaluation methodology for nuclear graphite components used in high-temperature gas reactors. The failure methodology is aimed at predicting the failure of real parts based on the mechanical testing results of material specimens. The method is a statistical failure methodology for calculating the probability of failure of graphite components, and has been developed and implemented numerically in conjunction with a finite element analysis. Therefore, it can be used on any geometry and load configuration that can be modelled using finite element analysis. The methodology is demonstrated by mechanical testing of NBG-18 nuclear grade graphite specimens with varying geometries under various loading conditions. Some tests were developed as an extension of the material characterisation, specifically engineered to assess the effect of stress concentrations on the failure of NBG-18 components. Two relevant statistical distribution functions, a normal distribution and a twoparameter Weibull distribution are fitted to the experimental material strength data for NBG-18 nuclear graphite. Furthermore, the experimental data are normalised for ease of comparison and combined into one representative data set. The combined data set passes a goodness-of-fit test which implies the mechanism of failure is similar between data sets. A three-parameter Weibull fit to the tensile strength data is only used in order to predict the failure of independent problems according to the statistical failure methodology. The analysis of the experimental results and a discussion of the accuracy of the failure prediction methodology are presented. The data is analysed at median failure load prediction as well as at lower probabilities of failure. This methodology is based on the existence of a “link volume”, a volume of material in a weakest link methodology defined in terms of two grouping criteria. The process for approximating the optimal size of a link volume required for the weakest link failure calculation in NBG-18 nuclear graphite is demonstrated. The influence of the two grouping criteria on the failure load prediction is evaluated. A detailed evaluation of the failure prediction for each test case is performed for all proposed link volumes. From the investigation, recommended link volumes for NBG-18 are given for an accurate or conservative failure prediction. Furthermore, failure prediction of a full-sized specimen test is designed to simulate the failure condition which would be encountered if the reactor is evaluated independently. Three specimens are tested and evaluated against the predicted failure. Failure of the full-size component is predicted realistically but conservatively. The predicted failure using link volume values for the test rig design is 20% conservative. The methodology is based on the Weibull weakest link method which is inherently volume dependent. Consequently, the conservatism shows that the methodology has volume dependency as experienced in the classic Weibull theory but to a far lesser extent. / AFRIKAANSE OPSOMMING: Hierdie tesis beskryf ‘n metode wat gebruik kan word om falings in kern grafiet komponente te voorspel. Hierdie komponente word in hoë temperatuur gas reaktore gebruik. Die falings metodologie beoog om die falings van regte komponente te voorspel wat gebaseer is op meganiese toets resultate van materiaal monsters. Dit is ‘n statistiese falings metodologie wat die waarskynlikheid van faling vir grafiet komponente bereken. Die metode is numeries ontwikkel en geïmplementeer deur middel van die eindige element metode, dus kan die metodologie toegepas word op enige geometrie en belastingsgeval wat dan gemodelleer kan word deur gebruik te maak van eindige element metodes. Die metodologie word gedemonstreer deur gebruik te maak van NBG-18 kern grafiet toets monsters. Sommige van hierdie toetse is ontwikkel as ‘n uitbreiding van die materiaal karakterisering wat spesifiek ontwerp is om die effek van die spannings konsentrasies op die faling van die NBG-18 komponente te evalueer. Twee relevante statistiese verspreiding funksies word gekoppel aan die eksperimentele sterkte data van die NBG-18 kern grafiet, naamlik ‘n normale verspreiding en ‘n twee-parameter Weibull verspreiding. Die data stelle word ook genormaliseer vir gemak van vergelyking en gekombineer in een verteenwoordigende data stel. Die gekombineerde data stel slaag ‘n korrelasie toets wat impliseer dat die meganisme van faling soortgelyk is tussen die data stelle. ‘n Drie-parameter Weibull korrelasie op die trek toets monsters word gebruik vir die statistiese falings metodologie. Die analise van die eksperimentele resultate sowel as ‘n bespreking van die akkuraatheid van die faling voorspelling metodologie word voorgelê. Die data word geanaliseer by gemiddelde faling voorspelling asook by laer voorspellings van falings. Hierdie metode is gebaseer op die bestaan van ‘n “ketting volume” wat die volume van ‘n materiaal wat gebruik word in die swakste ketting voorstel en koppel aan die metodologie. ‘n Metode vir die benadering van die ketting volume word voorgestel en daaropeenvolgend gebruik om die ketting volume te bereken vir NBG-18. ‘n Gedetailleerde evaluasie van die falings voorspelling vir elke toets geval word uitgevoer vir die voorgestelde ketting volumes. Gebaseer op hierdie ondersoek is voorgestelde ketting volumes vir NBG-18 gegee vir beide akkurate en konserwatiewe falings voorspellings. Verder was ‘n volgrootte strukturele toets ontwikkel om dieselfde falings omstandighede te simuleer wat verwag is gedurende normale werking van die reaktor. Drie monsters word getoets en geëvalueer teen die voorspelde faling vir beide die berekende ketting volume groottes. Faling van die volgrootte komponente word realisties asook konserwatief voorspel. Die voorpselling is 20% konserwatief. Die metodologie is gebaseer op die Weibull metode wat inherent volume afhanklik is; gevolglik dui die konserwatisme aan dat die metodologie oor volume afhanklikheid beskik soos ondervind word in die klassieke Weibull teorie, maar tot ‘n baie kleiner mate.

<p>The findings in this case study are mainly based on material collected during a field trip with a fellow student to Laos in February-March 2005, which was sponsored by Sida through a Minor Field Study scholarship. The original interest was to study a political aid project (Governance and Administrative Reform Project Luang Prabang, GPAR LP) in a country like Laos with very complex preconditions such as one-party polity. In Laos my interest in evaluation of political aid grew mainly because I found out that the two donors in the project, Sida and the UNDP, had reached different conclusions in their respective evaluations of the project.</p><p>The main interest in this thesis has been to study how donors have evaluated a political aid project. The study shows that Sida and UNDP used, what in the literature usually is referred to as, Conventional evaluation as methodology when they conducted their evaluations of GPAR LP. The study also shows that the main reason for the different points of view that are expressed in the evaluations depend on a lack of dialogue and consensus, regarding for the project crucial concepts, rather than choice of evaluation methodology. This lack of mutual understanding between the agencies also has implications for the possibilities of the receiving part to create a better understanding and ownership of the project. If Sida and UNDP had chosen to do a mutual evaluation a lot could have been won in the form of time, money and less irritation between the involved parties.</p><p>The study finally shows that a different choice of evaluation methodology, in this case Participatory evaluation, could have had some positive effects mostly due to its supposedly democratic methods which are in line with the goals of the GPAR LP- project. One example here is the use of studies of the political context which would have been useful in this case. Another example is that the methodology could be useful to create a better understanding and ownership of the project through its participatory methods.</p>

This thesis makes two significant contributions to the advancement of knowledge within crime prevention. The first of these is to evaluate the success of repeat victimisation prevention interventions. Interventions across four crime types are assessed herein, and the context-mechanisms-outcome configurations examined. The second contribution of this thesis is to assess two techniques of meta-evaluation: systematic reviews and realist syntheses. Each of these techniques is used in turn to assess the repeat victimisation prevention interventions. The contribution of each technique to the knowledge pool is then discussed, and the question of whether they are complementary or contradictory approaches answered. The thesis is framed in the context of evolutionary epistemology, which is the philosophy underpinning both approaches to meta-evaluation addressed herein. The thesis starts, with an examination of: firstly, how the evaluation methods in question have evolved, and the background to their scientific worth; and secondly, how situational crime prevention measures have evolved over time. The thesis then examines the two competing approaches for their contribution to the evaluation ecosystem by using both to assess repeat victimisation prevention interventions. Finally, the last section poses the question of whether it is survival of the fittest, or whether co-existence or adaptation could be the key to survival for these two meta-evaluative methodologies. Repeat victimisation prevention is revealed as an effective way of reducing crime, with a need for further research to apply the principle across further crime types. A requirement is identified for a greater breadth and depth of information to be included in future crime prevention evaluations. The systematic review is shown to be a useful way of assessing the overall effectiveness of the interventions, whilst the realist synthesis fills in the detail of why some interventions work and others fail. It is concluded that both approaches to meta-evaluation have useful contributions to make, and that a third way incorporating the best elements from each method should be developed.

This case study examined how senior exhibitions function in a progressive educational environment that employs traditional assessments. The researcher interviewed 18 students and three faculty members of The Crefeld School about the Creative Expression, one of the 14 exhibitions that students are required to complete for graduation. The researcher conducted a document analysis and content analysis of rubrics for two essays and the Creative Expression. The researcher conducted a survey of student participants and a questionnaire to assess students’ perceptions of authenticity. The researcher observed three students present their Creative Expression in front of faculty, students, and family. The researcher conducted a survey analysis based on Cooper’s (1976) and Whitney’s (1978) tests for polarity of sentiment and Hsu’s (1979) test for disagreement. The researcher used a program designed by Morris (1979) and transposed to Microsoft Excel by Liebermann and Morris (2015) in order to calculate p values and to determine the ES value based on the standard deviation. The researcher also attempted to conduct a point biserial Pearson Product Moment to determine if a relationship existed between results on a mathematics exam and results on the Creative Expression. The findings of the study revealed that students at The Crefeld School had a positive experience working on the Creative Expression; many of them identified how the project reflected their interests, prepared them for college and careers, and was a good representation of authentic assessment. Tests for polarity were used to measure participants’ perceptions of authenticity. The category of task had the least amount of disagreement, while the category of overall authenticity had the most disagreement. Document analysis and content analysis of rubrics showed how thorough the rubric descriptions were so students knew how they were being assessed; interviews with students and faculty revealed that the rubrics, while used as guidelines, were not followed strictly in any of the major assessments. Survey data revealed any disagreements in responses to survey questions, based on Cooper’s (1976) and Whitney’s (1978) tests for polarity. As no dichotomous variable existed, a point biserial test was unwarranted regarding results on a traditional assessments and on the Creative Expression. / Includes bibliography. / Dissertation (Ph.D.)--Florida Atlantic University, 2016. / FAU Electronic Theses and Dissertations Collection

NPOs and their funders are increasingly drawn to the Social Return on Investment (SROI) method to evaluate the social impact of programs, organizations or organization networks. While many claims about the benefits of SROI have been expressed, various points of criticism have also been raised. On the basis of both current research and our own experience in conducting SROI analyses, we develop a comprehensive assessment of this method, which is structured along two dimensions: the observer's paradigmatic perspective, on the one hand, and positive or negative valuation, on the other. We identify two major merits: SROI analysis can provide legitimacy to NPOs or their funders, and it can assist in allocating resources. We identify limitations from three perspectives: From an interpretative-sociological perspective, criticism of commensuration and utilitarianism calls the method as a whole into question. From a technical-instrumental perspective, there are a number of difficulties that could however be overcome as the method matures. From an intermediary perspective, a number of limitations become apparent that, while inherent to SROI analysis, are no reason for abandoning it, as long as they are thoroughly understood. We conclude by providing suggestions for the responsible use of SROI analysis.

Community arts are potentially valuable tools in building community and regenerating distressed neighbourhoods. Community-based art organizations exist in most major cities across North America and abroad. These groups are concerned with social and environmental community issues (e.g., youth poverty, sustainability, racism) and use art as a medium for social change through community empowerment and personal development. Many of these organizations operate on limited funding and are required to complete program evaluations to demonstrate the merit of their programs. While some program evaluation literature touches on the role of arts-based research methods, very little focuses specifically on using these methods with community-based art organizations—particularly organizations with programming intended for children and youth. This Major Degree Project seeks to address this gap and explore the role of creative, arts-based evaluation methods for community-based art organizations’ program evaluation.

This study is an examination of what teacher preparation institutions in the Canadian province of Alberta are teaching student teachers about ‘assessment-for-learning’ practices. A large body of research exists to show that assessment-for-learning practices are among the best tools educators have for improving student learning; therefore, it is important that assessment-for-learning practices be taught to prospective teachers. As a school administrator, I had encountered many first-year teachers who seemed to be lacking knowledge and skills in assessment-for-learning practices and I sought to determine whether or not the deficiencies could be traced to inadequate preparation during their undergraduate training. Interviews were conducted with instructors at seven Alberta education faculties on whether and how assessment-for-learning practices were integrated into their teacher education programs. There is some variance in the way courses are organized to teach assessment-for-learning practices to student teachers; some schools having a designated course on assessment, others embedding assessment into other teacher preparation courses. Overall, the findings in this study suggest that Alberta education faculties are providing future teachers with training in assessment-for-learning practices. Further research into other areas of teacher preparation and/or induction would be necessary to determine the reasons for deficiencies in new teacher knowledge of assessment-for-learning practices. This study also includes some recommendations for improving instruction in assessment for learning at Alberta teacher preparation institutions, as well as some suggestions for further study. / vii, 73 leaves ; 29 cm

Community arts are potentially valuable tools in building community and regenerating distressed neighbourhoods. Community-based art organizations exist in most major cities across North America and abroad. These groups are concerned with social and environmental community issues (e.g., youth poverty, sustainability, racism) and use art as a medium for social change through community empowerment and personal development. Many of these organizations operate on limited funding and are required to complete program evaluations to demonstrate the merit of their programs. While some program evaluation literature touches on the role of arts-based research methods, very little focuses specifically on using these methods with community-based art organizations—particularly organizations with programming intended for children and youth. This Major Degree Project seeks to address this gap and explore the role of creative, arts-based evaluation methods for community-based art organizations’ program evaluation.