With the increase in frequency of the use of mixed methods, both in research publications and in externally funded grants there are increasing calls for a set of standards to assess the quality of mixed methods research. The purpose of this mixed methods study was to conduct a multi-phase analysis to create a preliminary rubric to evaluate mixed methods research articles. This study included four research questions: 1. What are the common evaluation criteria found in the contemporary methodological literature pertaining to the design of mixed methods research? 2. What evaluation criteria do experts in the field of mixed methods research perceive as the most important when distinguishing top-quality research in mixed methods? 3. What differences are there in the outcome of the rubric for evaluating mixed methods research identified from the literature compared to those advocated most uniformly by a panel of mixed methods research experts? 4. What are disciplinary differences between the use of mixed methods and views about evaluating it, including the role of paradigms in mixed methods research? In the first phase of this multi-phase mixed methods study I used an inductive qualitative process to identify the quality criteria endorsed by 12 methodologists with a long-term involvement in mixed methods research. In the second phase of this study I conducted a quantitative analysis to pilot test a set of criteria identified in the qualitative phases. The sample for both phases of this study was comprised of the same eight males  and four females from multiple nationalities. Respondents to the on-line survey rated all 14 items as being important, with 11 of the 14 items being rated as very important or higher. When considered together, findings from the two phases of this study provide a interesting view of attitudes about the use and application of quality standards to the mixed methods literature. While there was agreement about what elements were important to evaluate, there was not an agreement about the idea that one set of standards could be applied to all mixed methods studies. / Ph. D.

The purpose of the study was to explore the overall perceptions that resource room teachers had of the required SMART goals, rubric outcome sampling, and the collaborative effort of Personal Program Planning team. This study included a descriptive, embedded single-case study having three sub-units. Each subunit consisted of one resource room teacher who was teaching in a central Saskatchewan urban school division at the elementary level. Each resource room teacher was asked to select one student with a cognitive, behavioural, or multiple disability and a previous PPP document written for him or her (i.e., this is not the students first year of meeting the criteria for Intensive Supports) by that particular resource room teacher. Each resource room teacher participated in three separate focus open-ended interviews designed to explore their perceptions of SMART goals, rubric outcome sampling, and the collaborative nature of the PPP process.<p> Pattern-matching and exploration building were the two analytic techniques used in this study. Numerous themes were identified in the data. The themes present in data collected from at least two of the participants included: the need to be flexible with parents; resource room teachers have large workloads; concern over EAs not being able to attend PPP meetings; the need for rubrics to be discussed within the context of a PPP meeting; the effect of having different knowledge bases and levels of expertise represented in a PPP team; the use of visual aides during the PPP meeting; and working with the dual role of resource room teacher and vice principal.

The purpose of the study was to explore the overall perceptions that resource room teachers had of the required SMART goals, rubric outcome sampling, and the collaborative effort of Personal Program Planning team. This study included a descriptive, embedded single-case study having three sub-units. Each subunit consisted of one resource room teacher who was teaching in a central Saskatchewan urban school division at the elementary level. Each resource room teacher was asked to select one student with a cognitive, behavioural, or multiple disability and a previous PPP document written for him or her (i.e., this is not the students first year of meeting the criteria for Intensive Supports) by that particular resource room teacher. Each resource room teacher participated in three separate focus open-ended interviews designed to explore their perceptions of SMART goals, rubric outcome sampling, and the collaborative nature of the PPP process.<p> Pattern-matching and exploration building were the two analytic techniques used in this study. Numerous themes were identified in the data. The themes present in data collected from at least two of the participants included: the need to be flexible with parents; resource room teachers have large workloads; concern over EAs not being able to attend PPP meetings; the need for rubrics to be discussed within the context of a PPP meeting; the effect of having different knowledge bases and levels of expertise represented in a PPP team; the use of visual aides during the PPP meeting; and working with the dual role of resource room teacher and vice principal.

There has been a continuous push among industry, educators, and accreditation organizations to infiltrate all levels of engineering education with design skills development instruments. At Texas A&M University there was the unique opportunity to modify a manufacturing and materials laboratory with this ideal in mind. Prior to 2001 the materials and manufacturing laboratories were independent initiatives. Recently, these courses have been combined into one entity. It was proposed that if these two courses integrated fully under the umbrella of one project, that the students would better understand the nature of product development in design and that this simple change would result in a higher level of learning. The proposed manufacturing and materials selection project was a 1/8th scale replica of a 12 lb. Civil War Napoleon Cannon in a field mount. The product was selected due to its ease of manufacture and potential for addressing a sufficient variety of materials during development. The development of the product followed a simple timeline. Initially, students took an existing model and used it to develop working drawings. Next the barrel material was selected by examining the performance of two materials using common testing methods. Selected materials were then subjected to heat treatment. Once the material processing was complete; Manual Machining, CNC Machining, Welding and a novel Rapid Manufacturing approach were used to produce the cannons. The cannons were then tested and destroyed for metallographic examination. A quasi-experimental two by two factorial design was used to evaluate the effects of the innovative laboratory treatment compared with the effects of standard laboratory treatment. Assessment was performed using two instruments. These instruments consisted of three student surveys and two open-ended qualitative essays graded for depth of learning using analytic rubrics. Preliminary results indicate that the students are highly enthused by the new class. Analysis of the open-ended qualitative essays indicate that the students in the treatment, or project-based, laboratory performed at an equal level to those in the non-treatment, or control group.

Critical thinking skills have been identified as learning outcomes expected of students for most courses of tertiary education in many countries including Malaysia. One of the courses where critical thinking is required is academic writing. Producing academic writing which is well argued, insightful, thought-provoking, characterised by evidence and wide reading is a challenge for undergraduate students. Not only do the students need to have a good command of the language, they also need to be critical as they examine viewpoints, facts and arguments and synthesise them. This thesis explores several approaches to developing critical thinking skills in an academic writing course for undergraduate students. The use of a rubric or a checklist and discussion with peers were identified in the study to support the development of critical thinking. Their potency was explored in a quasi- experimental study involving undergraduate students taking English for Academic Writing course. The three treatments groups were: peer review where students used a checklist and discussed their ideas; peer evaluation where students used the rubric and discussed their ideas and evaluations; self-evaluation where students used the rubric but did not discuss their ideas. The level of critical thinking for each groups and a control group who received no treatment, was measured before and after learning interventions using two instruments: the Cornell Critical Thinking Test Level X (CCTT-X) and the English for Academic Writing term paper. In addition, students’ and instructors’ perspectives on the learning activities were elicited by means of questionnaires and interviews. Classroom observations were also carried out. The rubric which was used in the peer evaluation and self-evaluation activities is called the Critical Thinking for Academic Writing Analytical Rubric (CAWAR). It contains 12 criteria with descriptions of the two ends of performance domains i.e. the best and the weakest points and a commentary space. The checklist used in the peer review activity, on the other hand, replicates the CAWAR except that it does not have the grading element. The study found that all treatments showed some potential for fostering the development of critical thinking skills. Theoretically, it is argued that peer evaluation has the greatest potential of the three treatments provided that both teachers and students understand the value of collaborative learning and the importance of giving sufficient time for discussion. The introduction of either the rubric or checklist or promoting peer discussion has promoted critical thinking in an academic writing course.

ESL writing teachers often deal with a heavy workload of giving feedback to students. Training students to self-assess their work can ease teachers' burdens; one self-assessment method is rubric training (RT), where teachers guide students in reading and grading sample essays. This research explores whether RT leads to positive emotional and regulatory gains. Twenty-one students enrolled in a first-year writing class received incremental exposure to RT. The same set of surveys, Self-efficacy Scale (SES) and Self-regulated Learning Perception Scale (SRLPS), was administered three times to measure the changes in their language self-efficacy (LSE) and self-regulated learning (SRL). The results were compared to the 15 students in the control group, where students also completed the same surveys on the same class days. Results showed that RT had a significant impact on students' LSE, but there was no presence of interaction or main effect for SRL. In post-assessment interviews, most students expressed feeling positive about RT because of the model essays; they learned about essay organization and coherence from the sample writing. However, some did not like the rubric because they thought it was difficult to read and the lexical complexity of the rubric was above students' reading level; it might be what made students feel less capable of completing the task. Still, LSE bridges students' self-assessment and language gains; therefore, RT should be used when teachers want to increase students' LSE.

Information communication technologies (ICT) in education is an expanding field, and within this field there is a need for development of effective systems for faculty and learners to communicate feedback and assess performance. The increasing migration of many academic disciplines and courses to an online format has prompted an increased need for the development of evidence-based tools and techniques that can be made available to educational institutions in order to connect and communicate with the learner. Many faculty and students have been faced with a less than optimal transition to the online environments and for these faculty and students, educational performance can be hindered. Guided by a developmental research methodology (DRM), the researcher examined the need for a novel communication and assessment system, the elements required for such a system to succeed, the building of such a system, and finally the testing of the initial effectiveness of the system. The developed system, known as an Online Assessment System (OAS) or Gradeswift (prototype name), incorporated criterion based assessment utility with multi-modal (text, voice, and video) functionality to create an asynchronous means for communicating effectively through online information storage and delivery. It was shown that the OAS had a significant impact on the educational landscape by providing better communication and performance assessment. Additionally, the OAS had been designed to increase the efficiency of the process of assessment, thus assisting faculty with workload and time required to assess student assignments. Last, students have the added benefit of receiving personalized feedback from instructors, enhancing the experience and perceived usefulness of online assessment.

There were five main purposes for the current thesis: (1) to address the need for more quantitative studies to evaluate student academic success within the inclusive classroom setting; (2) to apply a recently released program assessment rubric for special education services to determine the level of special education service delivery in the specified location; (3) to evaluate the reliability of the results of the rubric mandated by the Saskatchewan Ministry of Education (2008); (4) to compare the results of the standardized student achievement tests with the rubric results in relation to program effectiveness; and (5) to investigate potential confounding factors related to the current study design. The goal of this thesis was to provide information to the Living Sky School Division and to the Saskatchewan Ministry of Education on the implementation and success of the inclusion model in a rural Saskatchewan setting. In addition, results were intended to provide information on assessment instruments employed in the measurement of program effectiveness. The analysis was conducted as a mixed-methods case study that included two parts: (1) the first assessment indicated that students with learning difficulties scored significantly higher on standardized academic achievement measures while in an inclusive setting as opposed to scores while in a pullout setting; and (2) the second assessment determined that special education service delivery was <i>emerging/developing</i> to <i>evident</i>. The correlation coefficient of rubric results was calculated at á = .69. A variety of general measurement issues, including small sample size and use of historical data, in relation to the current study design, were discussed.

There were five main purposes for the current thesis: (1) to address the need for more quantitative studies to evaluate student academic success within the inclusive classroom setting; (2) to apply a recently released program assessment rubric for special education services to determine the level of special education service delivery in the specified location; (3) to evaluate the reliability of the results of the rubric mandated by the Saskatchewan Ministry of Education (2008); (4) to compare the results of the standardized student achievement tests with the rubric results in relation to program effectiveness; and (5) to investigate potential confounding factors related to the current study design. The goal of this thesis was to provide information to the Living Sky School Division and to the Saskatchewan Ministry of Education on the implementation and success of the inclusion model in a rural Saskatchewan setting. In addition, results were intended to provide information on assessment instruments employed in the measurement of program effectiveness. The analysis was conducted as a mixed-methods case study that included two parts: (1) the first assessment indicated that students with learning difficulties scored significantly higher on standardized academic achievement measures while in an inclusive setting as opposed to scores while in a pullout setting; and (2) the second assessment determined that special education service delivery was <i>emerging/developing</i> to <i>evident</i>. The correlation coefficient of rubric results was calculated at á = .69. A variety of general measurement issues, including small sample size and use of historical data, in relation to the current study design, were discussed.

In this study I investigated the interactions of 25 Grade 8 science students as they learned how to construct oral arguments using the Toulmin Argumentation Pattern framework. I collected the data during three recorded small group discussion sessions during a five week Earth Science unit between February and March of 2011. The first session recorded the students’ discussions prior to receiving either argumentation instruction or the science concept instruction. The second session recorded their discussions after receiving an introduction to argumentation and a scaffold, but not concept instruction. During the three weeks preceding the third session, the students received additional argumentation instruction and completed one-third of the Earth Science unit. The results showed the students collectively made more arguments during each subsequent session. The students’ individual arguments showed a correspondence between their purportedly most familiar topics and the most discussed topics. I also found that when students made counter arguments and/or invited or challenged group members to participate, their discussions contained comparatively more argument elements (claims, data and warrants) than discussions containing predominantly collaborative assertions. The key outcome of this study for developing students’ use of the elements of argumentation during classroom discussions was to recognize and incorporate opportunities for the students to tap into their prior-knowledge. To engage students in this process, the results indicate the importance of creating time for discussions relevant to the curriculum and to the students. / Graduate / 0727 / 0279 / 0714 / brianu@uvic.ca