The main purpose of this study was to explore the influence of using "scientific inquiry ability scoring rubric" to be the learning scaffold and self-assessment stratagem in experiment curriculum on students' scientific inquiry ability. A quasi-experimental study was implemented in a university for one semester. Of the two classes participated in this study, one received the designed teaching stratagem and the other the traditional experiment curriculum. Students' reports were scored and sufficient data of quality were collected after each experiment to test the differences of the two classess, further analysed the self-assessment data and learning-questionnaire to assess students¡¦ perceptions. In addition, the influence of students' motive on learning was also investigated. The results obtained from this study are summarized as follows: 1. Among six abilities studied those belonging to "Nature of scientific inquiry" and "Communication"¡]¡¨forming a question¡¨, ¡¨the completeness of procedures¡¨, ¡¨the sufficiency of data¡¨, ¡§the organization of data¡¨, ¡§transferring data¡¨, ¡§reviewing and evaluating¡¨¡^had a greater progress under this stratagem. 2. Students changed step by step on four inquiry-ability dimensions (i.e., bring up self inquiry questions, try to transfer data to improve science communication etc.) during this investigation period. 3. The correlation between assessments of teachers' and those of students' increased gradually, but there was no significant difference in performance between hit and miss self-assessors. 4. Students had positive attitudes toward the self-assessment questionnaire. About 80¢Mstudents accepted that using the rubrics was beneficial for learning."Attitude" and "Science Learning Value" are the two items of students' motive that showed significant relation with their learning effects.

The promise of increased student achievement through educational reform is delivered still-born if teachers do not know how to implement complex instructional practices and sophisticated analysis of student performance. Metacognitive awareness is crucial to the adoption and application of proven educational initiatives. Teachers who successfully implement criterion-referenced assessment instruction, scoring rubrics, transfer to their students the metacognitive knowledge and skills of how to learn. This study is predicated on the research assumptions that metacognition and its attendant skills are critical to the successful implementation of scoring rubrics. A researcher-developed instrument, the Scoring Rubrics Inventory (SRI) and the Metacognitive Awareness Inventory (MAI) were distributed to core-subject teachers from three large public schools in Southwest Louisiana. From a population of sixty-eight (N=68) voluntary participants, eighteen teacher-participants self-reported as high implementers of scoring rubrics, thirty-nine as mid-level implementers and eleven as low-level implementers. From this population, twelve subjects were randomly selected (four high, four mid-level, and four lowlevel implementers) by an outside rater for double-blind observations and interviews. Pearson Product Moment correlations of the SRI and the MAI revealed five significant pairings using an alpha level of .05. The statistical results, coupled with the observation and interview findings from the sample-subjects established the consistency and stability of the Scoring Rubrics Inventory. Further, the totality of the results reported here support the research hypothesis of the study: H1: There is a significant correlation between the metacognitive awareness of secondary school core-subject teachers and the successful implementation of scoring rubrics. The results of the study indicated that secondary school core-subject teachers who successfully implement scoring rubrics possess a metacognitive awareness that transcends professional development training. The findings also suggested that teacher-participants who do not implement scoring rubrics either cannot or lack commitment to the innovation. Implications for teacher educators and school leaders indicated the need to: identify those persons who require additional professional development training; include operational strategies and modeling of successful implementation during training; and maintain a consistent training program in scoring rubrics. Recommendations for future research were offered.

Thesis advisor: Henry Braun / The Next Generation Science Standards propose a multidimensional model of science learning, comprised of Core Disciplinary Ideas, Science and Engineering Practices, and Crosscutting Concepts (NGSS Lead States, 2013). Accordingly, there is a need for student assessment aligned with the new standards. Creating assessments that validly and reliably measure multidimensional science ability is a challenge for the measurement community (Pellegrino, et al., 2014). Multidimensional assessment tasks may need to go beyond typical item designs of standalone multiple-choice and short-answer items. Furthermore, scoring and modeling of student performance should account for the multidimensionality of the construct. This research contributes to knowledge about best practices for multidimensional science assessment by exploring three areas of interest: 1) item design, 2) scoring rubrics, and 3) measurement models. This study investigated multidimensional scaffolding and response format by comparing alternative item designs on an elementary assessment of matter. Item variations had a different number of item prompts and/or response formats. Observations about student cognition and performance were collected during cognitive interviews and a pilot test. Items were scored using a holistic rubric and a multidimensional rubric, and interrater agreement was examined. Assessment data was scaled with multidimensional scores and holistic scores, using unidimensional and multidimensional Rasch models, and model-data fit was compared. Results showed that scaffolding is associated with more thorough responses, especially among low ability students. Students tended to utilize different cognitive processes to respond to selected-response items and constructed-response items, and were more likely to respond to selected-response arguments. Interrater agreement was highest when the structure of the item aligned with the structure of the scoring rubric. Holistic scores provided similar reliability and precision as multidimensional scores, but item and person fit was poorer. Multidimensional subscales had lower reliability, less precise student estimates than the unidimensional model, and interdimensional correlations were high. However, the multidimensional rubric and model provide nuanced information about student performance and better fit to the response data. Recommendations about optimal combinations of scaffolding, rubric, and measurement models are made for teachers, policymakers, and researchers. / Thesis (PhD) — Boston College, 2018. / Submitted to: Boston College. Lynch School of Education. / Discipline: Educational Research, Measurement and Evaluation.

This research develops a new approach to the development of training inexperienced journalists in news writing using a web-based platform of instruction delivery. E-training is growing in the world as an instructional setting, and offers not only financial benefits, but also a range of specific advantages over the traditional classroom type of setting. Such advantages include the ability to personalise the content of training to the trainees' current competencies, to facilitate regular multi-faceted monitoring of the changes in these competencies and to combine learning with the immediate practising of what was learned. Two e-training tools have been created and validated in this research: the news text assessment system (NTA) - a comprehensive and effective online scoring rubric, i.e. a matrix describing different levels of competency in several dimensions of the assessed performance - to assess the quality of news writing; and a scoring e-textbook (SET), an asynchronous news writing training tool. The SET is built around the NTA as its core element and contains hundreds of self-learning modules including exercises, examples, instructional texts, and quizzes to be used in a non-linear fashion according to the specific needs of trainees. Both the exercises and the NTA are elements of corrective feedback, which in psychological literature has been shown to be most effective in changing the subsequent performance of trainees. The two tools help both the trainee and the instructor. They assist the instructor to identify and address journalists' weak and absent competencies in news writing and consistently upgrade the learning modules when needed. They help trainees to monitor their progress and to learn from their own mistakes in the short periods of spare time they have during their work or in other time they can spare for the training. To create the NTA, 53 top journalism experts, both practitioners and academics, used the prototype of the assessment rubric with 30 criteria of news writing to assess the quality of several supplied news stories. The results were then subjected to statistical analysis and the NTA rubric was created as a compromise between its comprehensiveness and user-friendliness. To evaluate the NTA and the SET, an experiment was conducted with journalists in one post-Communist country in the form of an action research project, where this researcher was also the instructor. The experiment consisted of four months of training and reflection on its results by both the journalists and the researcher. The results show improvement in news writing competencies to an internationally 'acceptable' news writing standard for most of the trainees. The suggested tools have been well received and the trainees appreciated the interactivity that was provided during the training.

Search Based Software Engineering is one of the important field of software engineering. Over the past few years even though there is a lot of study performed on SBSE and its search techniques in software development areas, it appears SBSE is not very industry relevant at the moment because most of the academic work was limited towards the application of search techniques. Hence, author proposes a study to know the trend of SBSE literature over the past few years and also analyze to what degree current SBSE research is industry relevant

Why this thesis is needed. This thesis is motivated by the falling school results of Swedish 15-year-olds, a lack of IT tools in Swedish schools and a call for turning the theory on formative assessment into practice. Previous research that is used in the thesis. This thesis mainly builds on the research done by John Hattie which is presented in his book Visible Learning from 2009, Wiliam & Thompson's research on effective formative assessment from 2007, Pachler et al. research on formative e-assessment from 2010 and Panadero & Jonsson's research on scoring rubrics from 2013. Research question. What key aspects should be taken into consideration when designing and implementing an IT tool based on scoring rubrics which aims to support formative assessment according to the key strategies for effective formative assessment as identified by Wiliam & Thompson (2007)? Method. To answer this question I chose a qualitative approach of parallel paper prototyping where I iteratively exposed the design and my ideas during focus groups to teachers, a headmaster, employees at The Swedish National Agency for Education and high school students, followed by an evaluation of the key topics that surfaced during the focus groups. Results. The results consist of eight key aspects to take into consideration when designing and implementing such a tool. Discussion and future research. Finally I discuss the implications of my findings and present directions for future research which include the construction of the tool, further investigation of the key aspects identified in this study, identification of additional key aspects and more. / Varför denna avhandling behövs. Denna avhandling motiveras av de fallande skolresultaten hos svenska 15-åringar, en brist på IT-verktyg i svenska skolor och ett rop efter att vända forskning inom formativ bedömning till praktik. Tidigare forskning som används. Denna avhandling bygger i huvudsak på forskningen av John Hattie som presenteras i hans bok Visible Learning från 2009, William & Thompsons forskning på effektiv formativ bedömning från 2007, Pachler et al. forskning på formativ e-bedömning från 2010 och Panadero & Jonssons forskning på betygsmatriser från 2013. Forskningsfråga. Vilka nyckelaspekter ska tas i åtanke vid utveckling av ett IT-verktyg baserat på betygsmatriser som stödjer formativ bedömning enligt de fem nyckelstrategierna för effektiv formativ bedömning som formulerats av Wiliam & Thompson (2007)? Metod. För att besvara denna fråga valdes en kvalitativ metod där parallell pappersprototypning användes under ett antal iterationer av fokusgrupper under vilka designen och tidigare diskussionpunkter diskuterades med lärare, en rektor, anställda vid Skolverket samt gymnasiestudenter. Fokusgrupperna följdes upp med en utvärdering av de största diskussionpunkterna som dök upp. Resultat. Resultaten består av åtta nyckelaspekter att ha i åtanke vid utveckling av ett sådant verktyg. Diskussion och framtida forskning. Slutligen diskuteras implikationerna av resultaten och direktioner för framtida forskning framförs. Dessa direktioner inkluderar utveckling av verktyget, vidare utredning av de nyckelaspekter som hittats i denna avhandling, identifikation av ytterligare nyckelaspekter och mer.

The use of analytic rubrics remains popular in the field of writing assessment. Previous work in second-language writing assessment and other fields like economics suggest that ratings produced using this method may have lower reliability on average than other methods. Currently, there is little research on the reliability of ratings of creative writing, specifically creative writing authored by adults. This study evaluated the reliability of ratings from an analytic rubric against those produced by a comparative method called Randomly Distributed Comparative Judgment and the rater experience of the methods. The author administered a science fiction and fantasy contest in which 9 raters rated subsets of 47 total contest entries. Raters used both methods on two occasions for a total of four ratings per assigned artifact. The analytic rubric ratings were analyzed using the Many-Facets Rasch Model to model story, rater, occasion, and interaction effects. The comparisons from the RDCJ method were used in a proprietary version of the Bradley-Terry Model to calculate true scores and rater effects. Analysis showed rater effects in the ratings of both methods, though greater for those associated with the rubric model. The ratings from the rubric also contained occasion effects, but the RDCJ ratings did not. Interviews with the raters found that raters generally favored the RDCJ method, though some would have preferred a modified version. However, they all found the rubric less useful, even though many thought that it covered the generally accepted factors of good creative writing. These findings may influence practitioners' decisions when choosing a rating method for shorter works of creative writing, particularly in contexts like story contests or university admissions. However, rating of creative writing is an understudied field compared to academic writing, and more work is needed in the areas of reliability and rating.

A relatively large number of 8th-grade public middle school students in the United States, particularly in urban communities, are not performing at acceptable levels in mathematics. One concept that poses significant difficulty for these students and negatively affects their overall mathematics achievement is fractions. Many researchers have attributed these difficulties primarily to traditional fraction instruction that emphasizes procedural rather than conceptual knowledge. Therefore this study was designed to investigate how students use their computational and conceptual knowledge and fraction representations to solve fraction-related performance-based mathematical tasks. Social constructivism was used as the theoretical framework in examining conceptual knowledge related to learning fractions. This qualitative study was implemented in an urban middle school in the southeast. It involved an initial sample of 37, 8th-grade, African American pre-algebra students who completed a fraction interest questionnaire and two fraction pretests. During the implementation period, 34 students in the researcher’s pre-algebra class completed three performance-based tasks, three reflection logs, and participated in an interview after completing each task. Of the 34 students who completed all tasks, three were purposefully selected as the informants for the study. In addition, observations, field notes, and artifacts (student work) were utilized to facilitate triangulation of the data. The findings of the study indicated the informants could compute fractions with an average of 85% of mastery but could conceptualize fractions only to a small extent. This validated prior findings and led to the conclusion that student deficiency with fractions results primarily from their level of conceptual knowledge. In the investigation of the ways in which 8th-grade students use fraction representations, this study found the informants used representations to develop a visual map of their mathematical thinking and reasoning and to check the accuracy of their computations. Therefore, this study suggests, when students’ mathematical learning experiences relative to fractions have not emphasized the use of representations to develop conceptual knowledge, they may not be comfortable with the accuracy of the solutions demonstrated in their fractions models.

A healthy civil society depends on citizens who have mature critical thinking skills and a willingness to entertain opposing points of view. The development of critical thinking in young adults has long been studied, but there has been little agreement on what the attributes of critical thinking are and how to reliably assess them. While many studies have attempted to assess the critical thinking abilities of college students, none have yet measured critical thinking through using the Critical Thinking Analytic Rubric (CTAR) to assess first-year college students' writing. This study used a modified version of the CTAR rubric to investigate students' critical thinking in writing completed for an American Heritage course. Four hypotheses were tested: (1) that raters would use the rubric with high inter-rater reliability estimates; (2) that there would be a significant relationship between the scores from the earlier holistic rubric used in the 2015 Hansen et al. study and the scores from the analytic rubric used in this study; (3) that there would be a significant relationship between analytic scores and ACT and GPA scores; (4) that there would be a significant relationship between essay score and gender. Findings included the following: (1) The inter-rater reliability for the overall scores of the papers was 0.898, which exceeds the 0.70 acceptable level. However, the inter-rater reliability for sub-scores was negative and required further investigation. (2) There was no significant relationship between the scores of the Hansen et al. study and this study. (3) There was no significant relationship between essay scores and ACT and GPA scores. (4) There was a significant relationship between essay scores and gender, with female students scoring higher than male students.

Studien tar avstamp i praktikens frågor. Svensklärare och elever på gymnasieskolan har intervjuats för att skönja uppfattningar och erfarenheter om bedömningsmatrisers tillämpning i undervisningen. Studiens resultat har diskuterats ur ett OTL-perspektiv. Tre lärare deltog i studien och enskilda semistrukturerade intervjuer genomfördes med samtliga. Sex elever intervjuades i fokusgrupper uppdelade om tre elever i varje.  Resultatet presenteras genom tematisk analys av transkriberade intervjuer.  Studiens resultat visar att lärarna och eleverna delar liknande uppfattningar om och av bedömningsmatrisers tillämpning i undervisning. Bedömningsmatriser ansågs ha både för- och nackdelar som är av betydande roll dels för lärarens yrkesutövning, dels för elevens lärande. Bedömningsmatrisens största utmaning sågs av både lärare och elever som för abstrakt för att eleverna ska förstå skillnaden i kvaliteter mellan de olika nivåerna. Den största möjligheten med matrisanvändning uppfattades vara en tydlig kartläggning av elevens kunskaper. / The study is based on practice questions. Swedish teachers and students in upper secondary school have been interviewed to discern perceptions and experiences about the application of scoring rubrics in teaching. The results of the study have been discussed from an OTL perspective. Three teachers participated in the study and individual semi-structured interviews were conducted with all of them. Six students were interviewed in focus groups divided into three students in each group. The result of the study is presented through thematic analysis of transcribed interviews. The results of the study show that teachers and students share similar views about and of the application of scoring rubrics in teaching. Scoring rubrics were considered to have both advantages and disadvantages that are of significant role for both the teacher's professional practice and for the student's learning. The biggest shortcoming of the scoring rubric was seen by both teachers and students as too abstract for the students to understand the difference in qualities between the different levels. The biggest advantage of the rubric was perceived to be a clear mapping of the student's knowledge.