Crowdsourcing involves releasing tasks on the internet for people with diverse backgrounds and skills to solve. Its adoption has come a long way, from scaling up problem-solving to becoming an environment for running complex experiments. Designing tasks to obtain reliable results is not straightforward as it requires many design choices that grow with the complexity of crowdsourcing projects, often demanding multiple trial-and-error iterations to properly configure. These inherent characteristics of crowdsourcing, the complexity of the design space, and heterogeneity of the crowd, set quality control as a major concern, making it an integral part of task design. Despite all the progress and guidelines for developing effective tasks, crowdsourcing still is addressed as an ``art'' rather than an exact science, in part due to the challenges related to task design but also because crowdsourcing allows more complex use cases nowadays, where the support available has not yet caught up with this progress. This leaves researchers and practitioners at the forefront to often rely on intuitions instead of informed decisions. Running controlled experiments in crowdsourcing platforms is a prominent example. Despite their importance, experiments in these platforms are not yet first-class citizens, making researchers resort to building custom features to compensate for the lack of support, where pitfalls in this process may be detrimental to the experimental outcome. In this thesis, therefore, our goal is to attend to the need of moving crowdsourcing from art to science from two perspectives that interplay with each other: providing guidance on task design through experimentation, and supporting the experimentation process itself. First, we select classification problems as a use case, given their importance and pervasive nature, and aim to bring awareness, empirical evidence, and guidance to previously unexplored task design choices to address performance concerns. And second, we also aim to make crowdsourcing accessible to researchers and practitioners from all backgrounds, reducing the requirement of in-depth knowledge of known biases in crowdsourcing platforms, experimental methods, as well as programming skills to overcome the limitations of crowdsourcing providers while running experiments. We start by proposing task design strategies to address workers' performance, quality and time, in crowdsourced classification tasks. Then we distill the challenges associated with running controlled crowdsourcing experiments, propose coping strategies to address these challenges, and introduce solutions to help researchers report their crowdsourcing experiments, moving crowdsourcing forward to standardized reporting.

This thesis investigates and analyses the level of challenge for able students in an 11-18 Academy. It is addressed from my position as the Principal of the case study Academy and a novice researcher. Eight teachers who formed the Teaching and Learning group within the Academy participated in the study, as part of a community of practice with an interest in the issue addressed and the research process. The study focused on concerns arising from Learning Walks and Ofsted feedback about the perceived lack of challenge for able students. Using a three layer action research methodology, the views and practices of staff and students about challenge in ILTs (Independent Learning Tasks) were explored. An initial brainstorming activity was followed by questionnaires, lesson observations and focus group sessions with a sample of 100 students (Years 7, 9, 10 and 11). At the close of the first layer of research, data analysis revealed a range of levels of challenge in different subject areas, and from these a Year 10 Geography group was selected, with the support of the teacher. The second action research layer involved the Geography teacher and 15 Geography students who had identified a lack of challenge in their ILTs. This shifted the focus of the research to consider the cognitive challenge incorporated into tasks, focusing on thinking skills and questioning techniques. The third and final action research layer resulted in a newly developed, collaboratively-constructed ‘student friendly’ thinking skills analysis which provided powerful and formative insights to ‘label’ challenge. The teacher responded reflexively to the outcomes by trying out a redeveloped approach to ILTs (homework) and questioning techniques within the Academy. The findings from this investigation suggest that, cognitively challenging, problem-solving tasks, co-constructed with students to include opportunities for Socratic questioning provide for greater challenge in the classroom. Finally, the benefits to be gained from establishing a research community where the Principal is the lead researcher, include an increased emphasis on staff as change agents and the critical contribution of student voice in pursuit of challenging teaching and learning.

In recent years many teachers and students have begun having good access to digital technology in their classrooms, and in the context of Sweden the majority of secondary schools are known as one-to-one schools, with students having their own computer or tablet. However, the mere presence of technology in the classroom is not a guarantee for improved teaching and learning. In fact, there is a challenge involved with integrating technology in the classroom and many teachers need support. Therefore, the aim of this thesis is to contribute to the knowledge about support for teachers integrating digital technology, especially a classroom response system (CRS), in the mathematics classroom. This is done by focusing on frameworks for CRS task design and technology integration. The thesis consists of two papers and a kappa. Both papers use data from a design research project including interventions in two cases. Paper I focuses on the development of design principles and task types for CRS tasks in a multiple-choice format aiming to engineer mathematical classroom discussions. The study generated three design principles, six task types, and 31 empirically evaluated tasks. The empirical evaluation shows that teachers consider the evaluated CRS tasks useful for engineering mathematical classroom discussions. Paper II focuses on exploring the potential of Ruthven’s (2009) SFCP framework as tool for analyzing empirical data in order to conceptualize and analyze teachers’ reasoning about critical aspects of technology integration in the mathematics classroom. The results show that the SFCP framework can be useful for capturing teachers’ reasoning about critical aspects of technology integration, but also that the framework does not capture teachers’ reasoning about students’ attitudes and behaviors. Therefore, the framework would benefit from taking into consideration students’ attitudes and behaviors, as these features are a challenge teachers need to deal with when integrating technology in the classroom. This thesis kappa, building on earlier research as well as the results and methods of its own papers, ends with an elaborated discussion on the challenges and support for teachers wanting to integrate CRS in their mathematics classroom. / Lärare och elever har idag god tillgång till digital teknik i klassrummen. I Sverige är numera majoriteten av högstadie- och gymnasieskolorna en-till-en skolor där alla elever har en egen dator eller surfplatta. Blotta närvaron av digital teknik i klassrummen är inte en garanti för förbättrad undervisning och ökat lärande. I själva verket så är det en utmaning att integrera digital teknik i klassrummet och många lärare behöver stöd för att lyckas med detta. Syftet med den här uppsatsen är att bidra till kunskap om stöd för lärare som integrerar digitala verktyg i matematikklassrummet. Detta görs dels genom att fokusera på ramverk för att konstruera matematikuppgifter till digitala responssystem och dels genom att undersöka ett ramverk för teknikintegrering i undervisningen. Uppsatsen består av två artiklar och en kappa. Båda artiklarna använder data från ett designforskningsprojekt bestående av interventioner i två separat fall. Artikel I fokuserar på utvecklingen av designprinciper och uppgiftstyper för flervalsuppgifter till digitala responssystem. Uppgifterna syftar till att initiera och fördjupa matematiska diskussioner i klassrummet. Studien genererade tre designprinciper, sex uppgiftstyper och 31 empiriskt utvärderade flervalsuppgifter som kan användas för att välja, värdera och konstruera uppgifter till responssystem. Den empiriska utvärderingen visar att lärarna anser att de testade uppgifterna och uppgiftstyperna är användbara för att initiera matematiska klassrumsdiskussioner. Artikel II fokuserar på att undersöka potentialen av Ruthvens (2009) ramverk ”Structuring Features of Classroom Practice” (SFCP), som ett analytiskt verktyg för att begreppsliggöra och analysera lärares resonemang om kritiska aspekter vid teknikintegrering i matematikklassrummet. Resultatet visar att ramverket är användbart för att fånga lärares resonemang om kritiska aspekter vid teknikintegrering i undervisningen, men också att ramverket inte fångade lärares resonemang om elevers attityder och uppförande. Studien föreslår att ramverket kan utvecklas och förbättras genom att även ta hänsyn till elevernas attityder och uppförande, eftersom detta är kritiska aspekter som lärare behöver hantera när de integrerar teknik i klassrummet. Med utgångspunkt från tidigare forskning och resultaten i artiklarna avslutas uppsatsens kappa med en utvecklad diskussion om utmaningar och stöd för lärare som integrerar digitala responssystem i matematikklassrummet.

<p>This thesis undertakes two main tasks; to explore, within the authentic educational context, variables that influence the quality and outcomes of the knowledge-constructing activity during the simulation exercise; and to find appropriate instruments and methods to measure these variables, processes and outcomes.</p><p>Closed-ended tasks that provided a high degree of structure, i.e., supported students’ regulation of learning during the simulation exercise, were more beneficial for perceived improvement of declarative knowledge and students’ motivation than open-ended tasks. Open-ended tasks did however lead to substantial shifts in students’ cognitive focus in subsequent laboratory exercises and improvement of students’ level of chemistry reasoning in interviews.</p><p>Student attitudes toward learning proved important in the first paper where significantly higher ability to apply chemistry knowledge in interviews could be found for students with relativistic attitudes compared to those with more dualistic perspectives on learning. In the subsequent papers, the effects of attitudes were not as clear, possibly due to too small differences in the learning situations that were compared.</p><p>Quality of the learning discourse during simulation was measured with three qualitatively different methods, focusing on partly different aspects. The different methods gave very similar results regarding the relative quality of the discourses. Thus, “quality” as such seems to be an underlying feature that permeates many aspects of the discourse and consequently could be targeted in different ways, e.g., focusing on quantitative as well as qualitative aspects. The analyses revealed several components of quality; co-operative activity, correctness and complexity of chemistry reasoning, discussion length and intensity, ability to realize cognitive conflict, and reference to theory while reasoning.</p><p>Doing the simulation exercise in a distributed learning setting (written e-communication), supported discussions with higher accuracy and complexity of chemistry reasoning and frequent references to theory while the face-to-face situation allowed for longer and more intense discussions and a higher degree of co-operative activity. Not very surprisingly, high-quality discussions were characterized by high scores in all these components. There were indications that relatively good pre-knowledge might be required to benefit fully from face-to-face discussions.</p><p>The validity of instruments and methods, used to measure flow experiences, attitudes (i.e., epistemological beliefs), knowledge accessibility (intuitive knowledge) and discourse quality are discussed thoroughly. Special interest has been devoted to whether qualitative data should be quantified or not, providing arguments in favor of quantitative methods for analyzing and reporting qualitative data.</p>

This thesis undertakes two main tasks; to explore, within the authentic educational context, variables that influence the quality and outcomes of the knowledge-constructing activity during the simulation exercise; and to find appropriate instruments and methods to measure these variables, processes and outcomes. Closed-ended tasks that provided a high degree of structure, i.e., supported students’ regulation of learning during the simulation exercise, were more beneficial for perceived improvement of declarative knowledge and students’ motivation than open-ended tasks. Open-ended tasks did however lead to substantial shifts in students’ cognitive focus in subsequent laboratory exercises and improvement of students’ level of chemistry reasoning in interviews. Student attitudes toward learning proved important in the first paper where significantly higher ability to apply chemistry knowledge in interviews could be found for students with relativistic attitudes compared to those with more dualistic perspectives on learning. In the subsequent papers, the effects of attitudes were not as clear, possibly due to too small differences in the learning situations that were compared. Quality of the learning discourse during simulation was measured with three qualitatively different methods, focusing on partly different aspects. The different methods gave very similar results regarding the relative quality of the discourses. Thus, “quality” as such seems to be an underlying feature that permeates many aspects of the discourse and consequently could be targeted in different ways, e.g., focusing on quantitative as well as qualitative aspects. The analyses revealed several components of quality; co-operative activity, correctness and complexity of chemistry reasoning, discussion length and intensity, ability to realize cognitive conflict, and reference to theory while reasoning. Doing the simulation exercise in a distributed learning setting (written e-communication), supported discussions with higher accuracy and complexity of chemistry reasoning and frequent references to theory while the face-to-face situation allowed for longer and more intense discussions and a higher degree of co-operative activity. Not very surprisingly, high-quality discussions were characterized by high scores in all these components. There were indications that relatively good pre-knowledge might be required to benefit fully from face-to-face discussions. The validity of instruments and methods, used to measure flow experiences, attitudes (i.e., epistemological beliefs), knowledge accessibility (intuitive knowledge) and discourse quality are discussed thoroughly. Special interest has been devoted to whether qualitative data should be quantified or not, providing arguments in favor of quantitative methods for analyzing and reporting qualitative data.

The overall problem that formed the basis for this thesis is that students get limited opportunity to develop their mathematical reasoning ability while, at the same time, there are dynamic mathematics software available which can be used to foster this ability. The aim of this thesis is to contribute to knowledge in this area by focusing on task design in a dynamic software environment and by studying the reasoning that emerges when students work on tasks in such an environment. To analyze students’ mathematical reasoning, a new analytical tool was developed in the form of an expanded version of Toulmin’s model. Results from one of the studies in this thesis show that exploratory tasks in a dynamic software environment can promote mathematical reasoning in which claims are formulated, examined and refined in a cyclic process. However, this reasoning often displayed a lack of the more conceptual, analytic and explanatory reasoning normally associated with mathematics. This result was partly confirmed by another of the studies. Hence, one key question in the thesis has been how to design tasks that promote conceptual and explanatory reasoning. Two articles in the thesis deal with task design. One of them suggests a model for task design with a focus on exploration, explanation, and generalization. This model aims, first, to promote semantic proof production and then, after the proof has been constructed, to encourage further generalizations. The other article dealing with task design concerns the design of prediction tasks to foster student reasoning about exponential functions. The research process pinpointed key didactical variables that proved crucial in designing these tasks. / Baksidestext Det övergripande problem som legat till grund för denna avhandling är att elever får begränsad möjlighet att utveckla sin resonemangsförmåga samtidigt som det finns dynamiska matematikprogram som kan utnyttjas för att stimulera denna förmåga. Syftet med avhandlingen är att bidra till den samlade kunskapen inom detta problemområde, dels genom att fokusera på design av uppgifter i en lärandemiljö med dynamiska matematikprogram och dels genom att studera och karakterisera de resonemang som utvecklas när elever jobbar med olika uppgifter i denna miljö. För att analysera elevernas resonemang utvecklades ett nytt analysverktyg i form av en utökad version av Toulmins modell. Resultat från en av studierna i avhandlingen visar att dynamiska matematikprogram i kombination med utforskande uppgifter kan stimulera till matematiska resonemang där hypoteser formuleras, undersöks och förfinas i en cyklisk process. Samtidigt visar samma studie att de resonemang som utvecklas i stor utsträckning saknar matematiskt grundade förklaringar. Detta resultat bekräftas till viss del av ytterligare en studie.  Frågan hur uppgifter bör designas för att främja matematiskt grundade resonemang har därför varit central i avhandlingen. Två av artiklarna behandlar uppgiftsdesign, men utifrån olika utgångspunkter.

This thesis concerns the challenge of integrating dynamic software environments into the teaching of mathematics. It investigates particular aspects of the design of tasks which employ this type of computer-based system, with a focus on improvement, both of the tasks themselves and of the design process through which they are developed and refined. The thesis reports two research projects: a small initial one preceding a larger main project. The initial case study, involving two graduate students in mathematics, develops a task design model for geometrical locus problems. The main study constitutes the first iteration of a design-based study, conducted in collaboration with four upper-secondary school teachers and their classes. It seeks to identify task design characteristics that foster students’ mathematical reasoning and proficient use of software tools, and examines teachers’ organisation of ‘follow-up’ lessons. The findings concern three particular aspects: features of tasks and task environment relevant to developing a specific plan of action for a lesson; orchestration of a particular task environment to support the instrumental genesis of specific dynamic software tools; how to follow up students’ work on computer-based tasks in a whole-class discussion.

Crowdsourcing has become a popular choice for tackling problems that neither computers nor humans alone can solve with adequate speed, cost, and quality. However, instructing crowds to execute tasks in the manner expected by the requesters is challenging. It depends on not only requesters’ task design abilities but also workers’ understanding of the tasks. Task design bridges the communication gap between workers and requesters, which consists of instructions, payment, time limit on task, and the interface for workers to work on. It remains an underdeveloped but important topic that needs further exploration for improving crowdsourcing experience. My research studies task delivery from requesters to crowd workers. The goal is to improve the communication between the two and, in turn, increase accuracy of results and decrease variability due to differing interpretations and perspectives. Specifically, this dissertation presents a series of studies to show that high-quality results can be obtained from human workers through improved task design, by 1) designing incentives to recruit workers with the appropriate skills for given tasks, 2) designing unambiguous instructions to clearly express task requirements, 3) choosing the correct strategy to communicate the requisite task knowledge with workers, and 4) enhancing requesters’ ability to rapidly prototype Augmented Reality (AR) instructions. This dissertation demonstrates that crowdsourcing quality is improved when the tasks are communicated using mediums and structures that align with workers’ preference and utility

Textbooks still dominate teaching materials in English classes in Sweden. This paper analyzes speaking tasks in two sets (textbook and workbook) of textbooks for year nine in Swedish schools, Happy, Workbook No. 3, and What’s Up? Textbook 9 and Workbook 9. The first analysis presents a content analysis, providing a general overview of speaking tasks found in both textbooks. Here, I will quantitatively account for the qualitative items types of speaking (monologue or dialogue), text types (narration, giving information, description, instruction, discussion/argumentation), cognitive operations (open/productive, open/reproductive, closed productive or closed/reproductive speech patterns), and classroom organization (single or pair work, pair or group work, or class work) in a matrix. This shows what kinds of tasks dominate the books and are required most of the students. The second is a close analysis of four speaking tasks against a framework of seven principles: scaffolding (actually demonstrating a solution), task dependency (tasks build upon each other), recycling (introducing language items in different contexts), active learning, integration (the task shows the relationship between meaning, form, and function of language items), from reproduction to creation (the order of tasks goes from reproductive to productive), and reflection (the task offers opportunity for reflection over one’s own learning). Two tasks will meet most, and two will meet only few of these principles.The content analysis reveals a vast majority of dialogues (100 out of 124 tasks, or 80.65%). This might not be surprising, keeping the communicative approach of the syllabus in mind. But it is surprising that discussion/argumentation is the text type most frequently asked for (46 out of 124 tasks, or 31.7%), not narration or giving information. They occupy a firm second and third place with 30 (24.19%) respectively 27 (21.77%) tasks out of 124. 21 (16.94%) tasks in total ask for description. Even more surprisingly, not one task demands that students give instructions. Giving instructions may not require as much two-way communication, but it still presents an important skill.This is an analysis, not an evaluation. An analysis aims at objectively accounting for what is presented and in what proportions without making some form of judgment on what is found. This would be the objective of an evaluation. This paper aims at analyzing speaking tasks and task design in English textbooks, not to pass or fail them against the needs of students or the demands of the Swedish syllabus for English.

Eine Kompetenz beschreibt im Kontext eines Lehr-/Lernprozesses fachliche und fachübergreifende Basisqualifikationen. Als latente Fähigkeit kann sie jedoch nicht direkt beobachtet werden. Eine indirekte Beobachtung ist möglich, indem sie durch eine Menge valider Aufgaben beschrieben wird, die nur erfolgreich bearbeitet werden können, wenn die Kompetenz existiert. Werden diese Aufgaben durch einen Lernenden bearbeitet, so liefert ihm das Ergebnis der Bearbeitung ein unmittelbares Feedback über seinen aktuellen Lernerfolg. Der Lehr-/Lernprozess verhält sich dabei wie ein Regelkreis, in dem das Feedback als Regelgröße Einfluss auf verschiedene Aspekte des Lernens nimmt, zum Beispiel die Motivation. Wurde eine Aufgabe nicht korrekt gelöst, wünschen sich die Lernenden Feedback, welches ihnen hilft, das Problem zu erkennen, zu verstehen und zu lösen. Dabei kann sie ein Tutor unterstützen. Betreut der Tutor jedoch mehrere Lernende gleichzeitig, zum Beispiel während einer Übung, kann es zu einer Verringerung der individuellen Betreuungszeit kommen. Dadurch fällt es dem Tutor schwerer, ein individuelles, ausführliches und sofortiges Feedback im Anschluss an die Aufgabenbearbeitung zu geben. Es wird anhand mehrerer empirischer Studien untersucht, ob zuvor entwickeltes FCF einen Beitrag zur Bewältigung dieser Herausforderung bei kompetenzorientierten Programmieraufgaben leisten kann.