Multiple-choice questions are acknowledged to be difficult for both English mother-tongue and second-language university students to interpret and answer. In a context in which university tuition policies are demanding explicitly that assessments need to be designed and administered in such a way that no students are disadvantaged by the assessment process, the thesis explores the fairness of multiple-choice questions as a way of testing second-language students in South Africa. It explores the extent to which two multiple-choice Linguistics examinations at Unisa are in fact ‘generally accessible’ to second-language students, focusing on what kinds of multiple-choice questions present particular problems for second-language speakers and what contribution linguistic factors make to these difficulties. Statistical analysis of the examination results of two classes of students writing multiple-choice exams in first-year Linguistics is coupled with a linguistic analysis of the examination papers to establish the readability level of each question and whether the questions adhered to eight item-writing guidelines relating to maximising readability and avoiding negatives, long items, incomplete sentence stems, similar answer choices, grammatically non-parallel answer choices, ‘All-of-the-above’ and ‘None-of-the-above’ items. Correlations are sought between question difficulty and aspects of the language of these questions and an attempt is made to investigate the respective contributions of cognitive difficulty and linguistic difficulty on student performance. To complement the quantitative portion of the study, a think-aloud protocol was conducted with 13 students in an attempt to gain insight into the problems experienced by individual students in reading, understanding and answering multiple-choice questions. The consolidated quantitative and qualitative findings indicate that among the linguistic aspects of questions that contributed to question difficulty for second language speakers was a high density of academic words, long items and negative stems. These sources of difficulty should be addressed as far as possible during item-writing and editorial review of questions.

Drawing upon the writing literature and the task-based language teaching literature, the study examined two cognitive complexity dimensions of L2 writing tasks: rhetorical task varying in reasoning demand and topic familiarity varying in the amount of direct knowledge of topics. Four rhetorical tasks were studied: narrative, expository, expo-argumentative, and argumentative tasks. Three topic familiarity tasks were investigated: personal-familiar, impersonal-familiar, and impersonal-less familiar tasks. Specifically, the study looked into the effects of these two cognitive complexity dimensions on L2 writing quality scores, their effects on complexity, accuracy, and fluency (CAF) of L2 production, and the predictive power of the CAF features on L2 writing scores for each task. Three hundred and seventy five Chinese university EFL students participated in the study, and each student wrote on one of the six writing tasks used to study the cognitive complexity dimensions. The essays were rated by trained raters using a holistic scale. Thirteen CAF measures were used, and the measures were all automated through computer tools. One-way ANOVA tests revealed that neither rhetorical task nor topic familiarity had an effect on the L2 writing scores. One-way MANOVA tests showed that neither rhetorical task nor topic familiarity had an effect on accuracy and fluency of the L2 writing, but that the argumentative essays were significantly more complex in global syntactic complexity features than the essays on the other rhetorical tasks, and the essays on the less familiar topic were significantly less complex in lexical features than the essays on the more familiar topics. All-possible subsets regression analyses revealed that the CAF features explained approximately half of the variance in the writing scores across the tasks and that writing fluency was the most important CAF predictor for five tasks. Lexical sophistication was however the most important CAF predictor for the argumentative task. The regression analyses further showed that the best regression models for the narrative task were distinct from the ones for the expository and argumentative types of tasks, and the best models for the personal-familiar task were distinct from the ones for the impersonal tasks.

Multiple-choice questions are acknowledged to be difficult for both English mother-tongue and second-language university students to interpret and answer. In a context in which university tuition policies are demanding explicitly that assessments need to be designed and administered in such a way that no students are disadvantaged by the assessment process, the thesis explores the fairness of multiple-choice questions as a way of testing second-language students in South Africa. It explores the extent to which two multiple-choice Linguistics examinations at Unisa are in fact ‘generally accessible’ to second-language students, focusing on what kinds of multiple-choice questions present particular problems for second-language speakers and what contribution linguistic factors make to these difficulties. Statistical analysis of the examination results of two classes of students writing multiple-choice exams in first-year Linguistics is coupled with a linguistic analysis of the examination papers to establish the readability level of each question and whether the questions adhered to eight item-writing guidelines relating to maximising readability and avoiding negatives, long items, incomplete sentence stems, similar answer choices, grammatically non-parallel answer choices, ‘All-of-the-above’ and ‘None-of-the-above’ items. Correlations are sought between question difficulty and aspects of the language of these questions and an attempt is made to investigate the respective contributions of cognitive difficulty and linguistic difficulty on student performance. To complement the quantitative portion of the study, a think-aloud protocol was conducted with 13 students in an attempt to gain insight into the problems experienced by individual students in reading, understanding and answering multiple-choice questions. The consolidated quantitative and qualitative findings indicate that among the linguistic aspects of questions that contributed to question difficulty for second language speakers was a high density of academic words, long items and negative stems. These sources of difficulty should be addressed as far as possible during item-writing and editorial review of questions.

In this study, a comparison was made between the various levels of group cognitive complexity and its relationship to task performance and task satisfaction. The goal of this research is to answer the general question, "Should decision-making groups consist of individuals who are similar in the way they differentiate and/or integrate various stimuli in order to increase performance and satisfaction?" The preceding research problem was analyzed in a laboratory setting using a 2 X 2 factorial design blocked on the variable, cognitive complexity. The Repertory Grid was used to measure the cognitive complexity of 228 student subjects. These subjects were stratified into groups of three based on their cognitive complexity score on the Repertory Grid (Kelly, 1955). Each group was treated randomly with one of two levels of task complexity (complex or not complex). Moreover, the groups received an imposedgroup structure that incorporated centralized or decentralized decision-making. Results indicated that groups consisting of cognitively complex members outperformed groups consisting of noncomplex members. No support was obtained for the two-way interaction between group cognitive complexity and either task complexity or group structure. Support was obtained for the interaction between task complexity and group structure on both task satisfaction measures. The highest satisfaction levels occurred with a complex task in a decentralized structure. In addition, the three-way interaction effect on the task satisfaction scale between group structure, task complexity, and group cognitive complexity was significant. The means, however, were not in the predicted direction. For cognitively simple groups, a complex task with a decentralized structure lead to the highest task satisfaction level; whereas, a less complex task with a decentralized group structure lead to the lowest task satisfaction score for noncomplex members. There were no significant differences for cognitively complex groups when analyzing the three-way interaction between group cognitive complexity, task complexity, and group structure.

Conduct an exploratory study on operationally focussed managers within a South African company, using both quantitative and qualitative analysis, to determine the correlation between the required and actual levels of complexity and time-span of control at specific hierarchical levels / Organisational design needs to be in line with capabilities of the individual-inrole. The structure of an organization directly impacts the overall effectiveness and ultimately the success of such an organization and the number of layers required in any given hierarchy is a product of the organization’s mission (Jaques, 1989). Stratified Systems Theory (Jaques, 1989) defines work in seven strata based on a basis of decision-making complexity. The research presented here identifies a specific organisation’s current level of work based on complexity and the time-span of decision-making. Research was done in one specific geographical region of a company operating in the Supply Chain and Logistics industry in South Africa. Qualitative data collection was done by means of interviews with a defined sample group that provided an adequate cross-section of the main functions of the business, however, the sampling technique used may not provide results representative of the entire population. ii The Brunel Institute for Organisation and Social Studies’ (BIOSS) Matrix of Working Relationships was used as main basis for reporting results. The research indicates that the organisation is presently, according to Jaques Stratified Systems Theory (Jaques, 1989), operating at one level below their intended level that will allow them to effectively meet their longterm strategic objectives. The report identifies shortcomings in terms of the current capabilities of the individual-in-role and the actual work requirements, setting a foundation for further analysis of individual capabilities for effective organisational design. / Graduate School of Business Leadership / MBL

It has been long known that groups of adults learn and enact their learning in certain ways; what is little known is how groups learn and how they develop in cognitive complexity. This dissertation proposes a theory of group cognitive development by arguing that intentional adult groups are complex and dynamic, and that they have the potential to evolve over time. Groups are complex in that they are made up of individuals within different orders of consciousness (Kegan), and they are dynamic in that different orders of consciousness interact and conflict (Lonergan) during the formation and enactment of group vision, values, and procedures. Dynamic complexity theory of group development as it is referred to in this study is grounded in Robert Kegan’s constructive developmental theory and in Bernard Lonergan’s transcendental method. While both Kegan and Lonergan attend to the growth of individuals, their theories are adapted to groups in order to understand the cognitive complexity of groups, intragroup and intergroup conflict, and the mental complexity of leader curriculum. This theory is applied to two case studies, one from antiquity in the case of the first century Corinthian community engaged in conflict with its founder, St. Paul, and in one contemporary study of American Catholic parishioners engaged in contentious dialogue with diocesan leaders from 1994 to 2004. The parish groups experienced a series of dialogues during a ten year period over the issues of parish restructuring and the priest sexual abuse crisis yielding cumulative and progressive changes in perspective-taking, responsibility-taking, and in group capacity to respond to and engage local and institutional authority figures. Group development is observed against a pedagogical backdrop that represents a mismatch between group complexity and leader expectations. In Corinth, Paul’s curriculum was significantly beyond the mental capacity of the community. In the case of Catholic parishioners the curriculum of diocesan leaders was beneath the mental capacities of most of the groups studied. It is proposed that individuals sharing the same order of consciousness, understood as cognitive constituencies, are in a dynamic relationship with other cognitive constituencies in the group that interact within an object-subject dialectic and an agency-communion dialectic. The first describes and explains the evolving cognitive complexity of group knowing, how the group does its knowing, and what it knows when it is doing it (the epistemologies of the group). This dialectic has implications for how intentional groups might be the critical factor for understanding individual growth. The second dialectic describes and explains the changing relationship between group agency, which is enacted either instrumentally or ideologically; and group communion, which is enacted ideationally. The agency-communion dialectic is held in an unstable balance in the knowing, identity, and mission of groups. With implications for the fields of adult education and learning organizations, dynamic complexity theory of group development notes predictable stages of group evolution as each cognitive constituency evolves, and notes the significance of internal and external conflict for exposing the presence of different ways of knowing and for challenging the group toward cognitive growth. / Thesis (PhD) — Boston College, 2008. / Submitted to: Boston College. Graduate School of Arts and Sciences. / Discipline: Religious Education and Pastoral Ministry.

A small group of profit seeking publishers dominates the American textbook market and guides the learning of the majority of our nation’s calculus students. The College Board’s AP Calculus curriculum is a de facto national standard for this gateway course that is critically important to 21st century STEM careers. A multi-representational understanding of calculus is a central pillar of the AP curriculum. This dissertation asks whether this multi-representational vision is manifest in popular calculus textbooks. This dissertation began with a survey of all AP Calculus AB Examination free response items, 2002-2011, and found that students score worse on items characterized by numerical anchors or verbal targets. Based on previously elucidated models, a new cognitive model of five levels and six principles is developed for the purpose of calculus textbook task analysis. This model explicates complexity as a function of representational input and output. Eight popular secondary calculus textbooks were selected for study based on Amazon sales rank data. All verbally anchored mathematical tasks (n=555) from sections of those books concerning the mean value theorem and all AP Calculus AB prompts (n=226) were analyzed for cognitive complexity and representational diversity using the model. The textbook study found that calculus textbooks underrepresented the numerical anchor and verbal target. It found that the textbooks were both explicitly and implicitly less cognitively complex than the AP test. The article suggested that textbook tasks should be less dense, avoid cognitive attenuation, move away from the stand-alone item, juxtapose anchor representations, scaffold student solutions, incorporate previously considered overarching concepts and include more profound follow-up questions. To date there have been no studies of calculus textbook content based on established research on cognitive learning. Given the critical role that their calculus course plays in the lives of hundreds of thousands of students annually, it is incumbent upon the College Board to establish a textbook review process at the very least in the same vain as the teacher syllabus auditing process established in recent years.

For many courses, mathematics included, there is an associated interactive e-learning system that provides assessment and tutoring. Some of these systems are classified as Intelligent Tutoring Systems. MyMathLab, Mathzone, and Assessment of LEarning in Knowledge Space (ALEKS) are just a few of the interactive e-learning systems in mathematics. In ALEKS, assessment and tutoring are based on the Knowledge Space Theory. Previous studies in a traditional learning environment have shown ALEKS users to perform equally or better in mathematics achievement than the group who did not use ALEKS. The purpose of this research was to investigate the effect of ALEKS on students’ achievement in mathematics in an online learning environment and to determine the cognitive complexity of mathematical tasks enacted by ALEKS’s initial (pretest) and final (posttest) assessments. The targeted population for this study was undergraduate students in College Mathematics I, in an online course at a private university in the southwestern United States. The study used a quasi-experimental One-Group non-randomized pretest and posttest design. Five methods of analysis and one model were used in analyzing data: t-test, correctional analysis, simple and multiple regression analysis, Cronbach’s Alpha reliability test and Webb’s depth of knowledge model. A t-test showed a difference between the pretest and posttest reports, meaning ALEKS had a significant effect on students’ mathematics achievement. The correlation analysis showed a significant positive linear relationship between the concept mastery reports and the formative and summative assessments reports meaning there is a direct relationship between the ALEKS concept mastery and the assessments. The regression equation showed a better model for predicting mathematics achievement with ALEKS when the time spent learning in ALEKS and the concept mastery scores are used as part of the model. According to Webb’s depth of knowledge model, the cognitive complexity of the pretest and posttest question items used by ALEKS were as follows: 50.5% required application of skills and concepts, 37.1% required recall of information, and 12.4% required strategic thinking: None of the questions items required extended thinking or complex reasoning, implying ALEKS is appropriate for skills and concepts building at this level of mathematics.

xvi, 116 p. A print copy of this thesis is available through the UO Libraries. Search the library catalog for the location and call number. / A significant challenge in developing alternate assessments is obtaining suitable sample sizes. This study investigated whether psychometric characteristics of mathematic alternate assessment items created for 2% students in grade 8 can be meaningfully estimated with data obtained from general education students in lower grades. Participants included 23 2% students in grade 8 and 235 general education students in grades 6-8. Twenty three 2% students were identified through the Student Performance Test (10 standard items and 10 2% items) and the Teacher Perception Survey. Performance on 10 2% items by the 2% students and the general education students were analyzed to address the questions: (a) are there grade levels at which the item parameters estimated from general education students in grade 6-8 are not different from those obtained using the 2% student sample? and (b) are there grade levels at which the estimated ability of general education students in grades 6-8 are not different the 2% student sample in grade 8? Results indicated that the item response patterns of 2% students in grade 8 were comparable to those of general education students in grades 6 and 7. Additionally, 2% students in grade 8 showed comparable mathematics performance on 2% items when compared to general education students in grades 6 and 7. Considering the content exposure of students in lower grades, this study concluded that data from general education students in grade 7 would be more appropriate to be used in designing alternate assessment for 2% students in grade 8 than data from students in grade 6. The general conclusion is that using data obtained from general education students in lower grade levels may be an appropriate and efficient method of designing alternate assessment items. / Advisers: Dr. Beth Ham, Co-Chair; Dr. Paul Yovanoff, Co-Chair

Doctor of Philosophy / Educational Leadership / Sarah Jane Fishback / The military devotes significant resources and time in the development of officers through education. Recently, there has been a great deal of emphasis placed on military Intermediate Service Schools (ISS’s) to enhance the ability of graduates to think with greater cognitive complexity in order to solve the kinds of problems they may face after graduation. The military environment in which these mid-career officer students will serve is highly complex and requires a significant ability to generate solutions to unique and complex problems. One hallmark of a developmental adult educational experience is the advancement of the student to higher levels of cognitive complexity. The purpose of this research was to determine if there was a relationship between the cognitive complexity of faculty, students, and expectations for student graduates, at a military Intermediate Service School. Along with the simultaneous measure of cognitive complexity, via a survey administration of the LEP instrument, the researcher also developed a technique for translating learning objectives from Blooms taxonomy into a corresponding Perry position. This translation method was used to translate the college learning objectives into an expected Perry position for graduates of the college. The study also included demographic data to look for significant results regarding a number of independent variables. For faculty only these included teaching department, years of teaching experience, age, and military status. For both populations the variables studied included education level, gender, combat experience and combat trauma, branch of service, commissioning source, and years of active duty service. The study found that the mean cognitive complexity of entering students (CCI = 360) was lower than the cognitive complexity required of graduates (CCI = 407). However, the faculty mean cognitive complexity (CCI = 398) was not significantly different from a student graduate. The faculty results indicated that there were no statistically significant relations between the independent variables studied and the measured cognitive complexity. For students there was a statistically significant relation between measured cognitive complexity and gender.