The validity of three key Business Excellence (BE) models used in the Asia Pacific—the Australian Business Excellence Framework (Australia), the Baldrige Criteria for Performance Excellence (New Zealand) and the Singapore Quality Award Criteria (Singapore)—was examined from a conceptual as well as a predictive standpoint. Unlike in many past studies, in this study the validity of the measurement criteria stipulated in BE models have been directly assessed. The conceptual validity of the three BE models was studied through a generic theoretical model using the partial least squares-based structural equation modelling (PLSBSEM) method. Apart from measurement validity, the strengths of the hypothesised causal relationships between the constructs of the BE models and their practical implementations were also examined under conceptual validity. The predictive validity of the three BE models was examined through linear predictive models involving enablers—being measures in BE models that cover what organisations actually do in order to achieve business outcomes—as predictors and business outcomes as responses. Alongside predictive validity, the reasonableness of the stipulated weights of the enablers was also examined. Other empirical and pragmatic inquiries covered in this study included: (a) a study of the effect of “industry attractiveness” on financial and market performance, and (b) a study of the relationship between BE constructs and “national cultural dimensions”. Results revealed that although the three BE models fulfilled the basic requirements of measurement validity, against more stringent criteria such as those used in psychometrics, they showed low levels of validity. The possible reasons for this were examined and the ways of overcoming the shortcoming were suggested. The generic theoretical model was found to be statistically significant across all three settings: Australia, New Zealand and Singapore. As regards predictive validity, it was observed that enablers appear to be good predictors of business outcomes (thus establishing predictive validity) although there was scope for improvement of the existing weighting scheme of the enablers. This study is important because many organisations in the region use BE models with the expectation of improving their performance in key results areas and hence there is a need to demonstrate that the BE models are based on sound concepts.

Differential item functioning (DIF) and differential distractor functioning (DDF) are methods used to screen for item bias (Camilli && Shepard, 1994; Penfield, 2008). Using an applied empirical example, this mixed-methods study examined the congruency and relationship of DIF and DDF methods in screening multiple-choice items. Data for Study I were drawn from item responses of 271 female and 236 male low-income children on a preschool science assessment. Item analyses employed a common statistical approach of the Mantel-Haenszel log-odds ratio (MH-LOR) to detect DIF in dichotomously scored items (Holland & Thayer, 1988), and extended the approach to identify DDF (Penfield, 2008). Findings demonstrated that the using MH-LOR to detect DIF and DDF supported the theoretical relationship that the magnitude and form of DIF and are dependent on the DDF effects, and demonstrated the advantages of studying DIF and DDF in multiple-choice items. A total of 4 items with DIF and DDF and 5 items with only DDF were detected. Study II incorporated an item content review, an important but often overlooked and under-published step of DIF and DDF studies (Camilli & Shepard). Interviews with 25 female and 22 male low-income preschool children and an expert review helped to interpret the DIF and DDF results and their comparison, and determined that a content review process of studied items can reveal reasons for potential item bias that are often congruent with the statistical results. Patterns emerged and are discussed in detail. The quantitative and qualitative analyses were conducted in an applied framework of examining the validity of the preschool science assessment scores for evaluating science programs serving low-income children, however, the techniques can be generalized for use with measures across various disciplines of research.

Die Cognitive Load Theory (CLT) postuliert, dass das Lernen neuer, komplexer Informationen mit einer Belastung des Arbeitsgedächtnisses des Lernenden einhergeht. Nachdem die Theorie erstmals 1988 von John Sweller postuliert wurde, arbeitete eine kleine Gruppe von Forschern an Universitäten in Australien und den Niederlanden an ihrer Weiterentwicklung. Sie entwickelte sich schnell zu einer der populärsten Theorien in der pädagogischen Psychologie und im Instruktionsdesign. Die CLT basiert auf Erkenntnissen der Kognitionspsychologie und geht davon aus, dass das Arbeitsgedächtnis nur eine begrenzte Anzahl von Informationselementen simultan verarbeiten kann, wodurch die Fähigkeit, neue Informationen zu verarbeiten, stark eingeschränkt ist. Auf der Grundlage der CLT wurden eine Reihe von Empfehlungen und Prinzipien abgeleitet, die den Lernenden helfen sollen, ihre verfügbaren kognitiven Ressourcen optimal für das Lernen zu nutzen, um Informationen in Form von Schemata in das Langzeitgedächtnis zu übertragen. Ziel ist es, eine Überlastung des Arbeitsgedächtnisses (d.h. eine kognitive Überlastung) zu vermeiden, da diese das Lernen behindern. Nach der Theorie sind drei Arten von kognitiver Belastung während des Lernens relevant. Während der Intrinsic Cognitive Load (ICL) die Komplexität der zu lernenden Informationen unter Berücksichtigung des Vorwissens des Lernenden beschreibt, wird der Extraneous Cognitive Load (ECL) durch die Gestaltung des Lernmaterials bestimmt, welcher sich negativ auf den Lernerfolg auswirken kann. Während ICL und ECL konzeptionell klar definiert sind, wird die Definition der dritten Belastungsart, des Germane Cognitive Load (GCL), innerhalb der CLT kontrovers diskutiert. In diesem Zusammenhang ist unklar, ob der GCL eine eigenständige Quelle kognitiver Belastung darstellt oder ob er sich lediglich auf die kognitiven Ressourcen bezieht, die der Lernende mental aufwendet, um Schemata aufzubauen. Daraus hat sich in den letzten Jahren eine Debatte darüber entwickelt, ob der GCL überhaupt ein Teil der CLT ist und ob es ausreicht, die kognitive Belastung während des Lernens mit dem ICL und dem ECL zu beschreiben. Auch die Annahme, dass die drei Arten unabhängig voneinander additiv zur Gesamtbelastung beitragen, wurde häufig in Frage gestellt. Neben den offenen theoretischen Fragen innerhalb der CLT stellt die Messung der verschiedenen Belastungsarten die größte Herausforderung für die Forschung dar. Latente Konstrukte wie die kognitive Belastung können nur indirekt gemessen werden, wodurch es umso wichtiger ist, die Reliabilität und Validität entsprechender Messungen zu überprüfen. Von besonderer Bedeutung ist dabei die multidimensionale Messung (d.h. die separate Messung der einzelnen Arten der kognitiven Belastung) sowie der optimale Zeitpunkt der Messung (d.h. während oder nach dem Lernen). Ziel dieser Dissertation ist es daher, die Chancen und Herausforderungen der Messung kognitiver Belastung zu diskutieren. Da die kognitive Belastung in der Praxis häufig mit Fragebögen gemessen wird, wird diese Erhebungsmethode in der vorliegenden Dissertation vorrangig betrachtet. So wurde in einer Metaanalyse überprüft, inwieweit die zu diesem Zeitpunkt verfügbaren Fragebögen in der Lage sind, die verschiedenen Arten kognitiver Belastung reliabel und valide zu messen. Dazu wurden experimentelle Studien aus dem Bereich des multimedialen Lernens herangezogen, in denen die CLT eine ganz wesentliche Rolle spielt. Darüber hinaus wurde aufgrund konzeptueller und psychometrischer Inkonsistenzen der verfügbaren Fragebögen ein neuer Fragebogen entwickelt und validiert, der vor allem den Lernenden helfen soll, besser zwischen den verschiedenen Arten der kognitiven Belastung zu unterscheiden. In einer experimentellen Studie wurde zudem untersucht, wie sich unterschiedliche Komplexitätsgrade innerhalb eines Lernmaterials auf die Gesamtbewertung der kognitiven Belastung durch den Lernenden auswirken. Damit verbunden war die Frage, ob sich Lernende einen Ankerpunkt (z.B. den ersten Eindruck) suchen, von dem aus sie das gesamte Lernmaterial hinsichtlich der kognitiven Belastung bewerten. Insgesamt zeigte sich, dass das Konstrukt der kognitiven Belastung mit hoher interner Konsistenz gemessen werden kann, wobei sich Einschränkungen hinsichtlich der Validität ergaben. So deuten die Korrelationen zwischen den einzelnen Arten der kognitiven Belastung darauf hin, dass die Lernenden nicht ausreichend zwischen ihnen differenzieren können. Dies gilt auch dann, wenn die Items im Fragebogen so formuliert sind, dass die Lernenden erkennen sollten, welche Art von kognitiver Belastung sie beurteilen sollen. Darüber hinaus spiegeln sich die theoretischen Annahmen der CLT nicht immer in den Messergebnissen wider. Es ist unklar, ob diese Ambivalenz auf theoretische Unzulänglichkeiten oder auf Messungenauigkeiten zurückzuführen ist. Forschende sollten ihre Messergebnisse vor diesem Hintergrund kritisch reflektieren. Da es sich bei der kognitiven Belastung um ein dynamisches Konstrukt handelt, dessen Intensität während des Lernprozesses stark variieren kann, werden die Forschenden dazu angehalten, die Messung zu geeigneten Zeitpunkten während des Lernprozesses durchzuführen. Basierend auf den Erkenntnissen dieser Dissertation werden Handlungsempfehlungen für den zukünftigen Einsatz von Fragebögen zur Messung kognitiver Belastung abgeleitet und theoretische Implikationen diskutiert.:Zusammenfassung 3 Summary 5 1 Introduction 7 2 Foundations of Cognitive Load Theory 9 2.1 Human Cognitive Architecture 9 2.2 Evolutionary Educational Psychology 12 2.3 The Construct of Cognitive Load 14 2.4 Types of Cognitive Load 16 3 Cognitive Load Theory in Multimedia Learning 22 4 Measuring Cognitive Load 25 4.1 Physiological Measures 25 4.2 Dual-task Approaches 26 4.3 Self-rating Scales 27 5 Conceptual and Psychometric Issues in Cognitive Load Research 31 5.1 Three-factor Model and Additivity Hypothesis 31 5.2 Reliable and Valid Measurement of Cognitive Load with Subjective Scales 33 5.3 Timing of Cognitive Load Measurement 36 6 Article 1: A systematic meta-analysis of the reliability and validity of subjective cognitive load questionnaires in experimental multimedia learning research 40 7 Article 2: Development and validation of a theory-based questionnaire to measure different types of cognitive load 98 8 Article 3: The distorting role of primacy effects when reporting cognitive load in a learning material with varying complexities 136 9 Discussion 193 9.1 Contributions of this Thesis 193 9.2 Limitations and Future Directions 196 9.3 Conclusion 197 10 References 198 Danksagung 208 / Cognitive Load Theory (CLT) postulates that learning new, complex information places a burden on the learner’s working memory. After the theory was first postulated by John Sweller in 1988, a small group of researchers at universities in Australia and the Netherlands worked to develop it further. It quickly became one of the most popular theories in educational psychology and instructional design. CLT is based on findings in cognitive psychology and posits that working memory can only process a limited number of information elements simultaneously, severely limiting its ability to process new information. Based on CLT, a set of recommendations and principles has been derived to help learners make optimal use of their available cognitive resources for learning in order to transfer information to long-term memory in the form of schemata. The goal is to avoid overloading working memory (i.e., cognitive overload), which impedes learning. According to the theory, three types of cognitive load are relevant during learning. While intrinsic cognitive load (ICL) describes the complexity of the information to be learned, taking into account the learner's prior knowledge, extraneous cognitive load (ECL) is determined by the design of the learning material and can have a negative impact on learning success. While ICL and ECL are conceptually well-defined, the definition of the third type of load, germane cognitive load (GCL), is controversial within CLT. In this context, it is unclear whether GCL is a source of cognitive load in its own right or whether it simply refers to the cognitive resources that the learner mentally expends to build schemata. This has led to a debate in recent years as to whether GCL is a part of CLT at all, and whether it is sufficient to use the ICL and ECL to describe cognitive load during learning. The assumption that the three types additively and independently contribute to the total cognitive load has also been questioned. In addition to the open theoretical questions within CLT, the measurement of the different types of cognitive load represents the greatest challenge for research. Latent constructs such as cognitive load can only be measured indirectly, making it all the more important to examine the reliability and validity of such measures. Of particular importance is multidimensional measurement (i.e., measuring each type of cognitive load separately) and the optimal timing of measurement (i.e., during or after learning). Thus, the purpose of this thesis is to discuss the opportunities and challenges of measuring cognitive load. Since cognitive load is often measured in practice with questionnaires, this thesis prioritizes this survey method. Therefore, a meta-analysis was conducted in order to examine the extent to which questionnaires available at this time are able to reliably and validly measure the different types of cognitive load. For this purpose, experimental studies from the field of multimedia learning, in which CLT plays a very important role, were used. In addition, due to conceptual and psychometric inconsistencies of available questionnaires, a new questionnaire was developed and validated, mainly to help learners better differentiate between different types of cognitive load. An experimental study was also conducted to test how different levels of complexity within a learning material affect the learner’s overall assessment of cognitive load. Related to this was the question of whether learners seek an anchor point (e.g., first impression) from which they evaluate the entire learning material in terms of cognitive load. Overall, it was found that the construct of cognitive load can be measured with a high degree of internal consistency, although there are limitations in terms of validity. For example, the correlations between the different types of cognitive load indicate that the learners cannot sufficiently differentiate between them. This is true even though the items in the questionnaire are worded in such a way that learners should be able to identify which type of cognitive load they are being asked to assess. Furthermore, the theoretical assumptions of CLT are not always reflected in the measurement results. It is unclear whether this ambivalence is due to theoretical shortcomings or measurement inaccuracies. In light of this, researchers should critically reflect on their measurement results. Because cognitive load is a dynamic construct whose intensity can vary widely during learning, researchers are encouraged to measure it at appropriate times during the learning process. Based on the findings of this thesis, recommendations for the future use of questionnaires to measure cognitive load are derived and theoretical implications are discussed.:Zusammenfassung 3 Summary 5 1 Introduction 7 2 Foundations of Cognitive Load Theory 9 2.1 Human Cognitive Architecture 9 2.2 Evolutionary Educational Psychology 12 2.3 The Construct of Cognitive Load 14 2.4 Types of Cognitive Load 16 3 Cognitive Load Theory in Multimedia Learning 22 4 Measuring Cognitive Load 25 4.1 Physiological Measures 25 4.2 Dual-task Approaches 26 4.3 Self-rating Scales 27 5 Conceptual and Psychometric Issues in Cognitive Load Research 31 5.1 Three-factor Model and Additivity Hypothesis 31 5.2 Reliable and Valid Measurement of Cognitive Load with Subjective Scales 33 5.3 Timing of Cognitive Load Measurement 36 6 Article 1: A systematic meta-analysis of the reliability and validity of subjective cognitive load questionnaires in experimental multimedia learning research 40 7 Article 2: Development and validation of a theory-based questionnaire to measure different types of cognitive load 98 8 Article 3: The distorting role of primacy effects when reporting cognitive load in a learning material with varying complexities 136 9 Discussion 193 9.1 Contributions of this Thesis 193 9.2 Limitations and Future Directions 196 9.3 Conclusion 197 10 References 198 Danksagung 208

No description available.

Non-monetary deprivation indicators are now widely used for studying and measuring poverty in Europe. However, despite their prevalence, the empirical performance of existing deprivation scales has rarely been examined. This thesis i) identifies possible conceptual problems of existing deprivation scales such as indexing, missing dimensions and threshold; ii) empirically assesses the extent of possible error in measurement related to these conceptual problems; and iii) offer an alternative way for constructing deprivation measures to mitigate the identified conceptual problems. The thesis consists of four stand-alone papers, accompanied by an overarching introduction and conclusion. The first three papers provide empirical evidence on the empirical consequences of the missing dimensions and threshold problems for the measurement and analysis of poverty, while the fourth paper exemplifies a concept-led multidimensional design that can reduce the error introduced by these conceptual problems. The analysis is generally held for 25 EU countries using European Survey of Income and Living Conditions (EU-SILC); only in the second paper, the analysis is done for the UK using British Household Panel Survey (BHPS).