The overall aim of this thesis was to examine the construct validity and the reliability of the instrument DOA; Dialogue about working ability. The theoretical foundation for DOA is the Model of Human Occupation. The instrument is divided into two sections; client self-assessment and professional assessment focusing on the individual’s working ability. The assessments are followed by a dialogue to distinguish goals for the return to-work process based on the client’s own preferences. In line with the theoretical base, the concept ‘work’ refers to both paid work and other productive activities related to work. In Study I, 21 professionals and 126 clients participated from three clinics in Sweden. All clients were involved as outpatients in psychiatric work rehabilitation in county councils or community services. The results from the Rasch analysis test indicated that items are well separated from each other and generally work together in five unidimensional continuums with the objective to measure abilities related to work. Twenty-nine of 34 items showed goodness-of-fit statistics, which means acceptable infit MnSq values from >0.6 to <1.4 in association with Z values from –2 to 2, and that the five misfit items have to be revised. Thirty-four outpatient clients and 14 professionals from four clinics within psychiatric work rehabilitation in Sweden participated in Study II. Two reliability aspects were studied: test-retest and inter rater reliability. The statistical analyses used were Spearman Rank Correlation test and Percentage of agreement (PA). Generally, the Spearman Rank Correlation test showed acceptable significant correlations between test retest since a greater part of items correlated “moderate to good” (r = 0.51 – 0.75). In the inter raters´ test PA was good, ranging from 93.1 % to 96.5 %, which is well above the acceptable criteria of 80 %. There were no significance differences between the raters. The study has shown that one item has to revised. The findings verify that DOA has potential as a means of making valid and reliable assessments of working ability, as described by the Model of Human Occupation, out of both clients´ and occupational therapists´ perspective.

<p>The overall aim of this thesis was to examine the construct validity and the reliability of the instrument DOA; Dialogue about working ability. The theoretical foundation for DOA is the Model of Human Occupation. The instrument is divided into two sections; client self-assessment and professional assessment focusing on the individual’s working ability. The assessments are followed by a dialogue to distinguish goals for the return to-work process based on the client’s own preferences. In line with the theoretical base, the concept ‘work’ refers to both paid work and other productive activities related to work.</p><p>In Study I, 21 professionals and 126 clients participated from three clinics in Sweden. All clients were involved as outpatients in psychiatric work rehabilitation in county councils or community services. The results from the Rasch analysis test indicated that items are well separated from each other and generally work together in five unidimensional continuums with the objective to measure abilities related to work. Twenty-nine of 34 items showed goodness-of-fit statistics, which means acceptable infit MnSq values from >0.6 to <1.4 in association with Z values from –2 to 2, and that the five misfit items have to be revised.</p><p>Thirty-four outpatient clients and 14 professionals from four clinics within psychiatric work rehabilitation in Sweden participated in Study II. Two reliability aspects were studied: test-retest and inter rater reliability. The statistical analyses used were Spearman Rank Correlation test and Percentage of agreement (PA). Generally, the Spearman Rank Correlation test showed acceptable significant correlations between test retest since a greater part of items correlated “moderate to good” (r = 0.51 – 0.75). In the inter raters´ test PA was good, ranging from 93.1 % to 96.5 %, which is well above the acceptable criteria of 80 %. There were no significance differences between the raters. The study has shown that one item has to revised.</p><p>The findings verify that DOA has potential as a means of making valid and reliable assessments of working ability, as described by the Model of Human Occupation, out of both clients´ and occupational therapists´ perspective.</p>

abstract: The existing minima for sample size and test length recommendations for DIMTEST (750 examinees and 25 items) are tied to features of the procedure that are no longer in use. The current version of DIMTEST uses a bootstrapping procedure to remove bias from the test statistic and is packaged with a conditional covariance-based procedure called ATFIND for partitioning test items. Key factors such as sample size, test length, test structure, the correlation between dimensions, and strength of dependence were manipulated in a Monte Carlo study to assess the effectiveness of the current version of DIMTEST with fewer examinees and items. In addition, the DETECT program was also used to partition test items; a second feature of this study also compared the structure of test partitions obtained with ATFIND and DETECT in a number of ways. With some exceptions, the performance of DIMTEST was quite conservative in unidimensional conditions. The performance of DIMTEST in multidimensional conditions depended on each of the manipulated factors, and did suggest that the minima of sample size and test length can be made lower for some conditions. In terms of partitioning test items in unidimensional conditions, DETECT tended to produce longer assessment subtests than ATFIND in turn yielding different test partitions. In multidimensional conditions, test partitions became more similar and were more accurate with increased sample size, for factorially simple data, greater strength of dependence, and a decreased correlation between dimensions. Recommendations for sample size and test length minima are provided along with suggestions for future research. / Dissertation/Thesis / M.A. Educational Psychology 2012

There are several generally accepted rules for detecting unidimensionality, but none are well tested. This simulation study investigated well-known methods, including but not limited to, the Kaiser (k>1) Criterion, Percentage of Measure Validity (greater than 50%, 40%, or 20%), Ratio of Eigenvalues, and Kelley method, and compares these methods to each other and a new method proposed by the author (McGill method) for assessing unidimensionality. After applying principal component analysis (PCA) to the residuals of a Latent Trait Test Theory (LTTT) model, this study was able to address three purposes: determining the Type I error rates associated with various criterion values, for assessing unidimensionality; determining the Type II error rates and statistical power associated with various rules of thumb when assessing dimensionality; and, finally, determining whether more suitable criterion values could be established for the methods of the study by accounting for various characteristics of the measurement context. For those methods based on criterion values, new modified values are proposed. For those methods without criterion values for dimensionality decisions, criterion values are modeled and presented. The methods compared in this study were investigated using PCA on residuals from the Rasch model. The sample size, test length, ability distribution variability, and item distribution variability were varied and the resulting Type I and Type II error rates of each method were examined. The results imply that certain conditions can cause improper diagnoses as to the dimensionality of instruments. Adjusted methods are suggested to induce a more stable condition relative to the Type I and Type II error rates. The nearly ubiquitous Kaiser method was found to be biased towards signaling multidimensionality whether it exists or not. The modified version of the Kaiser method and the McGill method, proposed by the author were shown to be among the best at detecting unidimensionality when it was present. In short, methods that take into account changes in variables such as sample size, test length, item variability, and person variability are better than methods that use a single, static criterionvalue in decision making with respect to dimensionality. / Ph. D.

This article explains what we mean by the unidimensionality of a measurement instrument, therefore we present some definitions and  theoretical contributions about this subject. Factor analysis is proposed as one of the many methods for assessing the unidimensionality of a measurement instrument. The use of Pearson correlations matrices on item-level factor analysis is identified as an important problem. Those correlations are problematic because items didn’t carry out the necessary assumptions in order to apply the Pearson correlation: interval-level measurement and normal distribution of the variable. As an alternative we propose and exemplify the use of tetrachoric and polychoric correlations. / Este artículo explica qué es lo que implica la unidimensionalidad de un instrumento de medición. Para ello se presentan algunas definiciones y aportes teóricos sobre el tema. Luego, el análisis factorial es propuesto como uno de los métodos para evaluar la dimensionalidad de un instrumento de medición. Se señala como un problema importante el uso de las matrices de correlaciones de Pearson en los análisis factoriales a nivel de ítems. Estas correlaciones son problemáticas porque los ítems no cumplen con los supuestos necesarios para aplicar la correlación de Pearson: nivel de medición de intervalo y distribución normal de la variable. Como alternativa se postula y ejemplifica el uso de las correlaciones tetracóricas y policóricas.

Eindimensionalität ist ein wichtiges Merkmal psychometrischer Tests und wird in der Psychologie momentan vor allem unter Anwendung der Faktorenanalyse geprüft. Da diese aber nicht das zu leisten scheint, was von ihr erwartet wird (z. B. Titz, 2022; Ziegler & Hagemann, 2015), wurde in der vorliegenden Arbeit eine Vorgehensweise mit der State-Trace-Analyse (Bamber, 1979) angewendet, um der Frage nach Eindimensionalität nachzugehen. Dazu wurden vier Persönlichkeitsfragebögen, die auf faktorenanalytischen Verfahren basieren und aus einem Datensatz der AIID-Studie stammen (Hussey & Hughes, 2020; Hussey et al., 2012), im Vorfeld der Arbeit State-Trace-Analysen unterzogen. Items, die die Bedingung der Eindimensionalität verletzten, wurden in der vorliegenden Arbeit mit Hilfe von Diagrammen näher untersucht und interpretiert. Es hat sich gezeigt, dass die State-Trace-Analyse Items identifizieren konnte, die die Eindimensionalität der Fragebögen infrage stellen und laut Faktorenanalyse unentdeckt blieben. Inhaltliche Interpretationen konnten unterstützen, dass die Items wahrscheinlich problematisch für die Eindimensionalität der untersuchten Verfahren sind und überarbeitet werden müssen. Die State-Trace-Analyse bietet hohes Potenzial für psychometrische Fragestellungen und muss in Zukunft mehr Aufmerksamkeit erhalten.