Diante da indisponibilidade de um instrumento específico de avaliação da qualidade de vida (QV) de pessoas com qualquer tipo de ferida, em nosso meio, o objetivo deste estudo foi construir e validar a versão feridas do Índice de Qualidade de Vida de Ferrans e Powers (IQVFP-VF). Inicialmente, o estudo recebeu autorização das autoras do instrumento original, aprovação pelo Comitê de Ética da EEUSP e autorizações dos campos para a coleta de dados. Classificado como do tipo metodológico, o estudo foi desenvolvido segundo os procedimentos teóricos, empíricos e analíticos, propostos por Pasquali, Os procedimentos teóricos incluíram a definição e análise dos itens por meio de revisão de literatura, avaliações de juizes (especialistas e pacientes) e pré-teste; os empíricos foram realizados por meio da aplicação do instrumento, resultante da etapa anterior, para análise de suas propriedades psicométricas, em amostras de teste (n=362), re-teste (n = 63) e validade convergente (n=179); e os analíticos, por meio das estratégias estatísticas destinadas às análises das propriedades psicométricas do instrumento final dos procedimentos teóricos. A confiabilidade foi obtida por meio da avaliação da consistência interna - CI (alfa de Cronbach 0,70) e da estabilidade do instrumento - duas semanas pós-teste (coeficiente de correlação intraclasse 0,70). Avaliaram-se a validade de conteúdo, pela concordância entre juizes ( 80%); a de critério concorrente, pela correlação entre os escores do item sua satisfação e dos domínios e instrumento total (coeficiente de correlação de Pearson ou Spearman 0,30); a validade de construto convergente, correlacionando-se os domínios e instrumento total do IQVFP-VF e WHOQOL-bref (esperando-se correlações moderadas a fortes 0,30 entre os domínios similares e < 0,30 entre os diferentes) e a validade de construto discriminante, comparando-se os escores dos domínios e instrumento total quanto a idade, número e duração da ferida e intensidade de dor, no momento e na última semana (Testes de Mann-Whitney, Kruskal-Wallis, t-Student e ANOVA). A análise fatorial confirmatória (AFC) foi testada pelo grau de correlação entre os itens e os domínios originais (saúde/funcionamento-SF; sócio-econômico-SE; psicológico e espiritual-PE e família-Fa) e medidas de ajuste de modelo (LISREL). A amostra foi composta de pacientes ambulatoriais com feridas, pertencentes a dezesseis instituições, que se enquadraram em critérios pré-estabelecidos. Os resultados mostram coeficientes alfa de Cronbach 0,90; 0,88; 0,65; 0,81; 0,55 para o instrumento total e domínios SF, SE, PE, Fa respectivamente, na avaliação da CI; para a estabilidade, encontraram-se correlações 0,83 (p < 0,000); para a validade concorrente, correlações de 0,28 a 0,69; para a validade convergente, correlações de 0,17 a 0, 60 e para a validade discriminante, correlações significativas entre: QV geral e número de feridas (p=0,047); PE e duração da ferida (p=0,017); SF (p= 0,043), SE (p= 0,008), PE (p= 0,000) e qualidade de vida geral (p=0,003) para as faixas etárias mais avançadas; e SE e a pior dor da semana anterior à entrevista (p = 0,015). À AFC sugeriu que o modelo é razoavelmente ajustado para o quatro domínios. Concluindo-se o estudo, pode-se considerar que o IQVFP-VF - como um instrumento específico e relacionado à saúde - é válido e tem confiabilidade atestada nos aspectos mais importantes para a população com feridas: a QV geral, a saúde e os aspectos psicológicos e espirituais / The scarcity of a specific tool to evaluate the quality of life (QOL) of persons with any kind of wounds, in our culture, lead to the development of this study. It aimed to develop and validate a wound version of the Ferrans & Powers QOL Index (QLI -WV). The study was authorized by the QLI authors and was approved by the Ethics Committee of Nursing College of the University of São Paulo and authorized by the institutions where data were collected. This methodological research was based on the procedures established by Paquali. The theoretical pole included the selection and analysis of the items to be included in the instrument (literature review, evaluations by specialists and patients committees and pre test). The empirical ones were developed through the application of the resulted instrument (from last step) to test sample (n=362), teste-retest sample (n=63) and convergent validity sample (n=179). And the analytical pole included the statistical strategies to analyse the instrument´s psychometric properties. The reliability was obtained through: internal consistency - IC (Cronbach´s alpha 0,70) and stability, two weeks after the first instrument apllication (Intraclass correlation coeficient 0,70). The content validity was analysed through the concordance level between judges ( 80%); the concurrent validity, through the correlation among the scores of the item your satisfaction and domains and overall QOL (Pearson or Spearman correlation coeficient 0,30); the convergent validity, through the correlation between the domains and overall QOL of QLI -WV and WHOQOL-bref (strong to moderate correlation between similar domains and weak between the different ones) and the discriminant validity through the comparison among the domains and total QOL and age, wound number and durability, and present and the worst last week pain (Mann Whitney, Kruskal-Wallis, t-Student and ANOVA tests). The confirmatory factor analysis (CFA) was tested by the correlation between the QLI-WV items and original domains (health & functioning - HF; socioeconomic - SE, psychological/spiritual - PS and family - Fa) and the adjustment measures model (LISREL). The sample was composed by outpatients who were assisted at 16 Health Services, and who met the pre established criteria. The results showed Cronbach´s alpha coeficients of 0.90; 0.88; 0.65; 0.81; 0.55 respectively for total QOL and domains (HF, SE, PS and Fa) for IC; r 0.83 (p < 0.000) for stability; r = 0.28 to 0.69 for concurrent validity; r = 0.17 to 0. 60 for convergent validity and significative correlation between: overall QOL and number of wounds (p=0.047); PS and wound durability (p=0.017); HF (p= 0.043), SE (p= 0.008), PS (p= 0.000) and overall QOL (p=0.003) for old patients; and SE and worst last week pain (p = 0.015). CFA suggested a reasonable fit for a four original subscale. The author concludes that QLI-WV - as a specific and health related QOL instrument - can be considered reliable and valid as it confirmed their psychometric properties, related to the most important wound care aspects as the total QOL, health and psychological and spiritual issues

Considerando-se a falta de um instrumento de avaliação de qualidade de vida específico para pessoas estomizadas no Brasil, verificado após revisão bibliográfica de estudos sobre o tema em nosso meio, o objetivo deste estudo foi realizar a adaptação cultural e validar o instrumento City of Hope Quality of Life Ostomy Questionnaire (COH-QOL-OQ), para a língua portuguesa no Brasil. Após contato com a autora do instrumento original e obtenção de sua autorização para o desenvolvimento do processo de adaptação cultural e validação, o projeto recebeu também autorização do Comitê de Ética em Pesquisa da Secretaria Municipal de Saúde de Porto Alegre/RS. É um estudo do tipo metodológico, cujas etapas para adaptação cultural foram baseadas em Beaton et al (2002), modificado conforme preconizado e utilizado no Instituto Municipal d´ Investigación Médica de Barcelona (IMIM). Assim o processo de adaptação cultural, que caracterizou a primeira parte deste estudo, seguiu os seguintes passos: tradução, avaliação por comitê de juízes, grupo focal com pessoas estomizadas e retrotradução. A versão retrotraduzida foi aprovada pela autora do instrumento original, Dra. Márcia Grant e essa versão foi aplicada a 215 pessoas estomizadas, em três municípios do estado do Rio Grande do Sul, para avaliação de algumas de suas propriedades psicométricas. A consistência interna do instrumento, analisada por meio do coeficiente alfa de Cronbach (> 0,70), resultou nos escores 0,92 para QV Total e 0,82, 0,86, 0,83, 0,79 para os domínios BES, BEP, BEF, BEE, respectivamente. A validade de conteúdo foi avaliada por meio do nível de concordância entre os juízes, obtendo-se 70% A validade de critério concorrente, avaliada por meio da correlação (Pearson e Spearman) entre o item 16 do domínio BEP e os domínios do COH-QOL-OQ, verificando-se correlações entre 0,26 e 0,66. Quanto à validade de construto convergente por meio da correlação entre os domínios do COH-QOLOQ e do WHOQOL-Abreviado (coeficientes de correlação de Pearson e Spearman > 30) - os resultados variaram de 0,41 a 0,77. Finalmente, a validade de construto discriminante, em que se utilizaram os testes de Mann-Whitney, Kruskal-Wallis, tstudent e ANOVA, obteve-se discriminância na comparação para o tempo de estomizado, o caráter da estomia,, a situação conjugal, a prática da religião. Os resultados permitiram concluir que o COH-QOL-OQ teve desempenho satisfatório, atestando sua confiabilidade e mostrando-se válido para medir a QV de pessoas estomizadas. / One considering the lack of an evaluation tool of specific quality of life for ostomized people in Brazil, verified after a bibliographical review of studies about the theme in our field, the goal of this study was to carry out the cultural adaptation and to valid the instrument City of Hope Quality of Life Ostomy Questionnaire (COH-QOL-OQ). After contacting the author of the original instrument and obtaining her permission, though it was an instrument of free use and developing of the cultural adaptation process, as shown in the site of the City of Hope National Medical Center (COHNMC), the project has also received authorization from the Committee of ethics in researches of the Municipal Health Secretary of Porto Alegre/RS. It is a study of methodological kind that was carried out based on the steps for cultural adaptation of Guillemin, Bombardier and Beaton adapted for Beaton et al and yet adjusted by the steps followed by the Municipal Institute of Medical Investigation of Barcelona (IMIM). Therefore, the process of cultural adaptation, that has described the first phase of this study, followed the following steps: translations, evaluation by a committee of judges, focal group with ostomized people and retro translation. The retro translated version was approved by the author of the original instrument, Dra. Marcia Grant and such version was clinically applied to 215 ostomized people in three towns of the State of Rio Grande do Sul for evaluation of the psychometric properties of measure. Reliability was analyzed through intern consistency given by coefficient alfa of Cronbach ( > 70%), that resulted in scores 0,92 for QV Total and 0,82, 0,86, 0,83, 0,79 for dominions BES, BEP, BEF, BEE, respectively. One has also evaluated the validity of the content with acceptance of the level of agreement among the judges around 70%. The validity of the opponent criteria was evaluated by the correlation among the item 16 of BEP dominion and the dominions of the COH-QOL-OQ and the validity of the convergent construct through the correlation among the dominions of the COH-QOL-OQ and of WHOQOL- Abbreviated (coefficients of correlation of Pearson and Spearman > 30). The results obtained were correlations between 0,26 to 0,66 for the opponent validity and 0,41 to 0,77 for the convergent validity. The validity of the discriminatory construct was evaluated by tests of Mann- Whitney, Kruskal-Wallis, t-student and ANOVA and there was discrimination when the dominions scores were compared with time of ostomized, sort of ostomy, marital status, religion practice. Concerning the type of ostonomy, income, education, work, religion and age there were no discrimination. One is able to conclude that the COH-QOL-OQ had a good performance, certifying its reliability and showing itself valid to measure the QV construct of ostomized people.

O Transtorno de Ansiedade Social (TAS) é caracterizado pelo medo acentuado ou persistente de situações nas quais o individuo poderia sentir vergonha, levando ao comportamento de esquiva fóbica. Contudo, ainda é pouco diagnosticado, devido às dificuldades em perceber suas características como sintomas. Desse modo, a validação de instrumentos para rastreamento do TAS são de grande importância para auxiliar no diagnóstico e conduta. Dentre os instrumentos disponíveis na literatura, a Escala de Ansiedade Social de Liebowitz (LSAS) tem sido a mais utilizada mundialmente, seja em situações clínicas ou de pesquisa, porém para uso no contexto brasileiro, este instrumento não foi consistentemente validado. Assim, o objetivo deste estudo é aferir as qualidades psicométricas de validade (concorrente, divergente e discriminativa) e fidedignidade da LSAS-SR, traduzida para o português do Brasil, em sua versão auto-aplicada, em uma amostra de universitários da população geral (N=413) e em uma amostra clínica de TAS (N=252). Para tanto, foram coletados dados com universitários de duas instituições do interior do estado de São Paulo. Foram utilizados instrumentos para avaliação específica do TAS e de seus subtipos e outros para avaliação de constructos correlacionados ao TAS, tais como, ansiedade geral, prejuízos funcionais, depressão e abuso de álcool, além da LSAS-SR e um questionário de identificação. A LSAS-SR apresentou excelente consistência interna (=0,90-0,96). A validade convergente com escalas específicas para avaliação do TAS apresentou correlações que variam de 0,33 a 0,84. Com a escala que avalia aspectos específicos do TAS (falar em púbico) as correlações foram fracas e pouco significativas, sendo que com a escala de ansiedade geral esses valores foram de 0,21 a 0,57, classificada como fraca a moderada. Apresentou correlações moderadas, na maioria dos casos, com construtos correlacionados, tais quais depressão (0,36-0,52) e prejuízo funcional (0,30-0,72). Foi observada também adequada capacidade de discriminação da escala no que tange a diferenciação casos e não-casos (S=0,96, E=0,80). Na análise fatorial exploratória apontou inicialmente uma solução de 12 fatores. Testaram-se também soluções de três, quatro e cinco fatores, sendo que os mesmos foram compostos por agrupamentos de itens diferentes dos estudos prévios. A análise fatorial confirmatória também não replicou os achados prévios, sinalizando, tal como encontrado na literatura, a dificuldade em conseguir-se uma estrutura fatorial de consenso, comum às diversas culturas nas quais o instrumento foi estudado. Para avaliação da fidedignidade teste-reteste, foi calculado o Coeficiente de Correlação Intraclasse (0,81) e de Pearson (0,82) os quais mostram-se satisfatórios. Os presentes achados vão ao encontro de estudos internacionais que certificam às excelentes qualidades psicométricas da LSAS-SR. / Social Anxiety Disorder (SAD) is characterized by the sharp or persistent fear of situations in which the individual could feel shame, leading to a behavior of phobic avoidance. However, it is still little diagnosed, due to the difficulty in recognizing its characteristics as symptoms. Therefore, the validation of instruments to screening SAD is highly important to help in the diagnosis and treatment. Among the instruments available in the literature, Liebowitz Social Anxiety Scale (LSAS) has been the most used worldwide, being in clinical or of research situations. However, in the Brazilian context the use of this instrument has not been consistently validated. Thus, the objective of this study is to verify the psychometric qualities of validity (convergent, divergent and discriminant) and reliability of LSAS-SR, translated into Portuguese, in its self-reported version, in a sample of university students from the general population (N = 413) and in a clinical sample of SAD (N = 252). For this purpose, data from university students of two institutions of the interior of the state of São Paulo were collected. They used instruments for specific evaluation of SAD and of their subtypes and others for evaluation of constructs correlated to SAD, such as general anxiety, functional impairment, depression and alcohol abuse in addition to LSAR-SR and an identification questionnaire. LSAS-SR showed excellent internal consistency (=0.90-0.96). The convergent validity with specific scales for the SAD assessment presented correlations that ranged from 0.33 to 0.84. With the scale that evaluates specific aspects of the SAD (to speak in public) the correlations were weak and insignificant and with the general scale of anxiety these values ranged from 0.21 to 0.57, being classified as weak to moderate. It presented moderate correlations, in most cases, with correlated constructs, such as depression (0.36 0.52) and functional impairment (0.30 0.72). It was also observed adequate capacity of discrimination of the scale concerning the differentiation between cases and non-cases (S=0.96, E=0.80). Initially, the exploratory factorial analysis pointed the solution of 12 factors. The models with three, four and five factors were tested too, howsoever they were composed of clusters of different items compared with the original studies. The confirmatory factorial analysis did not replicate previous findings, signaling, as found in the literature, the difficulty in getting a factorial structure of consensus, common to several cultures where the instrument was studied. For the evaluation of test-retest reliability the Interclass Correlation (0.81) and the Pearson (0.82) Coefficients were calculated. Both demonstrated to be satisfactory. The present findings are aligned with international studies that certify the excellent psychometric qualities of LSAS-SR.

Keywords: Equating, IRT, Testlet Respons Model, Rasch Testlet Model, LSC, Concurrent, FPC / Test equating allows direct comparison of test scores from alternative forms measuring the same construct by employing equating procedures to put the test scores on the same metric. Three equating procedures are commonly used in the literature including the concurrent calibration method, the linking separate calibration methods (e.g. the moment methods and the characteristic curve methods), and FPC (Fixed Parameter Calibration) method. The first two types of methods for traditional IRT model have been well developed. FPC is being emphasized recently because of its utility for constructing item bank and computerized adaptive testing (CAT). However, there are few studies that examine the equating accuracy of the FPC method compared to that of the linking separate calibration method and the concurrent calibration method. / The equating methods for the traditional IRT model are not appropriate for linking testlet-based tests because the local independence assumption of IRT model cannot be held for this type of tests. Some measurement models, such as testlet response model, bi-factor model, and Rasch testlet model, were advanced to calibrate the models for the testlet-based tests. Few equating methods, however, that take into consideration the additional local dependence among the examinees' responses to items within testlets have been developed for linking testelet-based tests. / The first study compared the equating accuracies of the FPC, the linking separate calibration, and the concurrent calibration method based on the IRT model to equate item parameters under different conditions. The results indicated that the FPC method using BILOG-MG performed as well as the linking separate calibration method and the concurrent calibration method for linking the equivalent groups. However, the FPC method produced larger equating errors than the other two methods did when the ability distributions of the base and target groups were substantially nonequivalent. Differences in difficulties between the common items set and the total test did not substantially affect the equating results with the three methods, with other conditions being held equal. As expected, both small sample size and less number of common items led to slight greater equating errors. / The last study used the concurrent calibration method under the multidimensional Rasch testlet model to link the testlet-based tests in which the testlets were composed of dichotomous, polytomous, and mixed-format items. The results demonstrated that the concurrent calibration method under the Rasch testlet model worked well in recovering the underlying item parameters. Again, equating errors were substantially increased if the local dependences were ignored in model calibration. And smaller testlet variances for the common testlets led to more accurate equating results. / The results of the studies contribute to a better understanding of the effectiveness of the different equating methods, particularly those for linking testlet-based tests. They also help clarify influences of the other factors, such as characteristics of the examinees, features of the common items and common testlets on equating results. Testing practitioners and researchers may draw useful recommendations from the findings about equating method selection. Nevertheless, generalizations of the findings from the simulated studies to practical testing programs should be cautious. / The second study developed an item characteristic curve method and a testlet characteristic curve method for the testlet response model to transform the scale of item parameters. It then compared the effectiveness of the characteristic curve methods and the concurrent calibration methods under different conditions in linking item parameters from alternate test forms which were composed of dichotomously scored testlet-based items. The newly developed item characteristic curve method and the testlet characteristic curve method were shown to perform similarly as or even better than the Stocking-Lord test characteristic curve method and the concurrent calibration method did. Ignoring the local dependence in model calibration substantially increased equating errors. And larger testlet variances for the common testlets led to greater equating errors. / To address the need to better understand the FPC method and to develop new equating methods for linking testlet-based tests, the studies were to compare the effectiveness of the three types of equating methods under different linking situations and to develop equating methods for linking testlet-based tests. Besides the equating methods concerned, other factors, including sample size, ability distribution, and characteristics of common items and testlets that might affect equating results were also considered. Three simulation studies were carried out to accomplish the research purposes. / Zhang, Zhonghua. / Adviser: Yujing Ni. / Source: Dissertation Abstracts International, Volume: 72-01, Section: A, page: . / Thesis (Ph.D.)--Chinese University of Hong Kong, 2010. / Includes bibliographical references (leaves 156-166). / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Electronic reproduction. Ann Arbor, MI : ProQuest Information and Learning Company, [200-] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Electronic reproduction. Ann Arbor, MI : ProQuest Information and Learning Company, [200-] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Abstract also in Chinese.

As the Test of English as a Foreign Language (TOEFL) is often used as a determiner for university admissions, this study observes the effect on the international student population at a large private university through the examination of the international student admissions data including TOEFL and first-year GPA from 2005-2015. With the anonymous data of 9,837 students, researchers analyzed the result of a cut-score change at the university. Results indicated that the number of international students decreased at the university. As expected, the TOEFL data revealed a normal distribution for the overall (combined) score and subsection scores, while the GPA data did not. The ANOVA for the TOEFL revealed that the change in cut-scores was not completely implemented in 2010. The GPA results from the ANOVA did not appear to be increasing. Correlation analysis reflected a decrease in the correlation coefficient when comparing results from before and after the cut-score change. Correlations of the subsection TOEFL score presented interesting findings. Multiple regression analysis indicated similar conclusions.

This study evaluates the construct validity of the Least Preferred Co-worker (LPC) Scale by testing the predictions made by the motivational hierarchy hypothesis. The respondents were one hundred fifty-nine supervisors and managers. The LPC determined leadership style as either relationship-oriented or task-oriented. Situational control was measured by the Leader-Member Relations scale, Task Structure scale, and Position Power scale. Consideration behavior, measured by the adapted LBDQ, reflected a need for interpersonal success. Initiation of structure behaviors, measured by the adapted LBDQ, reflected a need for task success. Two hypotheses were tested: (1) High LPC leaders in high control situations engage in more task-oriented behaviors than high LPC leaders in low control situations. Low LPC leaders' task-oriented behaviors are consistent across situational control. (2) Low LPC leaders in high control situations engage in more consideration behaviors than low LPC leaders in low control situations. High LPC leaders' consideration behaviors are consistent across situational control. The results of this study supplements previous research (Green, Nebeker & Boni, 1976; Michaelson, 1973) supportive of the motivational hierarchy inteipretation of the LPC. Hypothesis 1 was not supported. However, a significant interaction effect supported Hypothesis 2. Criticisms concerning the construct validity of the LPC, the motivational hierarchy inteipretation of the LPC, and the Contingency Model of Leadership Effectiveness are discussed. Several recommendations for future research are suggested.

A number of objective personality questionnaires have been published which aim to measure the six processes related to Acceptance and Commitment Therapy's model of treatment (acceptance, defusion, present moment awareness, self-as-context, values, and committed action). These measures operationally define these hypothesized processes in research settings. However, little research has been done to investigate whether these processes, as measured by these questionnaires, are differentiable from each other or from other, seemingly similar constructs such as distress tolerance and coping styles. Additionally, it is unclear whether these questionnaire measures have differing relationships with other potentially relevant constructs, such as psychopathology, functioning, and personality. The structure of these process measures was investigated across two participant samples. A multi-trait structure of ACT processes was found, with three higher order dimensions consisting of psychological inflexibility/cognitive fusion, mindfulness, and avoidance, as well as a number of distinguishable lower order traits. This structure was found across multiple samples, and measures of these factor analytically-derived traits were found to have incremental validity and to be distinguishable from other, superficially similar psychological processes. These results provide guidance for measurement selection and suggest future directions for scale development. Relevance to treatment outcome research is also discussed.

The main purpose of this study was to determine if validity coefficients for ACT scores, both composite scores and subject area test scores, and high school grade point average (HSGPA) decayed or held stable over eight semesters of undergraduate study in science, technology, engineering, and mathematics (STEM) fields at civilian four-year institutions, and whether the decay patterns differed from those found in non-STEM fields at the same institutions. Data from 62,212 students at 26 four-year institutions were analyzed in a hierarchical meta-analysis in which student major category (SMC), gender, and admission selectivity levels were considered potential moderators. Four sets of analyses were run. The first was by the three SMCs: STEM-Quantitative majors, STEM-Biological majors, and non-STEM majors. The second was SMC by gender. The third was SMC by admission selectivity level. The fourth was SMC by gender by admission selectivity level. The results across all four analyses indicated that ACT score validity coefficients for STEM-Quantitative and STEM-Biological majors decayed less over eight semesters than the validity coefficients for non-STEM majors did. This was true for the uncorrected and corrected validity coefficients. For the HSGPA validity coefficients, this was true for the corrected validity coefficients. Non-STEM majors had very similar validity decay patterns regardless of the level of analysis. However, four of the eight STEM subgroups in the final set of analyses had minimal amounts of decay, and in some instances small amounts of validity growth.

The purpose of this study was to evaluate the Wechsler Pre-School and Primary Scale of Intelligence for use in predicting school achievement. The Wechsler Pre-School and Primary Scale of Intelligence was administered to 22 children enrolled in the Operation Head Start Program in Logan, Utah. The Wide Range Achievement Test was administered five months later to the same 22 children. The scores on the Wechsler Pre-School and Primary Scale of Intelligence were correlated to the scores on the Wide Range Achievement Test. On the basis of the .53 correlation between the Wechsler Pre-School and Primary Scale of Intelligence Full Scale IQ and the Wide Range Achievement Test Average Standard Score, the Wechsler Pre-School and Primary Scale of Intelligence was judged to be a useful instrument in predicting school achievement.

Merrill’s 2007 5 Star instrument, hereafter called the M-5 Star, is based on Merrill’s well recognized First Principles of Instruction. However, the instrument has not been tested for reliability and validity. In a pilot study, Cropper’s version of his instrument (C-5 Star) showed some reliability and validity evidence, but the M-5 Star needed similar evidence. To address this gap in the literature, the purpose of this study is to assess the reliability and validity evidence for M-5 Star. Raters were drawn from a graduate course in online course evaluation and asked to rate a sample (N = 6) of exclusively online university classes using M-5 Star and three comparison instruments. The comparison instruments also purport to examine course quality but lack the emphasis on instructional strategies in M-5 Star. Interrater reliability evidence for the M-5 Star and the comparison instruments was moderate to substantial (M5-Star ICC = .56, p = .001); Texas IQ ICC = .43, p = .001; WebCT ICC = .75, p = .001: SREB ICC = .53, p = .001). However, interrater reliability was tentative because rater pair scores were averaged, biasing the scores toward agreement and inflating ICC. Low correlations between M-5 Star (the criterion) and the comparison measures indicate divergent validity support that M-5 Star is measuring a different core concept of quality in online classes. M-5 Star correlation with WebCT was r = .39, p = .44 (r2 = .15), with WebCT was r = .44, p = .38 (r2 = .20), and with SREB was r = .43, p = .39 (r2 = .19). In addition to divergent validity analysis, a content validity index (CVI) analysis was undertaken using experts in the area of First Principles. According to First Principles experts, other than a few items on the rubric, the vast majority of the M-5 Star CVI results support close alignment with Merrill’s First Principles of Instruction. Of the 63 M-5 Star individual items, 56 (89%) of them received high scores on Aiken’s CVI that were significant at the .10 level. Study limitations are discussed at length, alongside calls for future research and practical and scholarly significance for the research.