Thesis advisor: Ina V.S. Mullis / The eTIMSS mathematics PSIs were a new and pioneering effort to capitalize on the computer- and tablet-based mode of assessment delivery introduced in the eTIMSS 2019 assessments at the fourth and eighth grades. The PSIs were scenario-based mathematics problem solving tasks intended to enhance measurement of mathematics problem solving and reasoning skills and increase student engagement and motivation in the assessment. These unique tasks were designed to measure the same mathematics content as the rest of the mathematics items in the eTIMSS 2019 assessments, but because of their novelty, there was a question about whether the PSIs achieved this goal and could be reported together with the regular TIMSS mathematics items. Following a full-scale field test in 30 countries completed in May 2018, this dissertation conducted an in-depth investigation of the validity of the eTIMSS 2019 mathematics PSIs with the goals of informing analysis and reporting plans for TIMSS 2019 and providing insights for future assessments aspiring to capitalize on digital technology. This investigation involved three key tasks: 1) examining and documenting the methods and procedures used to develop the PSIs and promote validity by design, 2) investigating the characteristics of the PSIs in terms of the content coverage and fidelity of student responses, and 3) using the eTIMSS field test data to evaluate the internal structure of the PSIs. The results indicate that the PSIs are well-aligned with the TIMSS 2019 Mathematics Framework and elicited the intended interactions from students. The regular and PSI items were found to measure the same unidimensional construct, and therefore can be validly reported together on the TIMSS 2019 achievement scale. The lessons TIMSS learned in developing the PSIs for eTIMSS 2019 and suggestions for the future also are discussed. / Thesis (PhD) — Boston College, 2019. / Submitted to: Boston College. Lynch School of Education. / Discipline: Educational Research, Measurement and Evaluation.

O presente trabalho teve como objetivo estimar a confiabilidade e validade do Índice de Prioridade de Tratamento (IPT) na avaliação das alterações oclusais, através da análise de 200 modelos selecionados do Arquivo de Ortodontia da Faculdade de Odontologia de Bauru, apresentando os diferentes tipos de má oclusão. Estes modelos foram avaliados em dois momentos, o primeiro, por uma comissão formada por 16 experientes ortodontistas que os avaliou subjetivamente quanto à severidade da má oclusão, dificuldade e duração do tratamento necessário. Em um segundo momento, os modelos foram avaliados através do IPT por dois ortodontistas previamente calibrados para a utilização do índice. A confiabilidade intraexaminador foi testada a partir da reavaliação de 50 pares de modelos e foi calculada pelo coeficiente de correlação intraclasse (CCI), além do teste t dependente. A confiabilidade interexaminador foi estimada sobre os 200 pares de modelos de estudo, sendo calculada pelo CCI e pelo teste t independente. Por fim, a validade do índice foi avaliada pelo comparando-se as médias dos valores obtidos pelo IPT e as percepções subjetivas através do coeficiente de correlação de Pearson. Por motivos comparativos, os modelos também foram analisados através do índice PAR (Peer Assessment Rating), que é um instrumento válido e amplamente aceito na avaliação dos resultados oclusais. Os resultados mostraram que o IPT, assim como o PAR, é um índice altamente reprodutível, uma vez que revelou altos níveis de concordância inter (ICC=0,97) e intraexaminador (ICC1=0,97 e ICC2= 0,96), e válido para investigar a severidade da má oclusão (R=0,25), dificuldade do tratamento (R=0,24) e duração do tratamento (R=0,29). No entanto, apesar de significantes, as correlações encontradas pelo IPT durante a validação foram muito fracas, principalmente em comparação com o índice PAR. Conclui-se, portanto, que o IPT apesar de reprodutível, possui pouca validade como um instrumento de avaliação das alterações oclusais. / The present study sought to estimate the reliability and validity of the Treatment Priority Index (TPI) for assessment of occlusal changes through analysis of 200 dental casts from the files of the Department of Orthodontics at Bauru Dental School, selected as a representative sample of the various types of malocclusion. These casts were evaluated twice: first, by a panel of 16 experienced orthodontists, who carried out subjective assessments of the severity of malocclusion and predicted the difficulty and duration of the required treatment. In another moment, the casts were assessed with the TPI by two orthodontists calibrated beforehand. The intrarater reliability was tested by means of reassessment of 50 pairs of models, with an intraclass correlation coefficient (ICC) and the dependent t-test. The inter-rater reliability was estimated considering all 200 casts, through the ICC and the independent t-test. Finally, the validity of the index was assessed by comparison between average TPI scores and subjective perceptions through Pearsons correlation coefficient. For comparative purposes, dental casts were also analyzed using the PAR (Peer Assessment Rating) index, a valid and widely accepted instrument for assessment of occlusal outcomes. Results showed that the TPI is a highly reproducible index, as is the PAR, with high levels of inter-rater (ICC=0.97) and intra-rater reliability (ICC1=0.97, ICC2=0.96), and is a valid instrument for assessment of malocclusion severity (R=0.25), treatment difficulty (R=0.24) and treatment duration (R=0.29). However, despite their statistical significance, the TPI correlations were very weak, particularly on comparison with the PAR. Despite its reproducibility, the TPI has very limited validity for assessment of occlusal changes provided by the orthodontic treatment.

O presente trabalho teve como objetivo estimar a confiabilidade e validade do Índice de Prioridade de Tratamento (IPT) na avaliação das alterações oclusais, através da análise de 200 modelos selecionados do Arquivo de Ortodontia da Faculdade de Odontologia de Bauru, apresentando os diferentes tipos de má oclusão. Estes modelos foram avaliados em dois momentos, o primeiro, por uma comissão formada por 16 experientes ortodontistas que os avaliou subjetivamente quanto à severidade da má oclusão, dificuldade e duração do tratamento necessário. Em um segundo momento, os modelos foram avaliados através do IPT por dois ortodontistas previamente calibrados para a utilização do índice. A confiabilidade intraexaminador foi testada a partir da reavaliação de 50 pares de modelos e foi calculada pelo coeficiente de correlação intraclasse (CCI), além do teste t dependente. A confiabilidade interexaminador foi estimada sobre os 200 pares de modelos de estudo, sendo calculada pelo CCI e pelo teste t independente. Por fim, a validade do índice foi avaliada pelo comparando-se as médias dos valores obtidos pelo IPT e as percepções subjetivas através do coeficiente de correlação de Pearson. Por motivos comparativos, os modelos também foram analisados através do índice PAR (Peer Assessment Rating), que é um instrumento válido e amplamente aceito na avaliação dos resultados oclusais. Os resultados mostraram que o IPT, assim como o PAR, é um índice altamente reprodutível, uma vez que revelou altos níveis de concordância inter (ICC=0,97) e intraexaminador (ICC1=0,97 e ICC2= 0,96), e válido para investigar a severidade da má oclusão (R=0,25), dificuldade do tratamento (R=0,24) e duração do tratamento (R=0,29). No entanto, apesar de significantes, as correlações encontradas pelo IPT durante a validação foram muito fracas, principalmente em comparação com o índice PAR. Conclui-se, portanto, que o IPT apesar de reprodutível, possui pouca validade como um instrumento de avaliação das alterações oclusais. / The present study sought to estimate the reliability and validity of the Treatment Priority Index (TPI) for assessment of occlusal changes through analysis of 200 dental casts from the files of the Department of Orthodontics at Bauru Dental School, selected as a representative sample of the various types of malocclusion. These casts were evaluated twice: first, by a panel of 16 experienced orthodontists, who carried out subjective assessments of the severity of malocclusion and predicted the difficulty and duration of the required treatment. In another moment, the casts were assessed with the TPI by two orthodontists calibrated beforehand. The intrarater reliability was tested by means of reassessment of 50 pairs of models, with an intraclass correlation coefficient (ICC) and the dependent t-test. The inter-rater reliability was estimated considering all 200 casts, through the ICC and the independent t-test. Finally, the validity of the index was assessed by comparison between average TPI scores and subjective perceptions through Pearsons correlation coefficient. For comparative purposes, dental casts were also analyzed using the PAR (Peer Assessment Rating) index, a valid and widely accepted instrument for assessment of occlusal outcomes. Results showed that the TPI is a highly reproducible index, as is the PAR, with high levels of inter-rater (ICC=0.97) and intra-rater reliability (ICC1=0.97, ICC2=0.96), and is a valid instrument for assessment of malocclusion severity (R=0.25), treatment difficulty (R=0.24) and treatment duration (R=0.29). However, despite their statistical significance, the TPI correlations were very weak, particularly on comparison with the PAR. Despite its reproducibility, the TPI has very limited validity for assessment of occlusal changes provided by the orthodontic treatment.

An attitude is a mental state that influences the way an individual responds to relevant objects and situations. Attitudes toward the elderly have a potential impact on all Americans, as the population is growing older. These attitudes constitute an important area of study because their nature is unclear and their impact extensive. The Kogan Attitudes Toward Old People Scale (OP) was examined to establish reliability and validity estimates. A questionnaire, consisting of this scale and four others, was administered to a sample of adults. Factor analysis of the OP revealed two factors, which were somewhat ambiguous. Internal consistency estimates for these factors and the total OP ranged from .72 to .92. Validity estimates were in the expected direction and ranged from -.13 to .36. Gender, age, education level, and degree of tolerance for others immerged as predictive variables for reported attitudes toward the elderly. It was concluded that the use of the OP is of questionable utility. If one must use it, the total OP score or only the items from Factors 1 and 2 should be utilized, as long as the discussed weaknesses, such as the sampling and validation procedures, are taken into account

Social Validity is a concept used in behavioral intervention research. It focuses on whether the goals of treatment, the intervention techniques used, and the outcomes achieved are acceptable, relevant, and useful to the individual in treatment. The Social Validity Manual, 2e, provides background on the development of social validity, an overview of current research in social validity, and guidelines for expanding the practice of social validation. The book offers detailed information on scales and methods for measuring social validity across the goals, procedures, and effects of treatments utilized in various fields. The second edition incorporates advances in research findings and offers two new chapters on the use of social validity in the health sciences and how social validity plays an important role in increasing cultural awareness. / https://dc.etsu.edu/etsu_books/1218/thumbnail.jpg

Delay discounting (the devaluation of rewards delayed in time) has been studied extensively using animal models with psychophysical adjustment procedures. Similar procedures were soon developed to assess delay discounting in humans. Although across species the same mathematical function relates discounted value to imposed delay, several methodological concerns have been implicated in human delay discounting procedures. A procedure recently developed to address these concerns is the Experiential Discounting Task (EDT). This task arranges experienced delays and rewards that humans make decisions regarding—experiencing the outcomes of their choices within session before making additional choices. The popularity of this procedure has been fueled by reports of its sensitivity to acute experimental manipulation, and that it has been predictive of treatment success. Similar sensitivity results have not been found when a traditional delay discounting task (DDT) has been used. Though the EDT appears useful for a variety of reasons, it has not been subjected to the same rigorous internal validity and reliability tests that traditional DDTs have. In two experiments we examined the test-retest reliability of the EDT (Experiment 1) and the way in which choice trials are regulated (Experiment 2). Results demonstrate that the EDT is reliable across time and choice is insensitive to trial regulation differences. We conclude with a critique of the EDT as a procedure for assessing delay discounting and hypothesize other processes it may be measuring.

acase@tulane.edu / 1 / Kathryn M. Jones

<p>Did the right person win? In competitive sports the difference between winning and loosing can be very marginal but still the result will have a big impact. To finish second, even if it is an Olympic Game, can be felt like a failure for the athlete. In the light of this the quality of the measurements and the judgements in sports are an extremely important issue. In this dissertation, consisting of two separate reports and one summary report, the freestyle discipline, acroski, and the gymnastic discipline, rhythmic gymnastics (RG) where studied with a focus on the rules and judgements in the sports. The reliability and validity of the rule and judging systems were analysed together with changes in the rule and judging systems. The reliability, as the judges ability to interpret the rules and judge in a consistent way, and validity in the sense that the judgements, and the rules for the judgements, where in accordance with the idea of the sport. Information was gathered from rulebooks, judging manuals, meeting protocols, and historical documents. For information on how the rules and judgements where perceived among people involved in the two sports, interviews where carried out with two coaches, two judges and four athletes from each of the sports. In the summary report the results from the two reports where discussed and a comparison of the rule systems and judgements was made together with a discussion about reliability and validity problems in sports measured by time, length and height. The results showed that the rule and judging systems in acroski and RG in a short time have gone through major changes to increase the reliability and validity in the judgements. The quick evolution of these sports seems to be the main reason for these changes but also other factors where of vital importance. Finding the balance between making the judgements as objective as possible, without loosing the idea of the sport, together with the quick evolution of the sports, seemed to be the main challenges in the rule and judging systems of acroski and RG.</p>

<p><strong>Bakgrund</strong>: Det finns i nuläget inget svenskt self-efficacy instrument för barn och ungdomar med smärta. Den här undersökningen syftade till att översätta samt reliabilitets- och validitetstesta en engelskspråkig self-efficacy skala för barn och ungdomar med smärta.</p><p><strong>Metod</strong>: Urvalet bestod av 62 elever från tre olika klasser i Uppsala. Skalan översattes till svenska och ett frågeformulär utformades för att möjliggöra en utvärdering av test-retest reliabilitet samt kriterie- och ögonblicksvaliditet.</p><p><strong>Resultat</strong>: Värdena på viktad kappa för överensstämmelsen av två skattningar (S1 och S2) med self-efficacy instrumentet (SEIS) varierade mellan <em>K_w</em>₌0,37 -  <em>K_w</em>₌0,75. Sambandet mellan Functional disability inventory (FDI) och SEIS beräknades med Spearmans rangkorrelation där <em>r_s</em> = 0,364 (p<0,05). Ögonblicksvaliditeten för SEIS beskrevs som att den var lättförståelig, innehöll vardagliga aktiviteter men att vissa aktiviteter saknades.</p><p><strong>Konklusion</strong>: Slutsatsen av denna undersökning var att frågorna i SEIS bör omarbetas för att bli mer specifika och kunna användas. Det kan även utarbetas nya aktiviteter till instrumentet eftersom resultaten av undersökningen visade att vissa aktiviteter saknades. På grund av urvalets karaktär var resultaten troligtvis inte generaliserbara.</p>

Bakgrund: Det finns i nuläget inget svenskt self-efficacy instrument för barn och ungdomar med smärta. Den här undersökningen syftade till att översätta samt reliabilitets- och validitetstesta en engelskspråkig self-efficacy skala för barn och ungdomar med smärta. Metod: Urvalet bestod av 62 elever från tre olika klasser i Uppsala. Skalan översattes till svenska och ett frågeformulär utformades för att möjliggöra en utvärdering av test-retest reliabilitet samt kriterie- och ögonblicksvaliditet. Resultat: Värdena på viktad kappa för överensstämmelsen av två skattningar (S1 och S2) med self-efficacy instrumentet (SEIS) varierade mellan Kw= 0,37 -  Kw=0,75. Sambandet mellan Functional disability inventory (FDI) och SEIS beräknades med Spearmans rangkorrelation där rs = 0,364 (p&lt;0,05). Ögonblicksvaliditeten för SEIS beskrevs som att den var lättförståelig, innehöll vardagliga aktiviteter men att vissa aktiviteter saknades. Konklusion: Slutsatsen av denna undersökning var att frågorna i SEIS bör omarbetas för att bli mer specifika och kunna användas. Det kan även utarbetas nya aktiviteter till instrumentet eftersom resultaten av undersökningen visade att vissa aktiviteter saknades. På grund av urvalets karaktär var resultaten troligtvis inte generaliserbara.