Background: The Swedish driving licence test is a criterion-referenced test resulting in a pass or fail. It currently consists of two parts - a theory test with 65 multiple-choice items and a practical driving test where at least 25 minutes are spent driving in traffic. It is a high-stakes test in the sense that the results are used to determine whether the test-taker should be allowed to drive a car without supervision. As the only other requirements for obtaining a licence is a few hours of hazard education (and a short introduction if you intend to drive with a lay instructor) it is important that the test result, in terms of pass or fail, is reliable and valid. If this is not the case it could have detrimental effects on traffic safety. Examining all relevant aspects is beyond the scope of this licentiate thesis so I have focused on reliability. Methods Reliability for both the theoretical and practical test results was examined. As these are very different types of tests the types of reliability examined also differed. In order to examine inter-rater reliability of the driving test 83 examiners were accompanied by one of five selected supervising examiners for a day of tests. All in all 535 tests were conducted with two examiners assessing the same performance. At the end of the day the examiners compared notes and tried to determine the reason for any inconsistencies. Both examiners and students also filled in questionnaires with questions about background and preparation. As for studying decision consistency and decision accuracy of the theory test, three test versions (a total of around 12,000 tests) were examined with the help of methods devised by Subkoviak (Subkoviak, 1976, 1988) and Hanson & Brennan (Brennan, 2004; Hanson & Brennan, 1990). Results The results from two research studies concerning reliability were presented. Study I focused on inter-rater reliability in the driving test and in 93 per cent of cases the examiners made the same assessment. For the tests where their opinions differed there was no correlation to any of the background variables or other variables examined except for three, which had logical explanations and did not constitute a problem. Although there were cases where the differences were due to different stances on matters of interpretation the most common suggested cause was the placement in the car (back seat vs. front seat). Although the supervising examiners gave both praise and criticism as to how the test was carried out the study does not answer the question whether the tests were equal in terms of composition and difficulty. In Study II the focus was on decision consistency and decision accuracy in the theory test. Three versions of the theory tests were examined and, on the whole, found to be fairly similar in terms of item difficulty and score distribution, but the mean was so close to the cut-score (i.e. the score required to pass) that the pass rate differed somewhat between versions. Agreement coefficients were around .80 for all test versions (between .79 and .82 depending on method). Classification accuracy indicated an .87 probability of a correct classification. Conclusion It is important to examine the reliability and validity of the driving licence test since a misclassification can have serious consequences in terms of traffic safety. In the studies included here the rate of agreement between examiners is deemed as satisfactory. It would be preferable if the classification consistency and classification accuracy, as estimated by the methods used, were higher for the theory test, given its importance. While reliability in terms of agreement between raters/examiners or consistency and accuracy of classification are routinely examined in other contexts, such as large-scale educational testing, this is not often done for the driving licence tests. At the same time, the methods used here can be transferred to contexts where such properties are generally not examined. Collecting information about test-takers and examiners, like in Study I, can provide evidence concerning possible bias. Examining to what extent decisions are consistent is one important aspect of collecting evidence that shows that test results can be used to draw conclusions about driver competence. Still, regardless of outcome, validation is a process that never ends. There is always reason to examine various aspects and make further improvements. There are also many other relevant aspects to examine. A prerequisite for the validity of the score interpretation of a criterion-referenced test like this one is that the cut-score is appropriate and the content relevant. This should therefore be the subject of further research as the validation process continues.

Submitted by Renata Lopes (renatasil82@gmail.com) on 2016-04-12T17:18:39Z No. of bitstreams: 1 flaviaguimaraeshenriques.pdf: 667239 bytes, checksum: 5b93224cd56ee8b6710aabff2fcc94a4 (MD5) / Approved for entry into archive by Adriana Oliveira (adriana.oliveira@ufjf.edu.br) on 2016-04-24T03:31:32Z (GMT) No. of bitstreams: 1 flaviaguimaraeshenriques.pdf: 667239 bytes, checksum: 5b93224cd56ee8b6710aabff2fcc94a4 (MD5) / Made available in DSpace on 2016-04-24T03:31:32Z (GMT). No. of bitstreams: 1 flaviaguimaraeshenriques.pdf: 667239 bytes, checksum: 5b93224cd56ee8b6710aabff2fcc94a4 (MD5) Previous issue date: 2016-02-23 / CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / Os objetivos do presente estudo foram: 1) realizar uma revisão da literatura dos estudos que investigaram a precisão com que medidas de consciência fonológica (CF) classificam indivíduos como estando em risco ou não estando em risco de apresentar dificuldade de leitura/escrita; 2) avaliar a dimensionalidade da CF em falantes do português brasileiro; e 3) verificar a precisão de classificação do risco/não risco de dificuldade de leitura/escrita de diferentes medidas de CF. Os resultados da revisão da literatura evidenciaram, de uma forma geral, que as diferentes medidas de CF, quando analisadas isoladamente foram ruins ou razoáveis em classificar as crianças como em risco/sem risco de dificuldade de leitura/escrita. Duzentas e treze crianças foram avaliadas através de diferentes tarefas de CF quando estavam no último ano da Educação Infantil e, aproximadamente, um ano depois, foram avaliadas através de uma medida de leitura e uma medida de escrita. Resultados de Análises Fatoriais evidenciaram que as diferentes medidas de CF avaliam um construto predominantemente unidimensional e análises da curva ROC indicaram que duas medidas compostas de CF mostraram-se razoáveis para classificar as crianças como tendo ou não tendo risco de dificuldade de leitura ou de escrita, apresentando áreas sob a curva em torno de 0,75. / The aims of this study were: 1) to present a literature review of studies about the precision of phonological awareness measures to classify individuals as being or not at risk of presenting difficulties in reading or writing; 2) to evaluate the dimensionality of phonological awareness in Brazilian Portuguese speakers; and 3) to verify the classification accuracy of the risk/no risk of difficulty in reading and writing from different measures of phonological awareness. In general, among the reviewed studies, phonological awareness measures varied from poor to reasonable in reading/writing risk classification accuracy. Two hundred and thirteen Brazilian Portuguese speaking children took part on the present study. They were evaluated through different phonological awareness tasks in the last year of early childhood education. Nine months later, they were evaluated through a reading measure and a writing measure. Factorial Analysis results showed that the different phonological awareness measures index a single construct. Concerning the ROC curve analysis, two composite measures of phonological awareness proved reasonable to discriminate children in the groups with and without difficulty in reading/writing, presenting AUCs around 0,75.

Traditional latent class analysis (LCA) considers entropy R2 as the only measure of effect size. However, entropy may not always be reliable, a low boundary is not agreed upon, and good separation is limited to values of greater than .80. As applications of LCA grow in popularity, it is imperative to use additional sources to quantify LCA classification accuracy. Greater classification accuracy helps to ensure that the profile of the latent classes reflect the profile of the true underlying subgroups. This Monte Carlo study compared the quantification of classification accuracy and confidence intervals of three effect sizes, entropy R2, I-index, and Cohen’s d. Study conditions included total sample size, number of dichotomous indicators, latent class membership probabilities (γ), conditional item-response probabilities (ρ), variance ratio, sample size ratio, and distribution types for a 2-class model. Overall, entropy R2 and I-index showed the best accuracy and standard error, along with the smallest confidence interval widths. Results showed that I-index only performed well for a few cases.

This thesis deals with training dataset and validation dataset for Earth observation classification accuracy improvement. Experiments with training data and validation data for two classification algorithms (Maximum Likelihood - MLC and Support Vector Machine - SVM) are carried out from the forest-meadow landscape located in the foothill of the Giant Mountains (Podkrkonoší). The thesis is base on the assumption that 1/3 of training data and 2/3 of validation data is an ideal ratio to achieve maximal classification accuracy (Foody, 2009). Another hypothesis was that in a case of SVM classification, a lower number of training point is required to achieve the same or similar accuracy of classification, as in the case of the MLC algorithm (Foody, 2004). The main goal of the thesis was to test the influence of proportion / amount of training and validation data on the classification accuracy of Sentinel - 2A multispectral data using the MLC algorithm. The highest overal accuracy using the MLC classification algorithm was achieved for 375 training and 625 validation points. The overal accuracy for this ratio was 72,88 %. The theory of Foody (2009) that 1/3 of training data and 2/3 of validation data is an ideal ratio to achieve the highest classification accuracy, was confirmed by the overal accuracy and...

There are these methods implemented: circular sectors, color moments, color coherence vector and Gabor filters, they are based on low-level image features. These methods were evaluated after their optimal parameters were found. The finding of optimal parameters of methods is done by measuring of classification accuracy of learning operators and usage of operator cross validation on images in program RapidMiner. Implemented methods are evaluated on these image categories - ancient, beach, bus, dinousaur, elephant, flower, food, horse, mountain and natives, based on total average precision. The classification accuracy result is increased by 8 % by implemented modification (HSB color space + statistical function median) of original method circular sectors. The combination of methods color moments, circular sectors and Gabor filters with weighted ratio gives the best total average precision at 70,48 % and is the best method among all implemented methods.