<p>Det svenska förarprovet har vuxit fram ur praktisk yrkeserfarenhet. Oavsett vilken trafikinspektör som bedömer ett specifikt körprov ska bedömningen göras lika. Kognitiva egenskaper bedöms i körprovet genom att observera handlingar och beteenden. Syftet med denna uppsats var att ur ett teoretiskt perspektiv undersöka hur trafikinspektörer bedömer kognition, för att undersöka vilka likheter och skillnader som finns. Tre analyser gjordes utifrån tolv intervjuer med trafikinspektörer vid två av Vägverkets förarprovskontor.</p><p>Trafikinspektörerna ombads beskriva vad som är viktigt hos en bilförare. Samtliga formella kriterier för körprovet nämndes av någon informant. Utöver kriterierna nämndes även många andra begrepp av ett fåtal personer vardera. Dessa begrepp kan räknas till områdena kognitionspsykologi, allmän psykologi, bilkörning och övergripande egenskaper. De tre mest nämnda begreppen var uppmärksamhet, erfarenhet och riskmedvetande.</p><p>Sju körprovskriterier definierades av trafikinspektörerna. Många olika begrepp användes för att definiera kriterierna och samtliga kriterier definierades olika av informanterna. Definitionerna delades in i kategorier och de kategorier som användes för samtliga kriterier var förberedelser, handlingar och perception.</p><p>Ur det som sades av informanterna gjordes en analys om hur begrepp kopplas samman med varandra, för att hitta alla associationer som görs till varje begrepp. Många olika kopplingar kunde hittas genom denna analys och av dessa nämndes hälften av enbart någon informant. Två kopplingar nämndes av elva av informanterna. Den ena var manövrering – automatisering, den andra var uppmärksamhet – syn.</p><p>Studien påvisade både likheter och skillnader i åsikter informanterna emellan. I viss utsträckning använda sig informanterna av samma begrepp, men de lade olika innebörder i begreppen. Trafikinspektörernas olika synsätt medför en risk för att de bedömer körprov olika trots att de använder samma bedömningskriterier.</p> / <p>The Swedish driving license test has developed from practical professional experience. Regardless of which traffic inspector judges a specific driving test, the judgement is to be equal. Cognitive qualities are judged in the driving test by observing actions and behaviours. The purpose of this essay was to investigate, from a theoretical perspective, how traffic inspectors judge cognition, in order to investigate differences and similarities. Three analyses were made, based on twelve interviews with traffic inspectors working at two of the Swedish Road Administration’s offices for driving license tests.</p><p>The traffic inspectors were asked to describe what is important in a car driver. All formal driving test criteria were mentioned by some informant. In addition to the criteria, many other concepts were mentioned by a few persons each. These concepts belong to the areas cognitive psychology, general psychology, car driving and overall qualities. The three most mentioned concepts, including criteria and other concepts, were attention, experience and risk awareness.</p><p>Seven driving test criteria were defined by the traffic inspectors. Many different concepts were used to define the criteria and all criteria were defined differently by the informants. The definitions were divided into categories. The categories preparations, actions and perception were used for all criteria.</p><p>An analysis about how concepts were connected to each other was made, in order to find all associations made with each concept. Many different connections were found in this analysis and more than half of these were mentioned by only one informant.</p><p>Two connections were mentioned by eleven informants. One of these was manoeuvring – automation, the other was attention – vision.</p><p>Both similarities and differences in opinions were found among the informants in this study. The informants used the same concepts to some extent, but they associated different meanings with the concepts. The traffic inspectors’ differences in opinions lead to a risk of judging driving tests differently, although the same judgement criteria are used.</p>

Det svenska förarprovet har vuxit fram ur praktisk yrkeserfarenhet. Oavsett vilken trafikinspektör som bedömer ett specifikt körprov ska bedömningen göras lika. Kognitiva egenskaper bedöms i körprovet genom att observera handlingar och beteenden. Syftet med denna uppsats var att ur ett teoretiskt perspektiv undersöka hur trafikinspektörer bedömer kognition, för att undersöka vilka likheter och skillnader som finns. Tre analyser gjordes utifrån tolv intervjuer med trafikinspektörer vid två av Vägverkets förarprovskontor. Trafikinspektörerna ombads beskriva vad som är viktigt hos en bilförare. Samtliga formella kriterier för körprovet nämndes av någon informant. Utöver kriterierna nämndes även många andra begrepp av ett fåtal personer vardera. Dessa begrepp kan räknas till områdena kognitionspsykologi, allmän psykologi, bilkörning och övergripande egenskaper. De tre mest nämnda begreppen var uppmärksamhet, erfarenhet och riskmedvetande. Sju körprovskriterier definierades av trafikinspektörerna. Många olika begrepp användes för att definiera kriterierna och samtliga kriterier definierades olika av informanterna. Definitionerna delades in i kategorier och de kategorier som användes för samtliga kriterier var förberedelser, handlingar och perception. Ur det som sades av informanterna gjordes en analys om hur begrepp kopplas samman med varandra, för att hitta alla associationer som görs till varje begrepp. Många olika kopplingar kunde hittas genom denna analys och av dessa nämndes hälften av enbart någon informant. Två kopplingar nämndes av elva av informanterna. Den ena var manövrering – automatisering, den andra var uppmärksamhet – syn. Studien påvisade både likheter och skillnader i åsikter informanterna emellan. I viss utsträckning använda sig informanterna av samma begrepp, men de lade olika innebörder i begreppen. Trafikinspektörernas olika synsätt medför en risk för att de bedömer körprov olika trots att de använder samma bedömningskriterier. / The Swedish driving license test has developed from practical professional experience. Regardless of which traffic inspector judges a specific driving test, the judgement is to be equal. Cognitive qualities are judged in the driving test by observing actions and behaviours. The purpose of this essay was to investigate, from a theoretical perspective, how traffic inspectors judge cognition, in order to investigate differences and similarities. Three analyses were made, based on twelve interviews with traffic inspectors working at two of the Swedish Road Administration’s offices for driving license tests. The traffic inspectors were asked to describe what is important in a car driver. All formal driving test criteria were mentioned by some informant. In addition to the criteria, many other concepts were mentioned by a few persons each. These concepts belong to the areas cognitive psychology, general psychology, car driving and overall qualities. The three most mentioned concepts, including criteria and other concepts, were attention, experience and risk awareness. Seven driving test criteria were defined by the traffic inspectors. Many different concepts were used to define the criteria and all criteria were defined differently by the informants. The definitions were divided into categories. The categories preparations, actions and perception were used for all criteria. An analysis about how concepts were connected to each other was made, in order to find all associations made with each concept. Many different connections were found in this analysis and more than half of these were mentioned by only one informant. Two connections were mentioned by eleven informants. One of these was manoeuvring – automation, the other was attention – vision. Both similarities and differences in opinions were found among the informants in this study. The informants used the same concepts to some extent, but they associated different meanings with the concepts. The traffic inspectors’ differences in opinions lead to a risk of judging driving tests differently, although the same judgement criteria are used.

Background: The Swedish driving licence test is a criterion-referenced test resulting in a pass or fail. It currently consists of two parts - a theory test with 65 multiple-choice items and a practical driving test where at least 25 minutes are spent driving in traffic. It is a high-stakes test in the sense that the results are used to determine whether the test-taker should be allowed to drive a car without supervision. As the only other requirements for obtaining a licence is a few hours of hazard education (and a short introduction if you intend to drive with a lay instructor) it is important that the test result, in terms of pass or fail, is reliable and valid. If this is not the case it could have detrimental effects on traffic safety. Examining all relevant aspects is beyond the scope of this licentiate thesis so I have focused on reliability. Methods Reliability for both the theoretical and practical test results was examined. As these are very different types of tests the types of reliability examined also differed. In order to examine inter-rater reliability of the driving test 83 examiners were accompanied by one of five selected supervising examiners for a day of tests. All in all 535 tests were conducted with two examiners assessing the same performance. At the end of the day the examiners compared notes and tried to determine the reason for any inconsistencies. Both examiners and students also filled in questionnaires with questions about background and preparation. As for studying decision consistency and decision accuracy of the theory test, three test versions (a total of around 12,000 tests) were examined with the help of methods devised by Subkoviak (Subkoviak, 1976, 1988) and Hanson &amp; Brennan (Brennan, 2004; Hanson &amp; Brennan, 1990). Results The results from two research studies concerning reliability were presented. Study I focused on inter-rater reliability in the driving test and in 93 per cent of cases the examiners made the same assessment. For the tests where their opinions differed there was no correlation to any of the background variables or other variables examined except for three, which had logical explanations and did not constitute a problem. Although there were cases where the differences were due to different stances on matters of interpretation the most common suggested cause was the placement in the car (back seat vs. front seat). Although the supervising examiners gave both praise and criticism as to how the test was carried out the study does not answer the question whether the tests were equal in terms of composition and difficulty. In Study II the focus was on decision consistency and decision accuracy in the theory test. Three versions of the theory tests were examined and, on the whole, found to be fairly similar in terms of item difficulty and score distribution, but the mean was so close to the cut-score (i.e. the score required to pass) that the pass rate differed somewhat between versions. Agreement coefficients were around .80 for all test versions (between .79 and .82 depending on method). Classification accuracy indicated an .87 probability of a correct classification. Conclusion It is important to examine the reliability and validity of the driving licence test since a misclassification can have serious consequences in terms of traffic safety. In the studies included here the rate of agreement between examiners is deemed as satisfactory. It would be preferable if the classification consistency and classification accuracy, as estimated by the methods used, were higher for the theory test, given its importance. While reliability in terms of agreement between raters/examiners or consistency and accuracy of classification are routinely examined in other contexts, such as large-scale educational testing, this is not often done for the driving licence tests. At the same time, the methods used here can be transferred to contexts where such properties are generally not examined. Collecting information about test-takers and examiners, like in Study I, can provide evidence concerning possible bias. Examining to what extent decisions are consistent is one important aspect of collecting evidence that shows that test results can be used to draw conclusions about driver competence. Still, regardless of outcome, validation is a process that never ends. There is always reason to examine various aspects and make further improvements. There are also many other relevant aspects to examine. A prerequisite for the validity of the score interpretation of a criterion-referenced test like this one is that the cut-score is appropriate and the content relevant. This should therefore be the subject of further research as the validation process continues.