No description available.

No description available.

There is a general belief that British and North American EFL proficiency tests represents radically different approaches to language test development. The North American tradition in language testing is heavily based on psychometric properties of tests such as reliability, and concurrent and predictive validity, whereas the British tradition is more focused on the specification of test content and expert judgement. Language proficiency tests, in either the American or British tradition, are designed to serve different purposes, so they may not be comparable in terms of defined purposes. Nevertheless, the term '<i>language proficiency'</i>, no matter how it is defined, implies that we are referring to a monolithic concept. In the real world, test results are often used for screening purposes; the candidates' <i>ability</i> to cope with the future language medium is predicted by the proficiency criterion. If it is the case that language proficiency tests are used for similar purposes, i.e., measuring the general language ability of the candidates, comparability of such tests is a legitimate matter. This study compares two English language proficiency test batteries: TOEFL and IELTS. The main objectives of the research were to investigate the extent to which TOEFL and IELTS are comparable in terms of: a) the operational definitions of language proficiency on which the two tests are based, b) the degree to which the two tests provide similar information concerning the abilities of the testees. Analysis of test content suggests that both batteries are based on the notion that proficiency is divisible by skill (e.g. reading) and element of language (e.g. syntax), thus we have tests of reading, writing, listening, speaking, as well as tests of grammar and vocabulary. However, the tests <i>differ</i> in their representation of the scope of skills and elements of language proficiency. The analysis also shows that the TOEFL differs from the IELTS in this method of testing. Despite these differences in test methods and scope, to a great extent both tests measure a common aspect of the subjects' language ability, therefore their internal structures are unifactorial. A <i>g-factor</i> (general language proficiency) comprises much of the total variance in both tests.

For many students, English as a foreign language (EFL) assessments are high-stake examinations, the results of which will determine their future study and career paths. This thesis will present data gathered from questionnaires of students and examiners, filmed interviews, audio recordings and focus group feedback. The key question posed is: is it possible for international EFL exams to have global applicability and therefore maintain test validity? Furthermore, to what extent should international EFL test writers take into account the regional, socio-cultural context of the recipient student body when making question choices and devising assessment criteria? This thesis attempts to address these questions through interpretive case study research of oral interview assessment in a female campus of a tertiary college in the United Arab Emirates (UAE). CAT (Communication Accommodation Theory) was the theoretical tool used to examine the interaction between examiners and candidates during the IEL TS-style oral assessment. The study looked at the questions typically asked in an international EFL interview and the reaction of the participants, both linguistically and behaviourally, to the context of the interview and the method of assessment of the communication in those interviews. The aim was to have a better understanding of how female Emirati candidates respond to the various aspects of an international EFL speaking assessment in the context of their own cultural, social and religious constructs. Analysis of the data reveals that that there is a mismatch between the perceptions of the examiners and the participants, in several areas, and that this has the potential to affect grade outcomes, as seen in the case study interviews. The study concludes that there is scope for both a broader range in the choice of questions in speaking assessments and a need for examiners to be prepared to choose questions more judiciously, in line with the cultural context of the candidates and that this is possible without jeopardising the validity of the assessment. The findings also show that there are clear differences between the grading of face to face and audio 2 recorded interviews and that these should be considered when grading criteria are written. Overall the study contributes a variety of insights into the field of oral assessment and has implications for test writers, assessors, candidates and publishers, since, in the case of international EFL oral assessments, it appears that 'one-size' does not fit all.

No description available.

Contemporary societies are thought be characterised by a 'culture of suspicion' or 'trust malaise' - citizens are thought to be less trusting of institutions, office-holders and professions. This trend is of particular concern in relation to examinations, because examination results are a form of 'currency' and like all currencies they must be trusted by their users to hold any meaning within a social system. In England, the credibility of examinations has increasingly attracted the governments' attention. An independent watchdog of qualifications and examinations has been established, and research has been conducted into public perceptions of the reliability and validity of examinations. Whilst such research overlaps into the conceptual domain of trust, trust in examinations remains an elusive concept. This study describes the development and validation of a 20-item measure of trust in A-levels. A deductive approach to scale development was taken; meaning the construct and the scale developed simultaneously. Five stages of test construction and validation were undertaken. In stage one, six focus groups were convened with stakeholders to gain insights into the meaning of trust in the context of A-levels. In stage two, an initial item pool was reviewed by a panel of experts. In stage three, the trust-items were piloted at a sixth-form college and a tentative scale constructed. In stage four, the measurement invariance of the scale was tested. In the final stage, the temporal reliability of the scale was established. The Trust in A-levels scale displayed high internal consistency, and evidence of validity at different stages of the scale development process. Respondents also appeared to respond consistently to the scale overtime. Unfortunately, the scale did not display evidence of measurement invariance. Further research is needed to establish whether stakeholders interpret the construct trust in A-levels differently, or whether the findings were influenced by the sample composition and formatting effects.

Student feedback ratings are becoming an increasingly utilised means of measuring the quality of the student learning experience in U.K. universities. Despite this there has been little published statistical analysis on student feedback ratings using U.K. data. The research explores whether lecturers should have confidence in the validity of the student feedback ratings they receive. Guidance in the presentation and interpretation of the data is offered. This is intended to facilitate a more sophisticated understanding of the data, allowing decisions based on it to be made on a more informed basis. The research used the student feedback data collected on all taught modules (both undergraduate and postgraduate) in the Business School at Loughborough University over two academic years (October 1996–June 1998). This consisted of 305 modules and 13813 individual student feedback forms. The thesis contributes to the literature in the following ways: (1) Through illustrating the existence of heterogeneous groups of students in many Business School modules, which reflect the presence of different learning styles being utilised by Business School students, and discussing the consequences of this for: (a) the use of factor analysis on student feedback data; (b) the appropriateness of reporting the results of student feedback in the form of class averages. (2) Through illustrating the effects on student feedback ratings specific to modules taught by more than one lecturer. Two variables not previously reported in the student feedback literature are shown to influence the ratings that lecturers receive, namely: (a) the proportion of lecturing hours undertaken by a lecturer on a particular module; (b) the ratings of the lecturer(s) with whom a lecturer teaches alongside on a particular module. (3) Through examining the impact of external factors on the ratings lecturers receive. Regression analysis is used to model the influence of a set of nine predictor variables on student feedback ratings. Lecturers' ratings are shown to be significantly influenced by the level of the module, the class size and the subject area of the module. Characteristics of the lecturer, namely, the lecturer's age, rank and experience are shown to significantly influence lecturers' ratings for some aspects of lecturing.

No description available.

The present study evaluates psychometric and teacher-constructed tests on the bases of separate, but interrelated evidence obtained from: (a) A factor analysis of teachers' opinions about psychometric vs professional testing and test construction procedures. (b) Teachers' judgments of test item properties against students' actual performance on the test items. (c) Empirical evidence obtained from students' performance on tests and ratings of the students on the tests as a whole. The results are discussed in the light of current controversies. A problem encountered in comparing psychometric and professional tests was a lack of a third independent criterion for comparison. An attempt has been made to overcome the problem by combining the 3 sets of evidence. In Part I, the nature of measurement purposes, types, and their interpretations are introduced and discussed in the light of the current opposing views of subjective judgmental and empirical evidential approaches to tests and measurements. Part II concerns an investigation into teachers' attitudes toward psychometric vs professional techniques of assessment of student performances. The purpose was to obtain teachers' attitudes towards types of measurement techniques, to extract from teachers' responses to testing and test construction procedures, their criteria for evaluating test qualities; and to isolate correlates of teachers' positive attitudes towards tests. One hundred and thirty six teachers took part. Part III investigates teachers' ability to judge the suitability of items for inclusion into a test. The purpose was ii (a) to determine the degree of relationship between teachers' ratings of test item properties and empirical values obtained for the same item properties. The objective criterion against which the adequacy of teachers' judgments of the item properties were evaluated was an actual performance of students on the items. The degree of relationship between teacher ratings and students' actual performance was taken to indicate the competence of teachers to judge test item properties. (b) to determine the underlying methods used in judging by examining the relative importance given to different item properties. The degree of relationship between each property and the overall rating was taken to indicate the relative importance of each property to the judgment of the overall item quality. Twenty two teachers and 451 pupils took part in the section. Part IV investigates the relative worth of psychometric and professional tests as judged by the students themselves. The purpose was to ascertain the amount of improvement as judged by students which arises from statistical refinements of a test. Two psychometric and two professional tests were evaluated on evidence obtained from students' performances and ratings on the tests. Ratings were based on values of validity, fairness and relevance. Seventy six students took part. The results reveal (1) the criteria teachers use in judging tests for use with their students (2) the factors which influence these judgments (3) teachers' inability to judge psychometric properties (4) a conflict between the properties teachers and psychometricians value 111 (5) a lack of any distinct preference by the consumers (pupils) for either method of examination. These results are discussed in relation to theory and the practice of examining, especially in Somalia.

No description available.