Thesis (Ed.D.)--Boston University.

Thesis (Ed.D.)--Boston University

The wavefronts of high frequency (HF) radio waves received after reflection from the ionosphere exhibit both spatial non-linearities and temporal variations which limit the performance of large aperture receiving arrays. The objective of this investigation was to measure the phase and amplitude of ionospherically propagated signals in order to relate these parameters to the reflection process. This thesis describes the design and construction of a large aperture multi-element array and its implementation for wavefrot investigations. The hardware and software developed to control the equipment and to record the measurements are described. The procedures required to verify the performance of the experimental system are discussed and results are presented which demonstrate the accuracy of the measurements. The array was utilised for studies of signals received from several transmitters situated throughout Western Europe. The results obtained demonstrate the widely different behaviour of signals received over the various propagation paths and these have been related to the modal content of the received signals. Limited periods existed during which a single ionospheric mode was received and data corresponding to this condition have been compared with those which would be expected if the signal consisted of both a specular component and a cone of diffracted rays. This model is unable to explain the experimental results. Numerical models of the received signal were therefore developed. Results of these and comparisons with experimental results suggest that the measured parameters can be explained by the existence of a specular component with a varying direction of arrival (DOA), plus some contribution from random components. The experimental results indicate that the random or diffracted components normally contribute less than 10% of the received power in a single moded signal.

The wavefronts of high frequency (HF) radio waves received after reflection from the ionosphere exhibit both spatial non-linearities and temporal variations which limit the performance of large aperture receiving arrays. The first objective of this investigation was to measure the phase and amplitude of ionospherically propagated signals at several widely spaced antennas in order to relate these parameters to the reflection process. From the amplitudes and phases measured at pairs of spaced antennas, the direction of arrival (DOA) of the signal in both azimuth and elevation was determined. Furthermore. by combining the DOA and reflection height measurements the transmitter location can be estimated from a single receiving site. The second objective of this study was to investigate the ability of the system to determine DOA and transmitter locations correctly. Two seven element antenna arrays were employed with maximum apertures of 1526 m and 294 m respectively. The associated multi-channel receiving and data logging equipment is described together with a pulsed sounding system employed for mode identification. Signals received from several European transmitters exhibited widely differing behaviour and this was interpreted in terms of their modal content. For predominantly single moded signals the observations indicate that the diffracted components normally contribute less than 10% of the received power, moreover the DOA varies in both azimuth and elevation by approximately 1-2° over time periods of several minutes. The use of the smaller array for DF and SSL applications is discussed in detail. In particular, the performance of the system was severly affected by multi-moded propagation. Techniques were developed for recognising periods of single moded propagation, when accurate measurements are to be expected. Good position fixes were obtained when measurements were restricted to these periods provided accurate reflection height information was also available.

À l'origine, la recherche sur le degré de certitude (DC) et son utilisation se sont développées dans le cadre d'évaluations certificatives normees. Dans un contexte d'évaluation pour l'apprentissage, des indicateurs et des profils de réalisme sont développés pour vérifier à quel point un étudiant peut tirer profit d'un tel feedback. Dans un contexte diagnostique le degré de certitude sert à identifier les items pour lesquels l'étudiant n'a pas répondu au hasard et a surestimé ses habiletés sans le savoir. Ces "évènements critiques" nécessitent une remédiation. Cette recherche étudie le degré de réalisme en tenant compte des différences individuelles, des caractéristiques métriques et des demandes cognitives provenant des items. Les questions de recherche se formulent ainsi: Comment et dans quelle mesure l'expression du degré de certitude en la réponse choisie, et par conséquent le réalisme des étudiants, varient-ils en fonction du niveau taxonomique des items et de leurs propriétes métriques? Comment et dans quelle mesure le sexe et le niveau de performance des étudiants sont-ils liés à l'expression du degré de certitude en la réponse choisie, et par conséquent au réalisme de l'étudiant? L'échantillon visé se compose de 152 participants âgés de 18 ans et plus. Le test est de type questionnaire à choix multiples et comprend 40 items avec 4 choix de réponses. En plus des réponses aux items, l'étudiant doit exprimer son DC par rapport à ses reponses. Notre analyse permet de conclure que plus les étudiants sont performants, plus ils sont réalistes par rapport à la qualité de leurs réponses. Il ne semble pas y avoir de différences marquées entre les hommes et les femmes. Nous avons néanmoins remarqué des inégalités dans la linéarité des distributions. Plus l'item est facile, plus les étudiants sont réalistes. Il existe une relation probable entre le coefficient de discrimination d'un item et le réalisme. Le lien entre les niveaux taxonomiques des items et le réalisme des étudiants n'est pas clair et mérite d'être étudié ultérieurement en utilisant un devis construit spécifiquement à cette fin. Mots clés. Degré de certitude, regulation des apprentissages, test critérié, test diagnostique, métacognition, connaissance partielle.

<p> In general, newer educational assessments are deemed more demanding challenges than students are currently prepared to face. Two types of factors may contribute to the test scores: (1) factors or dimensions that are of primary interest to the construct or test domain; and, (2) factors or dimensions that are irrelevant to the construct, causing residual covariance that may impede the assessment of psychometric characteristics and jeopardize the validity of the test scores, their interpretations, and intended uses. To date, researchers performing item response theory (IRT)-based model simulation research in educational measurement have not been able to generate data, which mirrors the complexity of real testing data due to difficulty in separating different types of errors from multiple sources and due to comparability issues across different psychometric models, estimators, and scaling choices.</p><p> Using the context of the next generation K-12 assessments, I employed a computer simulation to generate test data under six test configurations. Specifically, I generated tests that varied based on the sample size of examinees, the degree of correlation between four primary dimensions, the number of items per dimension, and the discrimination levels of the primary dimensions. I also explicitly modeled the potential nuisance dimensions in addition to the four primary dimensions of interest, for which (when two nuisance dimensions were modeled) I also used varying degrees of correlation. I used this approach for two purposes. First, I aimed to explore the effects that two calibration strategies have on the structure of residuals of such complex assessments when the nuisance dimensions are not explicitly modeled during the calibration processes and when tests differ in testing configurations. The two calibration models I used included a unidimensional IRT (UIRT) model and a multidimensional IRT (MIRT) model. For this test, both models only considered the four primary dimensions of interest. Second, I also wanted to examine the residual covariance structures when the six test configurations vary. The residual covariance in this case would indicate statistical dependencies due to unintended dimensionality. </p><p> I employed Luecht and Ackerman&rsquo;s (2017) expected response function (ERF)-based residuals approach to evaluate the performance of the two calibration models and to prune the bias-induced residuals from the other measurement errors. Their approach provides four types of residuals that are comparable across different psychometric models and estimation methods, hence are &lsquo;metric-neutral&rsquo;. The four residuals are: (1) e0, which comprises the total residuals or total errors; (2) e1, the bias-induced residuals; (3) e2, the parameter-estimation residuals; and, (4) e3, the estimated model-data fit residuals.</p><p> With regard to my first purpose, I found that the MIRT model tends to produce less estimation error than the UIRT model on average (e2MIRT is less than e2UIRT) and tends to fit the data better than the UIRT model on average (e3MIRT is less than e3UIRT). With regard to my second research purpose, my analyses of the correlations of the bias-induced residuals provide evidence of the large impact of the presence of nuisance dimension regardless of its amount. On average, I found that the residual correlations increase with the presence of at least one nuisance dimension but tend to decrease with high item discriminations.</p><p> My findings shed light on the need to consider the choice of calibration model, especially when there are some intended and unintended indications of multidimensionality in the assessment. Essentially, I applied a cutting-edge technique based on the ERF-based residuals approach (Luecht &amp; Ackerman, 2017) that permits measurement errors (systematic or random) to be cleanly partitioned, understood, examined, and interpreted&mdash;in-context and in relative to difference-that-matters criteria&mdash;regardless of the choice of scaling, calibration models, and estimation methods. For that purpose, I conducted my work based on the context of the complex reality of the next generation K-12 assessments and based on my effort to maintain adherence to the established educational measurement standards (American Educational Research Association (AERA), the American Psychological Association (APA), and the National Council on Measurement in Education (NCME), 1999, 2014); International Test Commission (ITC) (ITC, 2005a, 2005b, 2013a, 2013b, 2014, 2015)).</p><p>

No description available.

Despite the prevalence of student learning objectives (SLOs) in teacher evaluation systems throughout the United States, research on the validity of student and teacher SLO scores used for high-stakes decisions is lacking. For this reason, this dissertation is comprised of two chapters that examine student and teacher-level SLO performance data from select districts in one Race to the Top state. In Chapter 1, I describe the quality of student assessment data and the comparability of student scores across alternative growth targets. I find that in the first year of implementation, assessments from half of the courses in the sample contained indicators of poor data quality, including anomalous score distributions and small to negative correlations between student prescores and postscores. However, in the second year of implementation, when student SLO performance is incorporated into final teacher evaluation scores, far fewer assessments contained anomalous score distributions, and there is no evidence to suggest manipulation of student scores. In addition to the assessments, the choice of student growth target does have an impact on the comparability of student and teacher scores across districts and years. Chapter 2 describes the validity and reliability of teacher SLO scores. I find that while teacher SLO scores are moderately stable across courses, they are not stable over time, likely due to changes made to the assessments and targets used to determine student SLO scores. Further, for teachers with both SLO scores and an alternative metric of performance based on student growth, the two metrics do not converge. Finally, teachers in courses with higher average student prescores and lower proportions of students with disabilities have slightly higher SLO scores. In general, results on teacher SLO scores were similar to those found with value-added based metrics of teacher performance. Findings from both chapters suggest that improvement in the quality of the assessments administered as well as greater consistency in the growth targets assigned to students, both within districts over time and across districts, will improve the validity of student and teacher SLO scores in this state. / Quantitative Policy Analysis in Education

Recent state and federal policy initiatives have led to the development of a multitude of statistics intended to measure school performance. Of these, statistics constructed from student test scores number among both the most widely-used and most controversial. In many cases, researchers and policymakers alike are not fully aware of the ways in which these statistics may lead to unjustified inferences regarding school effectiveness. A substantial amount of recent research has attempted to remedy this, although much remains unknown. This thesis seeks to contribute to these research efforts via three papers, each examining how a commonly-employed accountability statistic may be influenced by factors unrelated to student proficiency or school effectiveness. The first paper demonstrates how the discrete nature of test scores leads to biased estimates of changes in the percentage of “proficient” students between any two given years and examines estimators that provide better recovery of this parameter. The second paper makes use of a state-wide natural experiment to show that a change in testing program, from paper-and-pencil to computer-adaptive, may cause apparent changes in achievement gaps even when relative student proficiencies have remained constant. The third paper examines “growth-based” accountability metrics based on vertically-scaled assessments, showing that certain types of metrics based on gain scores can be modeled via nonlinear transformations of the underlying vertical scale. It then makes use of this result to investigate the potential magnitude of impacts of such transformations on growth-based school accountability ratings.

Typescript, etc.