Thesis (Ph. D.)--University of Texas at Austin, 2002. / Vita. Includes bibliographical references. Available also from UMI Company.

Thesis (Ph. D.)--University of Texas at Austin, 2005. / Supervisor: Hua-Hua Chang. Vita. Includes bibliographical references.

Thesis (M.S. in Operations Research)--Naval Postgraduate School, June 2006. / Thesis Advisor(s): Robert F. Dell. "June 2006." Includes bibliographical references (p. 35-37). Also available in print.

Thesis (Ph.D.)--University of Nebraska-Lincoln, 2006. / Title from title screen (site viewed June 8, 2007). PDF text: 141 p. : col. ill. ; 0.80Mb. UMI publication number: AAT 3239362. Includes bibliographical references. Also available in paper, microfilm and microfiche formats.

In this thesis, we firstly develop a model-adaptive checking method for partially parametric single-index models, which combines the advantages of both dimension reduction technique and global smoothing tests. Besides, we propose a dimension reduction-based model adaptive test of heteroscedasticity checks for nonparametric and semi-parametric regression models. Finally, to extend our testing approaches to nonparametric regressions with some restrictions, we consider significance testing under a nonparametric framework. In Chapter 2, “Model Checking for Partially Parametric Single-index Models: A Model-adaptive Approach", we consider the model checking problems for more general parametric models which include generalized linear models and generalized nonlinear models. We develop a model-adaptive dimension reduction test procedure by extending an existing directional test. Compared with traditional smoothing model checking methodologies, the procedure of this test not only avoids the curse of dimensionality but also is an omnibus test. The resulting test is omnibus adapting the null and alternative models to fully utilize the dimension-reduction structure under the null hypothesis and can detect fully nonparametric global alternatives, and local alternatives distinct from the null model at a convergence rate as close to square root of the sample size as possible. Finally, both Monte Carlo simulation studies and real data analysis are conducted to compare with existing tests and illustrate the finite sample performance of the new test. In Chapter 3,Heteroscedasticity Checks for Nonparametric and Semi-parametric Regression Model: A Dimension Reduction Approach", we consider heteroscedasticity checks for nonparametric and semi-parametric regression models. Existing local smoothing tests suffer severely from the curse of dimensionality even when the number of covariates is moderate because of use of nonparametric estimation. In this chapter, we propose a dimension reduction-based model adaptive test that behaves like a local smoothing test as if the number of covariates is equal to the number of their linear combinations in the mean regression function, in particular, equal to 1 when the mean function contains a single index. The test statistic is asymptotically normal under the null hypothesis such that critical values are easily determined. The finite sample performances of the test are examined by simulations and a real data analysis. In Chapter 4，Dimension Reduction-based Significance Testing in Nonparametric Regression", as nonparametric techniques need much less restrictive conditions than those required for parametric approaches, we consider to check nonparametric regressions with some restrictions under sufficient dimension reduction structure. A dimension-reduction-based model-adaptive test is proposed for significance of a subset of covariates in the context of a nonparametric regression model. Unlike existing local smoothing significance tests, the new test behaves like a local smoothing test as if the number of covariates is just that under the null hypothesis and it can detect local alternative hypotheses distinct from the null hypothesis at the rate that is only related to the number of covariates under the null hypothesis. Thus, the curse of dimensionality is largely alleviated when nonparametric estimation is inevitably required. In the cases where there are many insignificant covariates, the improvement of the new test is very significant over existing local smoothing tests on the significance level maintenance and power enhancement. Simulation studies and a real data analysis are conducted to examine the finite sample performance of the proposed test. Finally, we conclude the main results and discuss future research directions in Chapter 5. Keywords: Model checking; Partially parametric single-index models; Central mean subspace; Central subspace; Partial central subspace; Dimension reduction; Ridge-type eigenvalue ratio estimate; Model-adaption; Heteroscedasticity checks; Significance testing.

Throughout the social sciences, tests have been used for two primary - and different - purposes: a) to estimate where an examinee is located on an ability/trait continuum (e.g., intelligence tests), or b) to classify an examinee as either above or below a particular point on the ability continuum (e.g., criterion-referenced tests). From a psychometric perspective, the scoring procedure of the test should reflect the purpose for which the test is being used. From a practical perspective, the administration procedure should be as efficient as possible. The Myers-Briggs Type Indicator (MBTI; Briggs & Myers, 1976) is a personality inventory designed to classify examinees according to four bipolar dimensions. Although the MBTI is quite popular within corporate America, critics have threatened the validity of the MBTI with two seemingly contradictory faults: the test is too unreliable and too long. The purpose of this study was to examine the degree to which using an item response theory (IRT) / Ph. D.

It is not uncommon to use unidimensional item response theory (IRT) models to estimate ability in multidimensional data. Therefore it is important to understand the implications of summarizing multiple dimensions of ability into a single parameter estimate, especially if effects are confounded when applied to computerized adaptive testing (CAT). Previous studies have investigated the effects of different IRT models and ability estimators by manipulating the relationships between item and person parameters. However, in all cases, the maximum information criterion was used as the item selection method. Because maximum information is heavily influenced by the item discrimination parameter, investigating a-stratified item selection methods is tenable. The current Monte Carlo study compared maximum information, a-stratification, and a-stratification with b blocking item selection methods, alone, as well as in combination with the Sympson-Hetter exposure control strategy. The six testing conditions were conditioned on three levels of interdimensional item difficulty correlations and four levels of interdimensional examinee ability correlations. Measures of fidelity, estimation bias, error, and item usage were used to evaluate the effectiveness of the methods. Results showed either stratified item selection strategy is warranted if the goal is to obtain precise estimates of ability when using unidimensional CAT in the presence of two-dimensional data. If the goal also includes limiting bias of the estimate, Sympson-Hetter exposure control should be included. Results also confirmed that Sympson-Hetter is effective in optimizing item pool usage. Given these results, existing unidimensional CAT implementations might consider employing a stratified item selection routine plus Sympson-Hetter exposure control, rather than recalibrate the item pool under a multidimensional model.

Multistage tests (MSTs) have received renewed interest in recent years as an effective compromise between fixed-length linear tests and computerized adaptive test. Most MSTs studies scored the assessments based on item response theory (IRT) methods. Many assessments are currently being developed as mixed-format assessments that administer both standalone items and clusters of items associated with a common stimulus called testlets. By the nature of a testlet, a natural dependency occurs between the items within the testlet that violates the local independence of items. Local independence is a fundamental assumption of the IRT models. Using dichotomous IRT methods on a mixed-format testlet-based assessment knowingly violates local independence. By combining the score points within a testlet, researchers have successfully applied polytomous IRT models. However, the use of such models loses information by not using the unique response patterns provided by each item within a testlet. The three-parameter logistic testlet response theory (3PL-TRT) model is a measurement model developed to retain the uniqueness in response patterns of each item, while accounting for the local dependency exhibited by a testlet, or testlet effect. Because few studies have examined mixed-format MSTs administration under the 3PL-TRT model, the dissertation performed a simulation to investigate the administration of a mixed-format testlet based MSTs under the 3PL-TRT model. Simulee responses were generated based on the 3PL-TRT calibrated item parameters from a real large-scale passage based standardized assessment. The manipulated testing conditions considered four panel designs, two test lengths, three routing procedures, and three conditions of local item dependence. The study found functionally no bias across testing conditions. All conditions showed adequate measurement properties, but a few differences did occur between some of the testing conditions. The measurement precision was impacted by panel design, test length and the magnitude of local item dependence. The three-stage MSTs consistently illustrated slightly lower measurement precision than the two-stage MSTs. As expected, the longer test length conditions had better measurement precision than the shorter test length conditions. Conditions with the largest magnitude of local item dependency showed the worst measurement precision. The routing procedure had little impact on the measurement effectiveness. / text

After the stability and sensitivity of FLOR were investigated, the application of it in the CAT environment had become the main concern. The present studies found that both the test length and the number of exposed items affect the final value of FLOR. In the fixed length CAT, the FLOR has a much stronger sensitivity than lz and CUSUM in detecting item pre-knowledge. The sensitivity of FLOR in the fixed length CAT was the same as that in the fixed length fixed items test. If the test length could vary, the sensitivity of FLOR in CAT would be slightly weakened. The Adjusted FLOR index could increase the sensitivity. Concerning about the effect of ability on the sensitivity of FLOR in CAT, it was found that the abilities of the test takers in CAT did not affect the sensitivity of FLOR and Adjusted FLOR. / Item response theory is a modern test theory. It focuses on the performance of each item. Under this framework, the performance of test takers on a test item can be predicted by a set of abilities. The relationship between the test takers' item performances and the set of abilities underlying item performances can be described by a monotonically increasing function called an item characteristic curve. Due to various personal reasons, the performances of the test takers may depart from the response patterns predicted by the underlying test model. In order to calculate the extent of departure of these aberrant response patterns, a number of methods have been developed under the theme "person-fit statistics". The degree of aberration is calculated as an index called person-fit index. Inside the computerized adaptive testing (CAT), test takers with different abilities will answer different numbers of questions and the difficulties of the items administered to them are usually clustered at the abilities of the test takers. Due to this reason, the application of person-fit indices in the computerized adaptive testing environment to measure misfit is difficult. / The present study also found that FLOR has a much superior sensitivity over other indices in detecting item pre-knowledge. Concerning about the sensitivity over different abilities of test takers, it was found that the sensitivity of FLOR was the highest among low ability test takers and the weakest among strong ability test takers in the fixed length and fixed items tests. However, the sensitivities of FLOR became the same among different abilities of test takers if items with difficulties matching their abilities were used in the tests. The number of beneficiaries among the test takers did not affect the sensitivity of FLOR. Moreover, in a simulation to test the differentiating power of FLOR, it was found that FLOR could differentiate item pre-knowledge from other reasons of personal misfits (test anxiety, player, random response and challenger) effectively. / The present study assessed the stability of FLOR over other variables, which were unrelated to item pre-knowledge. It found that FLOR was stable over the discrimination and difficulty parameters of test items. It was also stable over positions of the exposed items in the test and the initial assignment of prior probability of item pre-knowledge. However, the asymptotes (guessing factor) and the probabilities of item exposure did affect the final values of FLOR seriously. / The present study used the hf plot to access the sensitivity of the person-fit indices. hf plot is a plot of hit rate against false alarm rate. For a higher hit rate, usually a higher false alarm rate is followed. hf plot provides a good tools for comparison between indices by inspection of the speed of rise of the curves. A sensitive index should give a faster rise of the curve. In this study, sensitivity of an index was defined as the speed of rise of the hf plot, which is represented by a parameter hftau estimated from the data obtained from hf plot. / When the frequent accesses to the item bank has become feasible, test takers may memorize blocks of test items and share these items with future test takers. Individuals with prior knowledge of some items may use that information to get high scores, in the sense that their test scores have been artificially inflated. FLOR is an index of posterior log-odds ratio used for detecting the use of item pre-knowledge. It can be applied both in the fixed item, fixed length test and the CAT environment. It is a model-based index in which aberrant models are defined in the situation of item pre-knowledge. FLOR describes the likelihood that a response pattern arises from the aberrant models. / Hui Hing-fai. / Adviser: Kit-tai Hau. / Source: Dissertation Abstracts International, Volume: 70-09, Section: A, page: . / Thesis (Ed.D.)--Chinese University of Hong Kong, 2008. / Includes bibliographical references (leaves 108-111). / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Electronic reproduction. [Ann Arbor, MI] : ProQuest Information and Learning, [200-] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Abstracts in English and Chinese. / School code: 1307.

The advantages of administering an adaptive test battery, a collection of multiple adaptive subtests that are specifically tailored to examinees’ abilities, include shortening the subtest length and maintaining the accuracy of individual subtest scores. The test battery can incorporate a range of subjects, though this study focused primarily on Math and Reading. This study compared different ways of incorporating collateral information (CI), supplementary information beyond examinees’ current test performance, under two frameworks (Unidimensional and Multidimensional computerized adaptive testing). It also investigated the impact of subtest intercorrelations (the relationship between an examinee’s test scores), as well as the sequences of subtest administration on ability estimation in a variable-length adaptive battery. Practical issues including content constraints and item exposure control were also considered. Findings showed that the CI methods improved measurement efficiency with an acceptable level of measurement precision. The CI was more beneficial when associated with higher intercorrelations among the subtests. Also, the CI was found to be advantageous during the early stages of the subtests which were not taken first. Therefore, the CI may improve the examinee experience by administering items more aligned with their abilities. In addition, the CI should reduce costs for testing organizations by requiring fewer items and possibly saving seat time, while still providing reliable scores. The results should help practitioners decide whether the use of the CI is worthwhile under their particular testing situation.