The main objective of this study was to investigate the improvement of the accuracy of small sample equating, which typically occurs in teacher certification/licensure examinations due to a low volume of test takers per test administration, under the Non-Equivalent Groups with Anchor Test (NEAT) design by combining previous and current equating outcomes using a meta-analysis technique. The proposed meta-analytic score transformation procedure was called "meta-equating" throughout this study. To conduct meta-equating, the previous and current equating outcomes obtained from the chosen equating methods (ID (Identity Equating), Circle-Arc (CA) and Nominal Weights Mean (NW)) and synthetic functions (SFs) of these methods (CAS and NWS) were used, and then, empirical Bayesian (EB) and meta-equating (META) procedures were implemented to estimate the equating relationship between test forms at the population level. The SFs were created by giving equal weight to each of the chosen equating methods and the identity (ID) equating. Finally, the chosen equating methods, the SFs of each method (e.g., CAS, NWS, etc.), and also the META and EB versions (e.g., NW-EB, CA-META, NWS-META, etc.) were investigated and compared under varying testing conditions. These steps involved manipulating some of the factors that influence the accuracy of test score equating. In particular, the effect of test form difficulty levels, the group-mean ability differences, the number of previous equatings, and the sample size on the accuracy of the equating outcomes were investigated. The Chained Equipercentile (CE) equating with 6-univariate and 2-bivariate moments log-linear presmoothing was used as the criterion equating function to establish the equating relationship between the new form and the base (reference) form with 50,000 examinees per test form. To compare the performance of the equating methods, small numbers of examinee samples were randomly drawn from examinee populations with different ability levels in each simulation replication. Each pairs of the new and base test forms were randomly and independently selected from all available condition specific test form pairs. Those test forms were then used to obtain previous equating outcomes. However, purposeful selections of the examinee ability and test form difficulty distributions were made to obtain the current equating outcomes in each simulation replication. The previous equating outcomes were later used for the implementation of both the META and EB score transformation procedures. The effect of study factors and their possible interactions on each of the accuracy measures were investigated along the entire-score range and the cut (reduced)-score range using a series of mixed-factorial ANOVA (MFA) procedures. The performances of the equating methods were also compared based on post-hoc tests. Results show that the behaviors of the equating methods vary based on the each level of the group ability difference, test form difficult difference, and new group examinee sample size. Also, the use of both META and EB procedures improved the accuracy of equating results on average. The META and EB versions of the chosen equating methods therefore might be a solution to equate the test forms that are similar in their psychometric characteristics and also taken by new form examinee samples less than 50. However, since there are many factors affecting the equating results in reality, one should always expect that equating methods and score transformation procedures, or in more general terms, estimation procedures may function differently, to some degree, depending on conditions in which they are implemented. Therefore, one should consider the recommendations for the use of the proposed equating methods in this study as a piece of information, not an absolute guideline, for a rule of thumbs for practicing small sample test equating in teacher certification/licensure examinations. / A Dissertation submitted to the Department of Educational Psychology and Learning Systems in partial fulfillment of the requirements for the degree of Doctor of Philosophy. / Fall Semester 2015. / October 28, 2015. / Collateral Information, Empirical Bayesian, Meta-Analysis, NEAT design, Small Samples, Test Equating / Includes bibliographical references. / Insu Paek, Professor Directing Dissertation; Victor Patrangenaru, University Representative; Russell Almond, Committee Member; Alysia Roehrig, Committee Member.

The primary purpose of this study was to determine whether an instructional materials formative evaluation model that incorporated both payoff and intrinsic criteria resulted in more effective training materials in an instructor-led environment than a model relying on intrinsic criteria alone. Two revised versions of materials were developed and delivered in a classroom setting. Version X$\sb1$ was revised using intrinsic criteria only, and Version X$\sb2$ on the basis of both intrinsic criteria and student data. / The study focused on the effects of these two versions in both a highly and less-controlled environment. The dependent measures were learner performance and attitude, trainer attitude, trainer effectiveness, and trainee intent to use the skills on the job. / Due to low test reliability in both environments, the posttest results were not interpretable. Participant intent to use course skills was not interpretable due to a ceiling effect. / Trainers' attitudes toward the course were more positive for Version X$\sb2$. The results for trainer effectiveness were mixed. In the highly controlled environment, the ratings for this indicator were higher for Version X$\sb2$ for one trainer, but not the other. In the less-controlled environment, the trainers for Version X$\sb2$ were rated slightly higher in most categories for both modules. A comparison of trainer delivery revealed that better trainers performed equally well delivering bother versions. Some of the weaker trainers were rated significantly higher in their delivery of Version X$\sb2$ for some categories. / Learner attitudes were more positive for Version X$\sb2$ in the highly controlled setting. In the less-controlled environment, the ratings for Version X$\sb2$ were mixed. These results indicate that the use of payoff data as a basis for revision decisions is likely to result in instruction that is better received by trainers and learners. / Source: Dissertation Abstracts International, Volume: 54-02, Section: A, page: 0416. / Major Professor: Robert A. Reiser. / Thesis (Ph.D.)--The Florida State University, 1993.

This study's purpose was to investigate the dimensionality of a test containing multiple-choice (MC) and student-produced-response (SPR) item formats. Parallel tests (SPR and MC) were developed to assess skills for Florida's 1994 High School Competency Test (HSCT). Each test contained nine items selected or prepared parallel to those in the contrasting test and corresponding to a HSCT skill. Both SPR and MC tests were administered to 556 tenth graders enrolled in Florida high schools. / Five analyses attempted to determine the dimensionality of the test. Exploring the test's unidimensionality, confirmatory factor analyses were performed, followed by higher order factor analyses, and a modified parallel analysis. To explore potential content factors, principal component analyses of the total item set and for each section of the test (SPR and MC) were conducted, as well as a multi-dimensional scaling analysis. / Analyses indicate, to varying degrees, that the test is sufficiently unidimensional for measurement purposes. Results of the second order analysis and the modified parallel analysis offer a qualified yes to whether this test could be considered unidimensional enough to be analyzed using IRT procedures. No format factors were present in the study. The question regarding similarity of content for the two item formats resulted in unexpected process factors for both item formats, however, findings were tentative. / Source: Dissertation Abstracts International, Volume: 54-03, Section: A, page: 0799. / Major Professor: Jacob G. Beard. / Thesis (Ph.D.)--The Florida State University, 1993.

This study was designed to compare traditional and nontraditional full-time degree-seeking students' regression equations and validity coefficients across five Florida post-secondary institutions. The single multiple regression method with dummy variables was selected to compare the two groups' regression planes across institutions (FSU, UF, UCF, USF, and FAMU), within age groups, and between age groups within institutions. The multiple partial statistic was selected to test for the interaction effect between the two indicator variables (Age and Institutions) and the three main effects (HSGPA, SATV and SATQ). The across institutions and within age groups validity coefficients variability of the high school grade point average and the Scholastic Aptitude Test verbal and quantitative scores was investigated, using meta-analysis methodology. / A sample of 883 students was retrieved from the State University System (SUS) students' files. With the exception of age, all the students were selected to be equivalent on the following characteristics: full-time enrollment, first time in college, degree-seeking, and accepted under regular admission policies. This selection procedure limited the sample size of the nontraditional group of students and, therefore, generalizations regarding the results of this study should be made cautiously. / It was concluded that a common prediction system was not practical and that a separate prediction system should be developed for each of the two groups compared within the five postsecondary institutions included in this study. The findings showed possible systematic overprediction or underprediction of the nontraditional students' performance in college when using a traditional student-derived regression equation to predict nontraditional students' performance. It was also apparent that nontraditional students' high school grade point average and traditional students' Scholastic Aptitude Test quantitative validity coefficients varied from institution to institution. There was no variation across institutions or within age groups of the Scholastic Aptitude Test verbal validity coefficients. As expected, high school grade point average was a better predictor of traditional students' performance in college, as Scholastic Aptitude Test verbal was for the nontraditional students. The average validity coefficients of the nontraditional students were, in all but one instance, lower than for the traditional group. It was recommended that differential validity and regression systems for traditional and nontraditional students be routinely studied. / Source: Dissertation Abstracts International, Volume: 47-05, Section: A, page: 1706. / Thesis (Ph.D.)--The Florida State University, 1986.

The issue of differential item functioning (DIF) in standardized tests has increasingly generated interest in the measurement and testing communities. An item is said to contain DIF if examinees of equal proficiency from different gender, ethnic or other groups have an unequal probability of responding correctly to the item. / Although the majority of DIF research has focused on its identification through statistical procedures, recent published studies have addressed the arguably more important issue of causes of DIF. To date, however, most studies of causes of DIF have been concerned with post-secondary situations. / The present study identified sources of DIF within a widely used secondary school achievement battery. Responses on the Vocabulary and Reading Comprehension sections of the Stanford Achievement Test were obtained from 1580 White and 3223 Hispanic eighth graders in Dade County, Florida. A quantitative technique was used to detect items exhibiting DIF. Once these items were identified, a review panel of expert bilingual judges examined them in terms of linguistic and cultural factors associated to DIF between Hispanics and Whites. / Results suggest that, when comparing Hispanic and White students of the same ability, the use in test items of true cognate words frequently used in Spanish will favor the performance of Hispanics. In contrast, several conditions may favor Whites, including the use of non true cognate words infrequently used in English, linguistic complexity, idiomatic expressions derived from technical language, and poetry. Moreover, words and phrases with a special cultural meaning for one of the groups will favor members of that group (White or Hispanic) and, the use of settings for which Hispanic students are likely to be less familiar with will favor the performance of Whites. / Findings from the study confirm the need to stress certain areas in the instruction of Hispanic students. Even though DIF is not necessarily indicative of item bias, its appearance in test items might be a sign of instructional deficiencies. Findings also should alert test developers to distinguish between construct-relevant DIF and DIF associated with test invalidity. / Source: Dissertation Abstracts International, Volume: 54-09, Section: A, page: 3411. / Major Professor: Albert C. Oosterhof. / Thesis (Ph.D.)--The Florida State University, 1993.

The purpose of this dissertation is to create a psychometric instrument, the Attorney Communication Skills Instrument (ASCI), which measures the communication strengths and weaknesses exhibited by attorneys during their client interviews. / The Inventory is composed of three tests, the ACSI-Climate, ACSI-Listening, and ACSI-Nonverbal, which were assessed in three empirical surveys. The first assessment involved a mail survey to 500 randomly selected members of The Florida Bar Association. The second assessment involved the distribution of 333 copies of each test to attorneys attending the 1993 Florida Bar Association's Annual Convention. The third survey was mailed to 3900 randomly selected members of The America Bar Association's Law Practice Management Section. / The results indicate a highly defensible psychometric inventory. The results evidence a strong set of homogeneous, unidimensional items which appear to reliably and validly measure the communication constructs of establishing a supportive climate, listening, and nonverbal communication skills. The resulting items may now be used to test the presence of certain communication skills and how effectively attorneys use those skills when interviewing and interacting with their clientele. / Source: Dissertation Abstracts International, Volume: 55-04, Section: A, page: 0800. / Major Professor: Marilyn Young. / Thesis (Ph.D.)--The Florida State University, 1994.

The purpose of the current study is to provide evidence of the possible repercussions of different teacher certification pathways on student achievement that can inform policy in order to improve the instruction students receive. In the current context of accountability, and with a teacher's effectiveness often defined by his or her students' test scores, policies should be based on empirical evidence, and not just political or ideological perspectives (Cochran-Smith et al., 2012). I used a hierarchical linear model (HLM) to investigate the relationship between teachers' pathway to certification, experience, and effectiveness for fourth through eighth grade English Language Arts (ELA) and math teachers who received a certification to teach in the state of Florida and taught at only one school in the 2016-17 school year. All data were retrieved from FDOE. The study included 4,967 math teachers and 3,567 ELA teachers. Effectiveness was based on a Value-added model (VAM) score. The ELA and math VAM scores used in this analysis include the same predictor variables as those used by the Florida Department of Education (FDOE). However, this analysis used a one-year aggregate score, while FDOE provides a suggested VAM category based on a three-year aggregate score and standard errors. The one-year aggregate was best suited for the current analysis because it standardizes the amount of years included for beginning and experienced teachers (the three-year aggregate can include anywhere from one to three years of teaching), and it allows teachers who have switched schools within the past three years to be included in the model. The analysis controlled for experience, the percentage of courses taught infield, and the number of general knowledge Florida Teacher Certification Exam (FTCE) and subject specific FTCE tests taken before passing at the teacher level, and school grades at the school level. No significant differences in teacher effectiveness in math were found for the different pathways. In ELA, however, there was a significant interaction between certification pathway and experience. Completers of District Professional Development Certification Programs (PDCP) were found to be more effective in ELA in their first three years of teaching than completers of the traditional Initial Teacher Preparation (ITP) programs or Educator Preparation Institute (EPI) programs. Findings provide evidence that allowing for alternative pathways to certification increases the number of effective teachers entering the workforce. / A Dissertation submitted to the Department of Educational Psychology and Learning Systems in partial fulfillment of the requirements for the degree of Doctor of Philosophy. / Fall Semester 2018. / October 26, 2018. / certification, education, effectiveness, pathway, preparation, teacher / Includes bibliographical references. / Alysia D. Roehrig, Professor Directing Dissertation; Elizabeth M. Jakubowski, University Representative; Jeannine E. Turner, Committee Member; Qian Zhang, Committee Member.

The purpose of this study was to investigate the effectiveness of an investigator designed sight-reading method (SRM) on the sight-reading skills of college music majors. The SRM concentrated on two factors relating to sight-reading: rhythmic accuracy and scanning. / Freshmen and sophomore instrumental music majors (N = 29) were given a pretest using the Watkins-Farnum Performance Scale (1954) during the first week of Fall Semester 1994. Subjects were divided into two groups, experimental (n = 14, treatment with SRM) and control (n = 15, no treatment). The treatments consisted of weekly fifteen-minute lessons using the SRM for a period of twelve weeks. / A posttest was given following the treatment period. Results of the pretest and posttest were compared in order to answer the following questions: Was there a significant difference in the pretest and posttest scores of the experimental group? Was there a significant difference in the pretest and posttest scores of the control group? Was there a significant difference in the experimental group's score change and the control group's score change? A two-tailed "t" test was utilized to analyze the data. It was concluded that both the experimental group and control group exhibited significant increases in sight-reading scores. However, there was no significant difference between the experimental group's increase and the control group's increase. It was concluded from these findings that the SRM did not produce any more of an increase in sight-reading scores than did the traditional activities of a music major. / Source: Dissertation Abstracts International, Volume: 56-04, Section: A, page: 1283. / Major Professor: James Croft. / Thesis (Ph.D.)--The Florida State University, 1995.

Thirty-eight seven year old children were tested to evaluate the effects of response format and stimulus expectancy (based on subject's label of the test item) on picture and word identification test scores and response latencies. Results of the study indicated that there are significant differences between the picture and word identification tasks. Response latencies were longer and identification scores were lower for the picture identification task. Fifty-nine percent of the test items that were labeled incorrectly were also identified incorrectly. Furthermore, the label given to the test items, by the individual, was found to have a significant effect on the overall test results. Results of this study indicate that the problems encountered when utilizing picture-identification tests stem from the pictures that comprise the test and not the picture test format itself. / Source: Dissertation Abstracts International, Volume: 49-12, Section: A, page: 3696. / Major Professor: William H. Haas. / Thesis (Ph.D.)--The Florida State University, 1988.

The purpose of this study is to examine whether individual's difference in A-E style, one of the cognitive style constructs, is a factor influencing how individuals perform on the task of solving insight problems, how individuals apply past experience to solving similar problems, and the extent to how much set effect or fixation that individuals will encounter. The A-E (assimilator-explorer) style suggests that individuals with a tendency to follow rules and search for past experience when solving problems can be labeled as assimilators; while individuals with a tendency to use the trial-and-error method can be labeled as explorers. Subjects were 87 undergraduate students and were classified as either assimilators or explorers according to their scores on the A-E Inventory, an instrument used to measure the A-E style construct. Then three insight problem-solving tests were administrated to all subjects. All the problems used in the three tests share the surface similarities, features which make two problems look like a similar one, but only the problems in the first two tests also share the structural similarities, features which allow two problems to be solved by the same strategies. It was predicted that: (a) the explorers would do better than the assimilators on test 1 because the explorers were more prone to think from different perspectives and thus more likely to come up with correct answers for novel problems; (b) the assimilators would do better than the explorers on test 2 because the assimilators were more capable of applying past experience to similar situations; while the explorers were less capable of doing so; and (c) the explorers would perform better than the assimilator on test 3 because the assimilator were more likely to rely on past experience thus prone to encounter set effects and fixation which degraded problem solving performance. / However, the findings showed that the performance of two groups was comparable and, therefore, none of the three hypotheses were supported. Possible reasons for why the hypotheses were not supported are discussed. Recommendations for future research are also presented. / Source: Dissertation Abstracts International, Volume: 56-08, Section: A, page: 3054. / Major Professor: Walter Wager. / Thesis (Ph.D.)--The Florida State University, 1995.