Thesis (MA) -- University of Stellenbosch, 2001. / ENGLISH ABSTRACT: The purpose of this study was to determine the applicability of a nomological model for the assessment of psychological measurement validity with reference to personnel selection. The model, which can be traced to Thorndike (1949), Campbell and Fiske (1959), Nunnally (1967, cited in Duvenage, 1990), Guion (1974, cited in Duvenage, 1990), Binning and Barrett (1989) and Duvenage (1990), provides the evidential bases for two empirically founded validation strategies, namely criterion- and constructrelated validity. Theoretically, the Nomological Model for Psychological Measurement Validity Assessment proved to be applicable for validating that a measure is able to predict job performance. An easily understandable empirical investigation was required to verify the scientific utility of the model for selection validation. The model was used to validate the procedure for selecting machine operators in a large food manufacturer. The investigation illustrated that by utilising Route 1 of the nomological model, evidence of criterion-related validity, as described by legal and professional standards, is generated. Furthermore, the study indicated that by utilising Route 2 of the model, evidential requirements of construct-related validity as set by legal and professional standards are met. The results of the study proved that the Nomological Model for Psychological Measurement Validity Assessment is theoretically, professionally, legally and practically applicable for the validation of personnel selection procedures. / AFRIKAANSE OPSOMMING: Die doel van hierdie studie was om die toepasbaarheid van 'n nomologiese model vir die bepaling van die geldigheid van sielkundige meting ten opsigte van personeelkeuring vas te stel. Die model, wat sy oorsprong in die werk van Thorndike (1949), Campbell en Fiske (1959), Nunnally (1967, aangehaal uit Duvenage, 1990), Guion (1974, aangehaal uit Duvenage, 1990), Binning en Barrett (1989) en Duvenage (1990) het, verskaf getuienis basisse vir twee ernpirles gefundeerde valideringsstrategiee, naamlik kriterlum- en konstruk-verwante geldigheid. Daar is teoretiese bewyse gevind vir die toepasbaarheid van die Nomologiese Model vir Sielkundige Meting Geldigheids Raming vir die validering van metingsvoorspelbaarheid van werksprestasie. 'n Maklik verstaanbare empmese ondersoek was nodig om die model se wetenskaplike nut vir keuringsgeldigheid te bepaal. Die model was gebruik vir die validering van keuringsprosedure-geldigheid van masjienoperateurs in 'n groot voedselverwerkingsmaatskappy. Die ondersoek het ge"illustreer dat deur die toepassing van Roete 1 van die nomologiese model, bewyse gegenereer is vir kriterium-verwante geldigheid, 5005 voorgeskryf deur relevante wetgewing en professionele standaarde. Verder, deur die toepassing van Roete 2 van hierdie model, word voldoende bewyse vir konstruk-verwante geldigheid, 5005 voorgeskryf deur wetlike en professionele standaarde, verkry. Die resultate van die studie bewys dat die Nomologiese Model vir Sielkundige Meting Geldigheids Raming teoreties, professioneel, wettig en prakties toepasbaar is vir die validering van personeel keu ri ngsprosed ures.

Dissertation (D.Phil.) -- University of Stellenbosch, 1999. / ENGLISH ABSTRACT: The conditions under which selection procedures are typically validated and those prevailing at the eventual use of a selection procedure normally differ to a sufficient extent to challenge the relevance of the validation research evidence. Statistical corrections to the validity coefficient are generally available. The remainder of the argument in terms of which a selection procedure is developed and justified could, however, also be biased by any discrepancy between the conditions under which the selection procedure is simulated and those prevailing at the eventual use of the selection procedure. Relatively little concern, however, seem to exist for the transportability of the decision function derived from the selection simulation or the descriptions/ assessments of selection decision utility and fairness. This seems to be a somewhat strange state of affairs. The external validity problems with validation designs are reasonably well documented. It is thus not as if the psychometric literature is unaware of the problem of generalizing validation study research fmdings to the eventual area of application. The decision function is probably the pivot of the selection procedure in that it firstly captures the underlying performance theory, but more importantly from a practical perspective, because it guides the actual accept and reject choices of applicants. Restricting the statistical corrections to the validity coefficient would leave the decision function unaltered even though it might also be distorted by the same factors affecting the validity coefficient. Basically the same logic also applies to the evaluation of the decision rule in terms of selection utility and fairness. Correcting only the validity coefficient would leave the "bottom-line" evaluation of the selection procedure unaltered. Restricting the statistical corrections to the validity coefficient basically means that practically speaking nothing really changes. The fundamental research objective is to determine whether any discrepancy between the conditions under which the selection procedure is simulated and those prevailing at the eventual use of the selection procedure produces bias in estimates required to specify and justify the procedure; to delineate appropriate statistical corrections of the validity coefficient, decision rule and descriptions/ assessments of selection decision utility and -1< fairness, required to align the contexts of evaluation/validation and application; and to determine whether the corrections should be applied in validation research. The study provides no unqualified answers to the question whether corrections for various forms of range restriction and/ or criterion unreliability should be applied to the validity coefficient, the standard error of the validity coefficient or the parameters of the regression of the criterion on the predictor. Under specific conditions the corrections do affect decisions on the validity of performance hypotheses due to its effect on decisions on the significance of the uncorrected versus the corrected validity coefficient. Under specific conditions the corrections do affect decisions on applicants, especially when selection decisions are not restricted by selection quotas, due to its effect on the slope and intercept parameters of the regression of Y on X, and/ or due to its effect on the standard error of estimate. / AFRIKAANSE OPSOMMING: Die toestande waaronder keuringsprosedures tipes gevalideer word en die toestande waaronder die prosedure uiteindelik gebruik word, verskil normaalweg tot so 'n mate dat die relevansie van die bevindinge in die gedrang kom. Statistiese korreksies vir die geldigheidskoeffisient is algemeen beskikbaar. Die res van die argument in terme waarvan 'n keuringsprosedure ontwikkel en regverdig word kan egter ook verwring word deur dieselfde verskille tussen die toestande waaronder die keuringsprosedure gesimuleer word en die waaronder die prosedure uiteindelik gebruik word. Relatief min kommer bestaan skynbaar egter ten opsigte van die oordraagbaarheid van die besluitnemingsfunksie wat onder die gesimuleerde toestande ontwikkel is of ten opsigte van die verkree beskrywings van nut en billikheid. Hierdie toedrag van sake val ietwat vreemd op. Die eksteme geldigheidprobleme geassosieer met validasie-ontwerpe is redelik goed gedokumenteer. Dit is dus nie asof die psigometrika-literatuur onbewus is van die probleem wat by die veralgemening van resultate van geldigheidstudies ter sprake is nie. Die besluitnemingsfunksie is waarskynlik die spil waarom die keuringsprosedure draai daarin dat dit die onderliggende prestasie-teorie vergestalt, maar meer belangrik, daarin dat dit die daadwerklike aanvaarding en verwerping van applikante bepaal. Indien statistiese korreksies tot die geldigheidskoeffisient beperk word bly die besluitnemingsfunksie onveranderd, alhoewel dit ook moontlik verwring mag word deur dieselfde faktore wat sydigheid in die geldigheidskoeffisient te weeg bring. Dieselfde logika geld ook ten opsigte van die evaluasie van die besluitnemingsfunksie in terme van nut en billikheid. Indien slegs die geldigheidskoeffisient gekorrigeer word bly d.e "bottom-line" evaluasie van die keuringsprosedure onveranderd. Prakties gesproke dus, verander niks indien statistiese korreksies tot die geldigheidskoeffisient beperk word. Die fundamentele navorsingsdoelstelling is om vas te stel of verskille tussen die toestande waaronder die keuringsprosedure gevalideer word, en die toestande waaronder die prosedure uiteindelik gebruik word, sydigheid te weeg bring in die maatstawwe wat vereis word om die keuringsprosedure te spesifiseer en te regverdig; om toepaslike statistiese korreksies vir die geldigheidskoeffisient, besluitnnemingsreel en beskrywings van nut en billikheid af te lei ten einde die kontekste van simulasie/ validasie en toepassing te versoen; en om vas te stel of sodanige korreksies wel in validasie-navorsing toegepas behoort te word. Die studie verskaf geen ongekwalifiseerde antwoord op die vraag of korreksies vir die verskeie vorms van variansie-inperking en/ of kriterium onbetroubaarheid op die geldigheidskoeffisient, die standaardfout van die geldigheidskoefisient of die parameters van die regressie van die kriterium op die voorspeller toegepas behoort te word nie. Die korreksies affekteer wel besluite aangaande die geldigheid van prestasiehipoteses onder spesifieke toestande. Die korreksies het ook onder bepaalde toestande 'n effek op besluite aangaande applikante deur hul effek op die regressiekoeffisiente en/ of die standaardskattingsfout.

This research attempts to confirm the reliability and construct validity of a personnel selection instrument called a Situational Judgment Test (SJT) through reliability analysis and factor analysis. The existing literature on SJTs is reviewed, including the advantages of using SJTs in personnel selection as well as the debate on whether SJTs measure a single construct or whether they can be multidimensional depending on the content. The specific SJT in this research was theoretically developed and received expert ratings to assess four general constructs: problem solving, planning, priority setting, and leadership. No support from alpha internal consistency reliability analysis was found for the assembly of these items into the four a priori subscales, thus assembly of these items into the theoretical subscales and scales was not supported.

Impetus in recruiting and testing candidates via the Internet results from the popularity of the World Wide Web. There has been a transition from paper-pencil to online testing because of large number of benefits afforded by online testing. Though the benefits of online testing are many, there may be serious implications of testing job applicants in unproctored settings. The focus of this field study was two-fold: (1) to examine differences between the proctored and unproctored online test administrations of the ipsative version of Occupational Personality Questionnaire (OPQ32i and (2) to extend online testing research using OPQ32i with a U.S population. A large sample (N = 5223) of archival selection data from a financial company was used, one group was tested in proctored and the other in unproctored settings. Although some statistical differences were found, very small to small effect sizes indicate negligible differences between the proctored and unproctored groups. Principal component analysis with varimax rotation was conducted. The scales not only loaded differently from the Great Eight factor model suggested by SHL, but also differently for the two groups, limiting their interpretability. In addition to the limitations and future directions of the study, the practical implications of the results for companies considering unproctored, online personality testing as a part of their selection process are discussed.

At times called a philosophy and other times called a process, cutting score methodology is an issue routinely encountered by Industrial/Organizational (I/0) psychologists. Published literature on cutting score methodology appears much more frequently in academic settings than it does in personnel settings where the potential for lawsuits typically occurs more often. With the passage of the 1991 Civil Rights Act, it is no longer legal to use within-group scoring. It has now become necessary for personnel psychologists to develop more acceptable selection methods that fall within established guidelines. Designating cutoff scores with the Angoff method appears to suit many requirements of personnel departments. Several procedures have evolved that suggest enhancing the accuracy and reliability of the Angoff method is possible. The current experiment investigated several such procedures, and found that rater accuracy methods significantly enhance cutoff score reliability and accuracy.

The purpose of this study was to investigate the criterion-related validity of an assessment center's competency dimension ratings, exercise ratings, and standardized test scores. Numerous studies have clearly demonstrated assessment centers display substantial evidence of content and criterion-related validity. However, the inability of assessment centers to display construct-related validity has caused a great deal of concern among researchers. The suggestions of these researchers are addressed through a more detailed examination of the criterion-related validity of an assessment center. Despite a number of methodological issues, two competency dimensions and two components stand out as viable predictors of the criteria used in this study. Examination of individual and incremental validity coefficients reveals the Strategic Focus and Attracting and Developing Talent competency dimensions, the In-Basket exercise, and the Watson-Glaser scaled score consistently predict the criteria used in this study. The implications of these results for future research are discussed.

The study investigated the predictive validity of the California Psychological Inventory (CPI) as a tool for sales selection. Two analyses were conducted. Study 1 consisted of 20 male home improvement representatives. The average net and gross closing ratios for a six month period were used as the criteria. The results indicted that none of the CPI scales differentiated between poor and good performers when correlated with the secondary criterion of gross closing ratios. These findings were contrary to a previous concurrent validity study. Study 2 investigated month to month retention/separation for 61 home improvement representatives, to determine if the CPI differentiated between short and long term success. Phi coefficients showed no statistical significance between retention/separation and the CPI profile score over time.

The current study employs Monte Carlo analyses to evaluate the effectiveness of various statistical procedures for determining specific values of interest within a population of 1,000,000 cases. Specifically, the proper procedures for addressing the opposing effects of direct range restriction and validity overestimation were assessed through a comparison of multiple correlation coefficients derived using various sequences of procedures in randomly drawn samples. A comparison of the average bias associated with these methods indicated that correction for range restriction prior to the application of a validity overestimation adjustment formula yielded the best estimate of population parameters over a number of conditions. Additionally, similar methods were employed to assess the effectiveness of the standard ΔR2F-test for determining, based on characteristics of the derivation sample, the comparative superiority of either optimally or unit weighted composites in future samples; this procedure was largely ineffective under the conditions employed in the current study.

The current study is a significant portion of a content validation study conducted to evaluate new selection tools developed by an organization. In it I assessed the content validity of a structured interview through the following steps: conducting a literature review to establish best practices, reviewing the data gathering process, conducting data analyses, recommending item weights, evaluating the validity of the interview, suggesting standardized materials, creating a plan for training for interview use, and ensuring documentation is kept as required by law. The data were collected internally by the organization and were provided for analyses. Analyses included one-way ANOVAs, and the calculation of r*wg for the Subject Matter Expert (SME) importance ratings. Results indicated that the SME ratings exhibited acceptable interrater agreement and that all competencies were important. Therefore, the interview based on these ratings has acceptable content validity. Recommendations were provided to the organization regarding possible alternatives and the impact that the decisions made would have on the resulting validity.

Title from title page of PDF (University of Missouri--St. Louis, viewed March 22, 2010). Includes bibliographical references (p. 68-81).