Survey data quality is influenced by the care and attention that respondents take in answering questions. Careless and inattentive (CI) responding is a confound in survey data that can distort findings and lead to incorrect conclusions. This quantitative study explored CI responding in job analysis studies supporting occupational certification programs and its relationship to survey features, data quality measures, and test content validity. Satisficing theory served as the framework, and secondary analysis of 3 job analysis surveys was undertaken. Results indicated that 9-33% of respondents engaged in CI responding, with the rate differing by CI index used (Mahalanobis distance, long string analysis, or person-total correlation) and by occupation. Each index detected a distinct pattern of carelessness, supporting the use of multiple indices. The indices performed best detecting carelessness in frequency ratings and may not be useful for all job analysis rating scales. Partial support was found for relationships between carelessness and survey features. CI responding had a minimal impact on mean ratings, correlations, and interrater reliability, and had no impact on certification test content outlines. By providing guidance and caution on the use of CI response detection methods with job analysis survey data, this study produced two potential avenues for social change. For practitioners conducting occupational job analyses, the use of CI detection methods can enhance the validity of data used to make certification decisions. For researchers, follow-up studies can yield a more nuanced understanding of the most appropriate use of these methods in the job analysis context.
| Identifer | oai:union.ndltd.org:waldenu.edu/oai:scholarworks.waldenu.edu:dissertations-8325 |
| Date | 01 January 2019 |
| Creators | Muenzen, Patricia M |
| Publisher | ScholarWorks |
| Source Sets | Walden University |
| Language | English |
| Detected Language | English |
| Type | text |
| Format | application/pdf |
| Source | Walden Dissertations and Doctoral Studies |
Page generated in 0.0021 seconds