Return to search

Method variance in the social sciences

A preliminary taxonomy has been developed that differentiates between trait characteristics, the situation, manifest and subtle methods and interaction effects. The proposed taxonomy divides method into manifest and subtle categories. The obvious, surface characteristics of method are considered manifest, while the deeper structures of method that are not usually seen are considered subtle. Seven manifest method categories are described: stimulus format, response format, response categories, raters, whether the measure is direct or summative, rating the stimulus or the response, and opaque or transparent measures. Numerous subtle method categories can also be seen within the method rather than on its surface. These include semantic or verbal characteristics, direction of wording, measures of amount, ability, latency of response, possessions, situational context, associations, and behavior. Other method categories include report of others reactions, body symptoms, and time frame. Seven item level methods were tested in this study in the context of measurement of Introversion-Extraversion (IE). They included direct questions, direction of wording, situational context, time frame, report of others reactions, preference (semantic or verbal structure), and behavior. Three of these methods (preference, behavior, and direction of wording) introduced substantial method variance in the measurement of IE, and there were also several sizable trait*method interactions: direction of wording, time frame, and preference. Generalizability theory analysis (GT) proved to be quite useful in estimating method effects and interactions. By comparing the findings from GT analysis to confirmatory factor analysis (CFA) results, it became clear that there were problems with the CFA results that could not be ignored. Although generalizability analysis is limited in its ability to provide estimates of the trait and method contributions for individual measures, unfortunately, it informs us that the estimates provided by CFA are probably erroneous.

Identiferoai:union.ndltd.org:arizona.edu/oai:arizona.openrepository.com:10150/289721
Date January 2001
CreatorsDavis, Melinda Fritchoff
ContributorsSechrest, Lee B.
PublisherThe University of Arizona.
Source SetsUniversity of Arizona
Languageen_US
Detected LanguageEnglish
Typetext, Dissertation-Reproduction (electronic)
RightsCopyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author.

Page generated in 0.0023 seconds