A common approach for evaluating complex multiattributed choice alternatives is judgment decomposition: the alternatives are decomposed into a number of value-relevant attributes, the decision maker evaluates each alternative with respect to each attribute, and those single-attribute evaluations are aggregated across the attributes by a formal composition rule. One primary assumption behind decomposition is that it would produce a more reliable outcome than direct holistic evaluations. Although there is some empirical evidence that decomposed procedures can improve the reliability of evaluations, the extent of decomposition can have a considerable effect on the resulting evaluations. This research investigated, theoretically and experimentally, the effects of decomposition level on intrarater reliability in multiattribute alternative evaluation.
In a theoretical study, using an additive value composition model with random variables, the composite variance of alternative evaluation was analyzed with respect to the level of decomposition. The composite variance of decomposed evaluation was derived from the variances in the components recomposed using a Statistical method of error propagation. By analyzing the composite variance as a function of the number of attributes used, possible effects of decomposition level were predicted and explained. The analysis showed that the variance of an alternative evaluation is a decreasing function with respect to the level of decomposition, in most cases, and that the marginal reduction of variance diminishes as decomposition level increases.
In an experimental study, intrarater test-retest Convergence was examined for a job evaluation with different levels of decomposition. Subjects evaluated six hypothetical job alternatives using four levels of decomposition that ranged from a single overall evaluation to evaluations on twelve highly specific attributes. Intrarater convergence was measured by mean absolute deviations and Pearson correlations between the evaluation scores in two identical sessions separated by two weeks. The mean absolute deviations decreased significantly with respect to the decomposition levels while the Pearson correlations were not significant. Further analyses indicated that the mean absolute deviations decreased with a diminishing rate of reduction, as the decomposition level increased.
The research results suggest that decomposition reduces the variability of each alternative evaluation, in most situations. The results, however, also suggest that decomposition may not improve the consistency of preference order of the alternatives that is often important in practical choice decisions. / Ph. D.
| Identifer | oai:union.ndltd.org:VTETD/oai:vtechworks.lib.vt.edu:10919/38478 |
| Date | 06 June 2008 |
| Creators | Cho, Young Jin |
| Contributors | Industrial and Systems Engineering, Koelling, C. Patrick, Casali, John G., Hauenstein, Neil M. A., Kurstedt, Harold A. Jr., Torgersen, Paul E. |
| Publisher | Virginia Tech |
| Source Sets | Virginia Tech Theses and Dissertation |
| Language | English |
| Detected Language | English |
| Type | Dissertation, Text |
| Format | ix, 197 leaves, BTD, application/pdf, application/pdf |
| Rights | In Copyright, http://rightsstatements.org/vocab/InC/1.0/ |
| Relation | OCLC# 26812684, LD5655.V856_1992.C563.pdf |
Page generated in 0.0093 seconds