Back to search

Digital Repository at the University of Maryland

A META-DATA INFORMED EXPERT JUDGMENT AGGREGATION AND CALIBRATION TECHNIQUE

Abstract

dc:description.abstract

Policy makers use expert judgment opinions elicited from experts as probability distributions, quantiles or point estimates, as inputs to decisions that may have economic or life and death impacts. While challenges in estimating probabilities in general have been studied, research that distinguished between non-probabilistic, i.e., physical, variables and probabilistic variables specifically in the context of meta-data based expert judgment aggregation techniques, and the errors associated with the predictions developed from such variables, was not identified. This research demonstrated that for two combined expert judgment meta-data bases, the distinction between physical and probabilistic variables was significant in terms of the extent of multiplicative error between elicited medians and realized values both before and after aggregation. The distinction also impacts the widths of bounds around aggregated point estimates. The research compared nine methods of aggregating estimates and obtaining calibrated bounds, including ones based on alpha stable distributions, quantile regression, and a Bayesian model. Simple parametric distributions were also fit to the meta-data. Methods were compared against criteria including accuracy, bounds coverage and width, sensitivity to outliers, and complexity. No single method dominated all criteria for either variable type. The research investigated sensitivity of results to level of realized value for a variable, such as infrequent events for probabilistic variables, as well as sensitivity of results to number of experts.

Degree

thesis:*
Department dc:contributor.department
Reliability Engineering
Grantor dc:publisher
Digital Repository at the University of Maryland
Year dc:date.issued
2016

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Feldman, Ellis
Advisor dc:contributor.advisor
  • Mosleh, Ali

Rights

Language dc:language.iso
en

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
University of Maryland
Base URL
api.drum.lib.umd.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Feldman, Ellis. A META-DATA INFORMED EXPERT JUDGMENT AGGREGATION AND CALIBRATION TECHNIQUE. Digital Repository at the University of Maryland, 2016. http://hdl.handle.net/1903/19081