The application of Bayesian networks within the field of institutional research is explored through the development of a Bayesian network used to predict first- to second-year retention of undergraduates. A hybrid approach to model development is employed, in which formal elicitation of subject-matter expertise is combined with machine learning in designing model structure and specification of model parameters. Subject-matter experts include two academic advisors at a small, private liberal arts college in the southeast, and the data used in machine learning include six years of historical student-related information (i.e., demographic, admissions, academic, and financial) on 1,438 first-year students. Netica 5.12, a software package designed for constructing Bayesian networks, is used for building and validating the model. Evaluation of the resulting model’s predictive capabilities is examined, as well as analyses of sensitivity, internal validity, and model complexity. Additionally, the utility of using Bayesian networks within institutional research and higher education is discussed. The importance of comprehensive evaluation is highlighted, due to the study’s inclusion of an unbalanced data set. Best practices and experiences with expert elicitation are also noted, including recommendations for use of formal elicitation frameworks and careful consideration of operating definitions. Academic preparation and financial need risk profile are identified as key variables related to retention, and the need for enhanced data collection surrounding such variables is also revealed. For example, the experts emphasize study skills as an important predictor of retention while noting the absence of collection of quantitative data related to measuring students’ study skills. Finally, the importance and value of the model development process is stressed, as stakeholders are required to articulate, define, discuss, and evaluate model components, assumptions, and results.

Ecological patterns and processes influence ecosystem function at scales from nanometres to global scales depending on the organisms involved. Predicting the presence and abundance of species, at scales appropriate to the organisms and the underlying processes, is central to ecology. Models of species’ distributions can provide important insights into pattern-process-scale relationships including the relative importance of various environmental factors and their interactions that influence habitat selection at the individual and population levels. Mapping current and potential distributions informs the conservation of threatened species by providing spatial information on where a species is likely to occur and the identification of habitat elements and their spatial configurations which influence occupancy and persistence. The aim of this thesis was to incorporate the principles of pattern, process and scale in the identification of habitat associations for threatened species within a species’ distribution modelling framework. Accurate modelling of species’ distributions depends on robust sampling designs, reliable data input and appropriate statistical methodologies that align with the ecological model. I applied a range of innovative statistical methods to various sources of data to identify important habitat associations for a threatened species at different scales and tested the discriminative ability of the resultant models. I integrated the results from extensive field sampling and expert elicitation to build connectivity networks using graph theory algorithms to identify important conservation priorities for threatened species. The threatened brush-tailed rock-wallaby (Petrogale penicillata) was chosen as a suitable study species for quantifying habitat relationships at multiple spatial scales using species’ distribution modelling. The distribution of brush-tailed rock-wallabies is restricted to a set of suitable habitat characteristics related to rocky terrain supporting cliffs and boulder piles that occur infrequently across a landscape. At the site scale, they require suitable resting and refuge sites provided by rocky habitats, while at a landscape scale their dispersal is dependent on the connectivity of suitable habitats. The species is listed as threatened throughout eastern Australia and endangered in some states. Information about its current distribution and occupancy status is essential to support habitat conservation and threat management. The first chapter provides a broad view of the literature on modelling of species’ distributions and the thesis aims and structure. In chapter 2, I assess the ecological scale relevant to habitat modelling for the brush-tailed rock-wallaby. In chapter 3 I test whether habitat models from one region can be extrapolated to neighbouring regions. I use a novel approach and elicitation tool in chapter 4 to collect expert knowledge and assess it with a comprehensive set of field data in a Bayesian framework. In chapter 5 I assess whether landscape connectivity is a determinant of site occupancy by using graph theory algorithms to identify important habitat patches and dispersal pathways for rock-wallaby movement in fragmented landscapes. The final chapter synthesises the individual chapters’ findings within the context of species’ distribution modelling. Management implications are discussed for the conservation of the brush-tailed rock-wallaby and its habitat network. Wider implications are also suggested for other rock-wallaby species and species living in similar environments. The results of the thesis showed the habitat of the brush-tailed rock-wallaby was affected by site-scale and landscape-scale factors, supporting the need for a multi-scale approach when investigating species-environment associations. I found that models performed well within a region at both scales. Extrapolating the models to neighbouring regions resulted in good predictive performance at the site scale but substantially poorer predictive performance at the landscape scale. When there is insufficient field data to build robust data models, management bodies would benefit from incorporating expert knowledge. The study demonstrates the potential errors in using experts with knowledge gained from outside the area of interest. Finally, I highlight the importance of accounting for the landscape connectivity between patches from the perspective of the individual animal. Least cost analysis, using graph theory algorithms, provides a cost-efficient and effective framework for identifying landscape connectivity patterns and key paths and patches to help inform suitable land management strategies for conservation of threatened species. There is much pressure from conservation and management agencies to produce models of species’ distributions that could be potentially be used in other regions or with similar species. The thesis combines ecological theory with rigorous statistical methodology to test different modelling techniques for species distribution modelling. It demonstrates how a combination of expert knowledge, extensive field data and landscape connectivity measures successfully predicts ecological relationships at a number of scales. Species’ distribution models can benefit from applying a robust sampling design and suitable modelling techniques to various data sources to generate ecologically-based information to improve our understanding of species-habitat associations and provide a reliable component to incorporate into conservation planning. This thesis therefore provides important advances to spatial ecology and ecological modelling of species distributions and management of threatened species.

An expert elicitation is a method of eliciting subjective probability distributions over key parameters from experts. Traditionally an expert elicitation has taken the form of a face-to-face interview; however, interest in using online methods has been growing. This thesis compares two elicitation modes and examines the effectiveness of an interactive online survey compared to a face-to-face interview. Differences in central values, overconfidence, accuracy and satisficing were considered. The results of our analysis indicated that, in instances where the online and face-to-face elicitations were directly comparable, the differences between the modes was not significant. Consequently, a carefully designed online elicitation may be used successfully to obtain accurate forecasts.

Yes / Expert knowledge elicitation lies at the core of judgmental forecasting—a domain that relies fully on the power of such knowledge and its integration into forecasting. Using experts in a demand forecasting framework, this work aims to compare the accuracy improvements and forecasting performances of three judgmental integration methods. To do this, a field study was conducted with 31 experts from four companies. The methods compared were the judgmental adjustment, the 50–50 combination, and the divide-and-conquer. Forecaster expertise, the credibility of system forecasts and the need to rectify system forecasts were also assessed, and mechanisms for performing this assessment were considered. When (a) a forecaster’s relative expertise was high, (b) the relative credibility of the system forecasts was low, and (c) the system forecasts had a strong need of correction, judgmental adjustment improved the accuracy relative to both the other integration methods and the system forecasts. Experts with higher levels of expertise showed higher adjustment frequencies. Our results suggest that judgmental adjustment promises to be valuable in the long term if adequate conditions of forecaster expertise and the credibility of system forecasts are met.

O propósito desta dissertação é apresentar o uso da opinião de especialistas e outras questões relevantes acerca do assunto na avaliação das incertezas em estudos de análise de risco e confiabilidade, com apresentação de um estudo de caso prático. Em estudos de confiabilidade umas das principais preocupações está na determinação das frequências de ocorrência dos eventos e seu comportamento ao longo do tempo. Muitas vezes, os dados de frequência estão obsoletos, não estão disponíveis ou mesmo, não são suficientes para se avaliar a probabilidade de ocorrência de eventos. Nestes casos, a elicitação da opinião de especialista surge como uma alternativa a suplementar estas ausências de dados possibilitando assim uma melhor análise das incertezas. Baseado na condição da subjetividade, a elicitação dos especialistas tem como objetivo quantificar as incertezas a partir da experiência prévia e estado atual de conhecimento. Combinado com métodos matemáticos, a elicitação possibilita o gerenciamento de conflitos de informações de forma a atingir o consenso e possibilitar uma análise subjetiva dos problemas. / The purpose of this dissertationis to present the use of expert opinion and other relevant issues on the subjective assessment of uncertainties in risk analysis and reliability studies, presenting a practical case study. In reliability studies a major concern is to determine the frequencies of occurrence of events and their behavior through time. Often, the available data are not representative enough to evaluate the event probability or it is obsolete for use. In these cases, the elicitation of expert opinion is an alternative to supplement these data absences, Thus enabling a better uncertainties analysis. Based on the subjectivity condition, the elicitation of experts aims to quantify the uncertainty considering the previous experiences and current state of knowledge. Combined with mathematical elicitation methods, it enables the manegement of information conflicts in order to reach consensus and makes possible a subjective analysis of problems.

<p>Urbanization causes myriad changes in watershed processes, ultimately disrupting the structure and function of stream ecosystems. Urban development introduces contaminants (human waste, pesticides, industrial chemicals). Impervious surfaces and artificial drainage systems speed the delivery of contaminants to streams, while bypassing soil filtration and local riparian processes that can mitigate the impacts of these contaminants, and disrupting the timing and volume of hydrologic patterns. Aquatic habitats where biota live are degraded by sedimentation, channel incision, floodplain disconnection, substrate alteration and elimination of reach diversity. These compounding changes ultimately lead to alteration of invertebrate community structure and function. Because the effects of urbanization on stream ecosystems are complex, multilayered, and interacting, modeling these effects presents many unique challenges, including: addressing and quantifying processes at multiple scales, representing major interrelated simultaneously acting dynamics at the system level, incorporating uncertainty resulting from imperfect knowledge, imperfect data, and environmental variability, and integrating multiple sources of available information about the system into the modeling construct. These challenges can be addressed by using a Bayesian modeling approach. Specifically, the use of multilevel hierarchical models and Bayesian network models allows the modeler to harness the hierarchical nature of the U.S. Geological Survey (USGS) Effect of Urbanization on Stream Ecosystems (EUSE) dataset to predict invertebrate response at both basin and regional levels, concisely represent and parameterize this system of complicated cause and effect relationships and uncertainties, calculate the full probabilistic function of all variables efficiently as the product of more manageable conditional probabilities, and includes both expert knowledge and data. Utilizing this Bayesian framework, this dissertation develops a series of statistically rigorous and ecologically interpretable models predicting the effect of urbanization on invertebrates, as well as a unique, systematic methodology that creates an informed expert prior and then updates this prior with available data using conjugate Dirichlet-multinomial distribution forms. The resulting models elucidate differences between regional responses to urbanization (particularly due to background agriculture and precipitation) and address the influences of multiple urban induced stressors acting simultaneously from a new system-level perspective. These Bayesian modeling approaches quantify previously unexplained regional differences in biotic response to urbanization, capture multiple interacting environmental and ecological processes affected by urbanization, and ultimately link urbanization effects on stream biota to a management context such that these models describe and quantify how changes in drivers lead to changes in regulatory endpoint (the Biological Condition Gradient; BCG).</p> / Dissertation

O propósito desta dissertação é apresentar o uso da opinião de especialistas e outras questões relevantes acerca do assunto na avaliação das incertezas em estudos de análise de risco e confiabilidade, com apresentação de um estudo de caso prático. Em estudos de confiabilidade umas das principais preocupações está na determinação das frequências de ocorrência dos eventos e seu comportamento ao longo do tempo. Muitas vezes, os dados de frequência estão obsoletos, não estão disponíveis ou mesmo, não são suficientes para se avaliar a probabilidade de ocorrência de eventos. Nestes casos, a elicitação da opinião de especialista surge como uma alternativa a suplementar estas ausências de dados possibilitando assim uma melhor análise das incertezas. Baseado na condição da subjetividade, a elicitação dos especialistas tem como objetivo quantificar as incertezas a partir da experiência prévia e estado atual de conhecimento. Combinado com métodos matemáticos, a elicitação possibilita o gerenciamento de conflitos de informações de forma a atingir o consenso e possibilitar uma análise subjetiva dos problemas. / The purpose of this dissertationis to present the use of expert opinion and other relevant issues on the subjective assessment of uncertainties in risk analysis and reliability studies, presenting a practical case study. In reliability studies a major concern is to determine the frequencies of occurrence of events and their behavior through time. Often, the available data are not representative enough to evaluate the event probability or it is obsolete for use. In these cases, the elicitation of expert opinion is an alternative to supplement these data absences, Thus enabling a better uncertainties analysis. Based on the subjectivity condition, the elicitation of experts aims to quantify the uncertainty considering the previous experiences and current state of knowledge. Combined with mathematical elicitation methods, it enables the manegement of information conflicts in order to reach consensus and makes possible a subjective analysis of problems.

Climate change is a risk issue of global proportions. Human health and environmental impacts are anticipated from hazards associated with changes in temperature and precipitation regimes, and climate extremes. Increased natural hazards include storms and flooding, extreme heat, drought, and wildfires. Reduced food and water quality and quantity, reduced air quality, new geographic range of infectious diseases, and increased exposure to ultra-violet radiation are also predicted. In order to make a measurable contribution to reducing carbon dioxide emissions at point source fossil fuel and industrial process sites that contribute to climate change, estimates suggest that up to 3,000 dedicated large scale carbon capture and geological sequestration (CCS) projects will be necessary by 2050. Integrated projects include carbon dioxide capture; compression into a supercritical stream; transport, most often by pipeline; deep injection at wellheads; and sequestration in suitable saline aquifer geological formations, usually 800 metres or more below the earth’s surface. In implementing CCS as part of an overall climate change mitigation strategy, it is important to note that population health and environmental risks are associated with each of these value chain components of integrated projects. Based on an assessment of existing regulatory and non-regulatory guidance for risk assessment/risk management (RA/RM), an analysis of the application, assessment, and approval process for four large scale Canadian projects, and findings from a structured expert elicitation focused on hazard and risk issues in injection and storage and risk management of low probability high impact events, this research developed an Integrated Risk Management Framework (IRMF) for CCS in the Canadian context. The IRMF is a step-wise systematic process for RA/RM during the life of a project, including engagement with wide ranging government and non-government partners that would contribute to a determination of acceptable risk and risk control options. The execution of the IRMF is an intervention that could reduce local hazards and associated risks in terms of likelihood and consequence, as well as identify and document risk management that could underpin broad acceptance of CCS as a climate change mitigation technology. This would thereby also have an important part in protecting global population health and wellbeing in the long term. Indeed, diverse stakeholders could be unforgiving if hazard assessment and risk management in CCS is considered insufficient, leading to ‘pushback’ that could affect future implementation scenarios. On the other hand, RA/RM done right could favourably impact public perception of CCS, in turn instilling confidence, public acceptance, and ongoing support for the benefit of populations worldwide. This thesis is composed of an introduction to the research problem, including a population health conceptual framework for the IRMF, followed by five manuscripts, and concluding with a discussion about other barriers to CCS project development, and a risk management policy scenario for both the present time and during the 2017-2030 implementation period.

Yes / In different engineering processes, the reliability of systems is increasingly evaluated to ensure that the safety-critical process systems will operate within their expected operational boundary for a certain mission time without failure. Different methodologies used for reliability analysis of process systems include Failure Mode and Effect Analysis (FMEA), Fault Tree Analysis (FTA), and Bayesian Networks (BN). Although these approaches have their own procedures for evaluating system reliability, they rely on exact failure data of systems’ components for reliability evaluation. Nevertheless, obtaining exact failure data for complex systems can be difficult due to the complex behaviour of their components, and the unavailability of precise and adequate information about such components. To tackle the data uncertainty issue, this chapter proposes a framework by combining intuitionistic fuzzy set theory and expert elicitation that enables the reliability assessment of process systems using FTA. Moreover, to model the statistical dependencies between events, we use the BN for robust probabilistic inference about system reliability under different uncertainties. The efficiency of the framework is demonstrated through application to a real-world system and comparison of the results of analysis produced by the existing approaches. / The full text will be available at the end of the publisher's embargo, 9th April 2025

abstract: In the three phases of the engineering design process (conceptual design, embodiment design and detailed design), traditional reliability information is scarce. However, there are different sources of information that provide reliability inputs while designing a new product. This research considered these sources to be further analyzed: reliability information from similar existing products denominated as parents, elicited experts' opinions, initial testing and the customer voice for creating design requirements. These sources were integrated with three novels approaches to produce reliability insights in the engineering design process, all under the Design for Reliability (DFR) philosophy. Firstly, an enhanced parenting process to assess reliability was presented. Using reliability information from parents it was possible to create a failure structure (parent matrix) to be compared against the new product. Then, expert opinions were elicited to provide the effects of the new design changes (parent factor). Combining those two elements resulted in a reliability assessment in early design process. Extending this approach into the conceptual design phase, a methodology was created to obtain a graphical reliability insight of a new product's concept. The approach can be summarized by three sequential steps: functional analysis, cognitive maps and Bayesian networks. These tools integrated the available information, created a graphical representation of the concept and provided quantitative reliability assessments. Lastly, to optimize resources when product testing is viable (e.g., detailed design) a type of accelerated life testing was recommended: the accelerated degradation tests. The potential for robust design engineering for this type of test was exploited. Then, robust design was achieved by setting the design factors at some levels such that the impact of stress factor variation on the degradation rate can be minimized. Finally, to validate the proposed approaches and methods, different case studies were presented. / Dissertation/Thesis / Doctoral Dissertation Industrial Engineering 2014