Spelling suggestions: "subject:"[een] MODEL EVALUATION"" "subject:"[enn] MODEL EVALUATION""
11 |
Evaluation of a Water Budget Model for Created Wetland Design and Comparative Natural Wetland HydroperiodsSneesby, Ethan Paul 04 April 2019 (has links)
Wetland impacts in the Mid-Atlantic USA are frequently mitigated via wetland creation in former uplands. Regulatory approval requires a site-specific water budget that predicts the annual water level regime (hydroperiod). However, many studies of created wetlands indicate that post-construction hydroperiods frequently are not similar to impacted wetland systems. My primary objective was to evaluate a water budget model, Wetbud (Basic model), through comparison of model output to on-site water level data for two created forested wetlands in Northern Virginia. Initial sensitivity analyses indicated that watershed curve number and outlet height had the most leverage on model output. Addition of maximum depth of water level drawdown greatly improved model accuracy. I used Nash-Sutcliffe efficiency (NSE) and root mean squared error (RMSE) to evaluate goodness of fit of model output against site monitoring data. The Basic model reproduced the overall seasonal hydroperiod well once fully parameterized, despite NSE values ranging from -0.67 to 0.41 in calibration and from -4.82 to -0.26 during validation. For RMSE, calibration values ranged from 5.9 cm to 12.7 cm during calibration and from 8.2 cm to 18.5 cm during validation. My second objective was to select a group of "design target hydroperiods" for common Mid-Atlantic USA wetland types. From > 90 sites evaluated, I chose four mineral flats, three riverine wetlands, and one depressional wetland that met all selection criteria. Taken together, improved wetland water budget modeling procedures (like Wetbud) combined with the use of appropriate target hydroperiod information should improve the success of wetland creation efforts. / Master of Science / Wetlands in the USA are defined by the combined occurrence of wetland hydrology, hydric soils, and hydrophytic vegetation. Wetlands serve to retain floodwater, sediments and nutrients within their landscape. They may serve as a source of local groundwater recharge and are home to many endangered species of plants and animals. Wetland ecosystems are frequently impacted by human activities including road-building and development. These impacts can range from the destruction of a wetland to increased nutrient contributions from storm- or wastewater. One commonly utilized option to mitigate wetland impacts is via wetland creation in former upland areas. Regulatory approval requires a site-specific water budget that predicts the average monthly water levels (hydroperiod). A hydroperiod is simply a depiction of how the elevation of water changes over time. However, many studies of created wetlands indicate that post-construction hydroperiods frequently are not representative of the impacted wetland systems. Many software packages, called models, seek to predict the hydroperiod for different wetland systems. Improving and vetting these models help to improve our understanding of how these systems function. My primary objective was to evaluate a water budget model, Wetbud (Basic model), through comparison of model output to onsite water level data for two created forested wetlands in Northern Virginia. Initial analyses indicated that watershed curve number (CN) and outlet height had the most influence on model output. Addition of a maximum depth of water level drawdown below the ground surface greatly improved model accuracy. I used statistical analyses to compare model output to site monitoring data. The Basic model reproduced the overall seasonal hydroperiod well once inputs were set to optimum values (calibration). Statistical results for the calibration varied between excellent and acceptable for our selected measure of accuracy, the root mean squared error. My second objective was to select a grouping of “design target hydroperiods” for common Mid-Atlantic USA wetland types. From > 90 sites evaluated, I chose four mineral flats, three riverine wetlands, and one depressional wetland that met all selection criteria. Taken together, improved wetland water budget modeling procedures (like Wetbud) combined with the use of appropriate target hydroperiod information should improve the success of wetland creation efforts.
|
12 |
Improved Usage Model for Web Application Reliability TestingWan, Bo 31 July 2012 (has links)
Testing the reliability of an application usually requires a good usage model that accurately captures the likely sequences of inputs that the application will receive from the environment. The models being used in the literature are mostly based on Markov chains. They are used to generate test cases that are statistically close to what the applica-tion is expected to receive when in production. In this thesis, we propose a model for reli-ability testing that is created directly from the log file of a web application. Our proposed model is also based on Markov chains and has two components: one component, based on a modified tree, captures the most frequent behaviors, while the other component is another Markov chain that captures infrequent behaviors. The result is a statistically cor-rect model that shows clearly what most users do on the site.
The thesis also presents an evaluation method for estimating the accuracy of vari-ous reliability-testing usage models. The method is based on comparison between ob-served users’ traces and traces inferred from the usage model. Our method gauges the accuracy of the reliability-testing usage model by calculating the sum of goodness-of-fit values of each traces and scaling the result between 0 and 1. Finally, we present an experimental study on the log of a real web site and discuss the way to use proposed usage model to generate test sequences, as well as strength and weakness of the model for reliability testing.
|
13 |
HYDRUS modelling to predict field trafficability under different drainage design and weather conditions in Southern ManitobaKaja, Krishna Phani 12 April 2017 (has links)
Advancements in computation and development of physically based hydrologic models to simulate complex vadose zone scenarios helped the research community to evaluate different scenarios easily compared to long-term field experiments. However, some field data collection is necessary to obtain input data such as soil properties, water usage and land management practices to validate the model performance specific to the site. Data obtained from field experiments conducted in 2011 at Hespler farms, Winkler, MB was used in this research for model calibration and validation. The hydrologic model, HYDRUS (2D/3D) was evaluated using parameters such as visual and statistical analysis. Model evaluation during the calibration and validation stage gave RMSE values of 0.019 and 0.015 cm3 cm-3; PBIAS values of -1.01 and -0.14, respectively, suggesting that the model was efficient in simulating soil water content similar to the field observed data. The validated models were then used to simulate outcomes for different scenarios such as 30-year rainfall data (1986 – 2015), different soil physical properties, and drainage system design parameters. Models simulating free drainage predicted lower soil water content compared to controlled drainage leading to 6 – 60 more trafficable days for 8 m spacing and 0.9 drain base depth. Free drainage predicted 8 – 110 additional trafficable days compared to controlled drainage for 15 m spacing and 1.1 drain depth. Heavier than normal rainfall events caused high water contents leading to a few years with a very low to no trafficable days under controlled drainage conditions. The comparisons are presented based on models using free drain conditions. Models with 8-m drain spacing predicted a 1 to 10-day increase in the number of trafficable days compared to the 15-m drain spacing. Drains placed at a base depth of 1.1 m below the soil surface predicted 4 - 40 more trafficable days compared to those installed at a base depth of 0.9 m. / October 2017
|
14 |
Probabilistic pairwise model comparisons based on discrepancy measures and a reconceptualization of the p-valueRiedle, Benjamin N. 01 May 2018 (has links)
Discrepancy measures are often employed in problems involving the selection and assessment of statistical models. A discrepancy gauges the separation between a fitted candidate model and the underlying generating model. In this work, we consider pairwise comparisons of fitted models based on a probabilistic evaluation of the ordering of the constituent discrepancies. An estimator of the probability is derived using the bootstrap.
In the framework of hypothesis testing, nested models are often compared on the basis of the p-value. Specifically, the simpler null model is favored unless the p-value is sufficiently small, in which case the null model is rejected and the more general alternative model is retained. Using suitably defined discrepancy measures, we mathematically show that, in general settings, the Wald, likelihood ratio (LR) and score test p-values are approximated by the bootstrapped discrepancy comparison probability (BDCP). We argue that the connection between the p-value and the BDCP leads to potentially new insights regarding the utility and limitations of the p-value. The BDCP framework also facilitates discrepancy-based inferences in settings beyond the limited confines of nested model hypothesis testing.
|
15 |
Development, Application and Evaluation of Statistical Tools in Pharmacometric Data AnalysisLindbom, Lars January 2006 (has links)
<p>Pharmacometrics uses models based on pharmacology, physiology and disease for quantitative analysis of interactions between drugs and patients. The availability of software implementing modern statistical methods is important for efficient model building and evaluation throughout pharmacometric data analyses.</p><p>The aim of this thesis was to facilitate the practical use of available and new statistical methods in the area of pharmacometric data analysis. This involved the development of suitable software tools that allows for efficient use of these methods, characterisation of basic properties and demonstration of their usefulness when applied to real world data. The thesis describes the implementation of a set of statistical methods (the bootstrap, jackknife, case-deletion diagnostics, log-likelihood profiling and stepwise covariate model building), made available as tools through the software Perl-speaks-NONMEM (PsN). The appropriateness of the methods and the consistency of the software tools were evaluated using a large selection of clinical and nonclinical data. Criteria based on clinical relevance were found to be useful components in automated stepwise covariate model building. Their ability to restrict the number of included parameter-covariate relationships while maintaining the predictive performance of the model was demonstrated using the antiarrythmic drug dofetilide. Log-likelihood profiling was shown to be equivalent to the bootstrap for calculating confidence intervals for fixed-effects parameters if an appropriate estimation method is used. The condition number of the covariance matrix for the parameter estimates was shown to be a good indicator of how well resampling methods behave when applied to pharmacometric data analyses using NONMEM. The software developed in this thesis equips modellers with an enhanced set of tools for efficient pharmacometric data analysis. </p>
|
16 |
Development, Application and Evaluation of Statistical Tools in Pharmacometric Data AnalysisLindbom, Lars January 2006 (has links)
Pharmacometrics uses models based on pharmacology, physiology and disease for quantitative analysis of interactions between drugs and patients. The availability of software implementing modern statistical methods is important for efficient model building and evaluation throughout pharmacometric data analyses. The aim of this thesis was to facilitate the practical use of available and new statistical methods in the area of pharmacometric data analysis. This involved the development of suitable software tools that allows for efficient use of these methods, characterisation of basic properties and demonstration of their usefulness when applied to real world data. The thesis describes the implementation of a set of statistical methods (the bootstrap, jackknife, case-deletion diagnostics, log-likelihood profiling and stepwise covariate model building), made available as tools through the software Perl-speaks-NONMEM (PsN). The appropriateness of the methods and the consistency of the software tools were evaluated using a large selection of clinical and nonclinical data. Criteria based on clinical relevance were found to be useful components in automated stepwise covariate model building. Their ability to restrict the number of included parameter-covariate relationships while maintaining the predictive performance of the model was demonstrated using the antiarrythmic drug dofetilide. Log-likelihood profiling was shown to be equivalent to the bootstrap for calculating confidence intervals for fixed-effects parameters if an appropriate estimation method is used. The condition number of the covariance matrix for the parameter estimates was shown to be a good indicator of how well resampling methods behave when applied to pharmacometric data analyses using NONMEM. The software developed in this thesis equips modellers with an enhanced set of tools for efficient pharmacometric data analysis.
|
17 |
Improved Usage Model for Web Application Reliability TestingWan, Bo 31 July 2012 (has links)
Testing the reliability of an application usually requires a good usage model that accurately captures the likely sequences of inputs that the application will receive from the environment. The models being used in the literature are mostly based on Markov chains. They are used to generate test cases that are statistically close to what the applica-tion is expected to receive when in production. In this thesis, we propose a model for reli-ability testing that is created directly from the log file of a web application. Our proposed model is also based on Markov chains and has two components: one component, based on a modified tree, captures the most frequent behaviors, while the other component is another Markov chain that captures infrequent behaviors. The result is a statistically cor-rect model that shows clearly what most users do on the site.
The thesis also presents an evaluation method for estimating the accuracy of vari-ous reliability-testing usage models. The method is based on comparison between ob-served users’ traces and traces inferred from the usage model. Our method gauges the accuracy of the reliability-testing usage model by calculating the sum of goodness-of-fit values of each traces and scaling the result between 0 and 1. Finally, we present an experimental study on the log of a real web site and discuss the way to use proposed usage model to generate test sequences, as well as strength and weakness of the model for reliability testing.
|
18 |
Testing the Effectiveness of Various Commonly Used Fit Indices for Detecting Misspecifications in Multilevel Structure Equation ModelsHsu, Hsien-Yuan 2009 December 1900 (has links)
Two Monte Carlo studies were conducted to investigate the sensitivity of fit indices in detecting model misspecification in multilevel structural equation models (MSEM) with normally distributed or dichotomous outcome variables separately under various conditions. Simulation results showed that RMSEA and CFI only reflected within-model fit. In addition, SRMR for within-model (SRMR-W) was more sensitive to within-model misspecifications in factor covariances than pattern coefficients regardless of the impact of other design factors. Researchers should use SRMR-W in combination with RMSEA and CFI to evaluate the within-mode. On the other hand, SRMR for between-model (SRMR-B) was less likely to detect between-model misspecifications when ICC decreased. Lastly, the performance of WRMR was dominated by the misfit of within-model. In addition, WRMR was less likely to detect the misspecified between-models when ICC was relative low. Therefore, WRMR can be used to evaluate the between-model fit when the within-models were correctly specified and the ICC was not too small.
|
19 |
On the Arctic Seasonal CycleMortin, Jonas January 2014 (has links)
The seasonal cycle of snow and sea ice is a fundamental feature of the Arctic climate system. In the Northern Hemisphere, about 55 million km2 of sea ice and snow undergo complete melt and freeze processes every year. Because snow and sea ice are much brighter (higher albedo) than the underlying surface, their presence reduces absorption of incoming solar energy at high latitudes. Therefore, changes of the sea-ice and snow cover have a large impact on the Arctic climate and possibly at lower latitudes. One of the most important determining factors of the seasonal snow and sea-ice cover is the timing of the seasonal melt-freeze transitions. Hence, in order to better understand Arctic climate variability, it is key to continuously monitor these transitions. This thesis presents an algorithm for obtaining melt-freeze transitions using scatterometers over both the land and sea-ice domains. These satellite-borne instruments emit radiation at microwave wavelengths and measure the returned signal. Several scatterometers are employed: QuikSCAT (1999–2009), ASCAT (2009–present), and OSCAT (2009–present). QuikSCAT and OSCAT operate at Ku-band (λ=2.2 cm) and ASCAT at C-band (λ=5.7 cm), resulting in slightly different surface interactions. This thesis discusses these dissimilarities over the Arctic sea-ice domain, and juxtaposes the time series of seasonal melt-freeze transitions from the three scatterometers and compares them with other, independent datasets. The interactions of snow and sea ice with other components of the Arctic climate system are complex. Models are commonly employed to disentangle these interactions. But this hinges upon robust and well-formulated models, reached by perpetual testing against observations. This thesis also presents an evaluation of how well eleven state-of-the-art global climate models reproduce the Arctic sea-ice cover and the summer length—given by the melt-freeze transitions—using surface observations of air temperature. / <p>At the time of the doctoral defense, the following papers were unpublished and had a status as follows: Paper 1: In press. Paper 4: Submitted.</p>
|
20 |
Improved Usage Model for Web Application Reliability TestingWan, Bo January 2012 (has links)
Testing the reliability of an application usually requires a good usage model that accurately captures the likely sequences of inputs that the application will receive from the environment. The models being used in the literature are mostly based on Markov chains. They are used to generate test cases that are statistically close to what the applica-tion is expected to receive when in production. In this thesis, we propose a model for reli-ability testing that is created directly from the log file of a web application. Our proposed model is also based on Markov chains and has two components: one component, based on a modified tree, captures the most frequent behaviors, while the other component is another Markov chain that captures infrequent behaviors. The result is a statistically cor-rect model that shows clearly what most users do on the site.
The thesis also presents an evaluation method for estimating the accuracy of vari-ous reliability-testing usage models. The method is based on comparison between ob-served users’ traces and traces inferred from the usage model. Our method gauges the accuracy of the reliability-testing usage model by calculating the sum of goodness-of-fit values of each traces and scaling the result between 0 and 1. Finally, we present an experimental study on the log of a real web site and discuss the way to use proposed usage model to generate test sequences, as well as strength and weakness of the model for reliability testing.
|
Page generated in 0.0417 seconds