Over the past few decades, various computational and mathematical methodologies have been proposed for forecasting seasonal epidemics. In recent years, the deadly effects of enormous pandemics such as the H1N1 influenza virus, Ebola, and Zika, have compelled scientists to find new ways to improve the reliability and accuracy of epidemic forecasts. The improvement and variety of these prediction methods are undeniable. Nevertheless, many challenges remain unresolved in the path of forecasting the outbreaks using surveillance data. Obtaining the clean real-time data has always been an obstacle. Moreover, the surveillance data is usually noisy and handling the uncertainty of the observed data is a major issue for forecasting algorithms. Correct modeling assumptions regarding the nature of the infectious disease is another dilemma. Oversimplified models could lead to inaccurate forecasts, whereas more complicated methods require additional computational resources and information. Without those, the model may not be able to converge to a unique optimum solution. Through the last decade, there has been a significant effort towards achieving better epidemic forecasting algorithms. However, the lack of standard, well-defined evaluating metrics impedes a fair judgment on the proposed methods. This dissertation is divided into two parts. In the first part, we present a Bayesian particle filter calibration framework integrated with an agent-based model to forecast the epidemic trend of diseases like flu and Ebola. Our approach uses Bayesian statistics to estimate the underlying disease model parameters given the observed data and handle the uncertainty in the reasoning. An individual-based model with different intervention strategies could result in a large number of unknown parameters that should be properly calibrated. As particle filter could collapse in very large-scale systems (curse-of-dimensionality problem), achieving the optimum solution becomes more challenging. Our proposed particle filter framework utilizes machine learning concepts to restrain the intractable search space. It incorporates a smart analyzer in the state dynamics unit that examines the predicted and observed data using machine learning techniques to guide the direction and amount of perturbation of each parameter in the searching process. The second part of this dissertation focuses on providing standard evaluation measures for evaluating epidemic forecasts. We present an end-to-end framework that introduces epidemiologically relevant features (Epi-features), error measures, and ranking schema as the main modules of the evaluation process. Lastly, we provide the evaluation framework as a software package named Epi-Evaluator and demonstrate the potentials and capabilities of the framework by applying it to the output of different forecasting methods. / PHD / Epidemics impose substantial costs to societies by deteriorating the public health and disrupting economic trends. In recent years, the deadly effects of wide-spread pandemics such as H1N1, Ebola, and Zika, have compelled scientists to find new ways to improve the reliability and accuracy of epidemic forecasts. The reliable prediction of future pandemics and providing efficient intervention plans for health care providers could prevent or control disease propagations. Over the last decade, there has been a significant effort towards achieving better epidemic forecasting algorithms. The mission, however, is far from accomplished. Moreover, there has been no significant leap towards standard, well-defined evaluating metrics and criteria for a fair performance judgment between the proposed methods. This dissertation is divided into two parts. In the first part, we present a Bayesian particle filter calibration framework integrated with an agent-based model to forecast the epidemic trend of diseases like flu and Ebola. We model the disease propagation via a large scale agent-based model that simulates the disease spread across the contact network of people. The contact network consists of millions of nodes and is constructed based on demographic information of individuals achieved from the census data. The agent-based model’s configurations are mostly unknown parameters that should be properly calibrated. We present a Bayesian particle filter calibration approach to estimate the underlying disease model parameters given the observed data and handle the uncertainty in the reasoning. As particle filter could collapse in very large-scale systems, achieving the optimum solution becomes more challenging. Our proposed particle filter framework utilizes machine learning concepts to restrain the intractable search space. It incorporates a smart analyzer unit that examines the predicted and observed data using machine learning techniques to guide the direction and amount of perturbation of each parameter in the searching process. The second part of this dissertation focuses on providing standard evaluation measures for evaluating and comparing epidemic forecasts. We present a framework that introduces epidemiologically relevant features (Epi-features), error measures, and ranking schema as the main modules of the evaluation process. Lastly, we provide the evaluation framework as a software package named Epi-Evaluator and demonstrate the potentials and capabilities of the framework by applying it to the output of different forecasting methods.

<p>Businesses use forecasts in order to gather information concerning phenomena that are important to them. Since electronic commerce has grown in importance for businesses, forecasts concerning this area are becoming increasingly important. The work presented in this report aims at gathering information useful when improving forecast quality. In practice the report presents a collection of forecasts, concerning business-to-consumer electronic commerce, and a collection of factors affecting electronic commerce forecast outcomes. A categorisation and evaluation of the collected forecasts is performed, this evaluation is done by comparing the forecasts in the categories to the actual outcomes. Problems that occur during the evaluation process, such as problems with forecast wording and scope, are described, and suggestions of how to avoid these problems are provided. Structured methods to categorise and evaluate the forecasts are also presented. Finally, the outcome from the evaluation is analysed using the compiled factors and indications are given of how to use the results in order to improve future forecasting.</p>

Businesses use forecasts in order to gather information concerning phenomena that are important to them. Since electronic commerce has grown in importance for businesses, forecasts concerning this area are becoming increasingly important. The work presented in this report aims at gathering information useful when improving forecast quality. In practice the report presents a collection of forecasts, concerning business-to-consumer electronic commerce, and a collection of factors affecting electronic commerce forecast outcomes. A categorisation and evaluation of the collected forecasts is performed, this evaluation is done by comparing the forecasts in the categories to the actual outcomes. Problems that occur during the evaluation process, such as problems with forecast wording and scope, are described, and suggestions of how to avoid these problems are provided. Structured methods to categorise and evaluate the forecasts are also presented. Finally, the outcome from the evaluation is analysed using the compiled factors and indications are given of how to use the results in order to improve future forecasting.

Chapter 1 examines alternative specifications of futures-based forecasting models to improve upon existing approaches constrained by restrictive assumptions and limited information sets. We replace historical averages with rolling regressions and incorporate current market information through the deviation of the current basis from its historical average. To address potential non-stationarity and structural changes in the cash-futures price relationship, we employ a five-year rolling estimation window. Our findings indicate that the rolling regression approach yields significant improvements in both accuracy and information content of cotton season-average price forecasts, primarily at short forecast horizons. Chapter 2 addresses challenges in vulnerability assessment for semi-arid regions dependent on rainfed agriculture, where extreme weather events pose significant risks to household livelihoods. Despite advancements in remotely sensed technology, accurately estimating weather variability's impact on household livelihoods remains challenging. This study evaluates the effects of weather anomaly measures, spatial resolutions (i.e., geographic level at which the weather anomaly measures are evaluated), and household characteristics on household likelihood of falling into poverty (i.e., vulnerability) estimates. Combining household consumption data for Niger with remotely sensed agro-environmental measures, we find significant variations in vulnerability estimates based on the use of various weather condition measures (3 percentage points, equivalent to 600,000 households), spatial resolutions (8 percentage points, totalling 1.6 million households), and household characteristics (10 percentage points, equivalent to approximately 2 million households). Chapter 3 evaluates student learning outcomes from student involvement in hands-on learning settings, specifically focusing on student-managed investment funds. To assess the changes in the obtained technical and practical skills, we combine knowledge tests with grading rubrics. As part of practical skills, we consider commodity market analysis, critical thinking, informed decision-making, and insightful interpretation of market analysis results. We evaluate our students' understanding of commodity markets and their practical trading skills before and after joining the student-managed investment fund program. We find significant improvements in student learning outcomes, with students showing an average increase of 28% in disciplinary or technical knowledge and 38% in practical skills. Our findings highlight the importance of hands-on learning experiences to bridge the gap between theoretical knowledge and real-world application and in developing the well-rounded skill set demanded by the job market. / Doctor of Philosophy / Chapter 1 explores several alternative specifications of futures-based forecasting models to improve existing approaches constrained by restrictive assumptions and limited information sets. Accurate prediction of cotton prices is vital for the agricultural sector, significantly impacting decisions made by farmers, traders, and policymakers. Reliable forecasts enable farmers to optimize their planting and harvesting strategies, allow traders to manage risk more effectively, and guide policymakers in developing informed agricultural policies. However, the inherent volatility of commodity markets, particularly cotton, presents substantial challenges to price forecasting. Traditional forecasting methods often struggle to capture rapid market changes, resulting in less reliable predictions. Our proposed more responsive forecasting approaches lead to a significant gain in accuracy and information content of cotton price projection and provide valuable insights that can enhance decision-making processes throughout the cotton industry. Chapter 2 explores how extreme weather events, like droughts, affect households in semi-arid regions where people's livelihood largely depends on rain-fed farming. While satellite technology helps monitor environmental changes, it is still challenging to accurately measure how weather changes impact people's lives. Our study focuses on Niger and uses household survey data to assess how various factors influence our understanding of the risk of falling into poverty (i.e., household vulnerability) due to adverse weather events. We found that the methods we use to measure weather conditions, the geographic scale at which we measure them, and the household information we include can all significantly alter our estimates of how many households are at risk of becoming poor. For example, different methods for measuring weather impacts can change estimates of household vulnerability by about 3 percentage points, affecting around 600,000 households. The geographic level (administrative unit level or within a 20 km buffer around an enumeration area) at which we assess weather conditions can shift our estimates by 8 percentage points, which is equivalent to 1.6 million households. Additionally, considering different household characteristics can change our estimates by 10 percentage points, impacting around 2 million households. Our findings are crucial for policymakers who aim to better understand and address the effects of weather on vulnerable communities. Chapter 3 evaluates student learning outcomes from participation in the Commodity Investing by Students program, a student-managed investment fund within the Department of Agricultural and Applied Economics at Virginia Tech. Our study focuses on students from the 2022/23 and 2023/24 academic years, assessing both their technical knowledge and practical skills gained during a year-long involvement in the program. To measure changes in technical skills, we administered knowledge-testing quizzes before and after the training class. Practical skills, such as commodity market analysis, critical thinking, informed decision-making, and insightful interpretation of market analysis results, we evaluated through trading projects submitted during and at the end of the training class. We grade these student submissions using a specific practical skill evaluation rubric. We find significant improvements in student learning outcomes. On average, students demonstrated a 28% increase in disciplinary knowledge and a 38% improvement in practical skills. Our findings highlight the effectiveness of hands-on learning in improving both technical knowledge and practical skills that are highly valued in today's job market.

This study was issued by Swedbank because they wanted too improve their GDP growth forecast capabilites.  A program was developed and tested on six countries; USA, Sweden, Germany, UK, Brazil and Norway. In this paper I investigate if I can reduce forecasting error for GDP growth by taking a smart average from a variety of models compared to both the best individual models and a random walk. I combine the forecasts from four model groups: Vector autoregression, principal component analysis, machine learning and random walk. The smart average is given by a system that give more weight to the predictions of models with a lower historical error. Different weighting schemas are explored; how far into the past should we look? How much should bad performance be punished? I show that for the six countries studied the smart average outperforms the single best model and that for five out of six countries it beats a random walk by at least 25%. / Den här studien beställdes av Swedbank eftersom de ville förbättra sin BNP-prediktionsförmåga. Ett dataprogram utvecklades och testades på sex länder; USA, Sverige, Tyskland, Storbritannien, Brasilien och Norge. I den här rapporten undersöker jag om jag kan minska felmarginalen för BNP-utvecklingsprognoser genom att ta ett smart genomsnitt från flera olika modeller jämfört med både den bästa individuella modellen och en random walk. Jag kombinerar prognoser från fyra modellgrupper: Vektor autoregression, principalkomponentanalys, maskininlärning och random walk. Det smarta genomsnittet skapas genom att ge mer vikt till de modeller som har lägst historiskt felmarginal. Olika viktningsscheman utforskas; hur långt bak i tiden ska vi mäta? Hur hårt ska dåliga prediktioner bestraffas? Jag visar att för de sex länderna i studien presterar det smarta genomsnittet bättre än den enskilt bästa modellen och fem av de sex länderna slår en random walk med mer än 25%.

In the field of aviation forecasting is used, among other things, to determine the number of passengers to expect for each flight. This is beneficial in the practice of revenue management, as the forecast is used as a base when setting the price for each flight. In this study, a forecast evaluation has been done on seven different routes with a total of 61 different flights, using four different methods. These are: Mean Absolute Scaled Error (MASE), Mean Absolute Percentage Error (MAPE), Tracking Signal, and a goodness of fit test to determine if the forecast errors are normally distributed. The MASE has been used to determine if the passenger forecasts are better or worse than a naïve forecast, while the MAPE provides an error value for internal comparisons between the flights. The Tracking Signal and the normal distribution test have been used in order to determine whether a flight has bias or not towards under- or overforecasting. The results point towards a general underforecast across all studied flights. A total of 89 % of the forecasts perform better than the naïve forecast, with an average MASE value of 0,78. As such, the forecast accuracy is better than that of the naïve forecast. There are however large error values among the observed flights, affecting the MAPE average. The MAPE average is 38,53 % while the median is 30,60 %. The measure can be used for internal comparisons, and one such way is to use the average value as a benchmark in order to focus on improving those forecasts with a higher than average MAPE. The authors have found that the MASE and MAPE are useful in measuring forecast accuracy and as such the recommendation of the authors is that these two error measures can be used together to evaluate forecast accuracy at frequent intervals. In addition to this there is value in examining the error distribution in conjunction with the Mean Error when searching for bias, as this will indicate if there is systematic error present.

[pt] O presente trabalho investiga o uso de métodos de Machine Learning (ML) para efetuar estimativas para o trimestre corrente (nowcasts) da taxa de crescimento do PIB Real dos EUA. Esses métodos conseguem lidar com um grande volume de dados e séries com calendários de publicação dessincronizados, e os nowcasts são atualizados cada vez que novos dados são publicados ao longo do trimestre. Um exercício pseudo-out-of-sample é proposto para avaliar a performance de previsão e analisar o padrão de seleção de variável desses modelos. O método de ML que merece o maior destaque é o Target Factor, que supera o usualmente adotado DFM para alguns vintages dentro do trimestre. Ademais, as variáveis selecionadas apresentam consistência entre os modelos e com a intuição. / [en] This paper examines the use of Machine Learning (ML) models to compute estimates of current-quarter US Real GDP growth rate (nowcasts). These methods can handle large data sets with unsynchronized release dates, and nowcasts are updated each time new data are released along the quarter. A pseudo-out-of-sample exercise is proposed to assess forecasting performance and to analyze the variable selection pattern of these models. The ML method that deserves more attention is the Target Factor, which overcomes the usually adopted dynamic factor model for some predictions vintages in the quarter. We also analyze the variables selected, which are consistent between models and intuition.

<p>The United States Department of Agriculture (USDA) forecasts the Farm Balance Sheet  each year. The Farm Balance Sheet provides an estimate of the value of physical and financial  assets in the United States agriculture sector over time (USDA, 2023). The forecasts evaluated in  this paper are related to assets and debt in the farm sector, including total farm assets, farm assets  real estate, total farm debt, farm debt real estate, and farm debt non-real estate. These forecasts predict the growth in the agricultural sector and help various stakeholders, such as policy makers, USDA program administrators, and agricultural lenders make important decisions. Given the  importance of these forecasts in the agricultural sector, it is the main objective of this research to examine the degree to which the Farm Balance Sheet forecasts are optimal (unbiased and efficient).  During this study, forecasts from the Farm Balance Sheet in the 1986-2021 period are found to be unbiased using Holden and Peel test (1990). Also, using efficiency tests by Nordhaus (1987), it  was found that forecasts from the Farm Balance Sheet are inefficient. This, suggests all the  information is not efficiently incorporated when the forecast is produced .</p>

<p> This dissertation consists of three essays that evaluate long-term agricultural projections. The first essay focus on evaluating Congressional Budget Office’s (CBO) baseline projection of United States Department of Agriculture (USDA) mandatory farm and nutrition programs. The second essay examine USDA soybean ending stock projections, and the third essay investigate impact of macroeconomic assumptions on USDA’s baseline farm income projections.  We use publicly available data from Congressional Budget Office (CBO) and United States Department of Agriculture (USDA)</p>

Yes / This is a pioneering effort to test the comparative performance of two competing models for out-of-sample forecasting the term structure of volatility of crude oil price changes employing both symmetric and asymmetric evaluation criteria. Under symmetric error statistics, our empirical model using the estimated growth factor of volatility through time is overall superior, and it beats in most cases the benchmark model of the square-root-of-time for holding periods between one and 250 days. Under asymmetric error statistics, if over-prediction (under-prediction) of volatility is undesirable, the empirical (benchmark) model is consistently superior. Relative performance of the empirical model is much higher for holding periods up to fifty days.