<p>In this thesis a mixture density network has been constructed to predict steel hardenability for a given alloy composition. Throughout the work hardenability is expressed in terms of jominy profiles according to the standard jominy test. A piecewise linear description of the jominy profile has been developed to solve the problem of missing data, model identification from data based on different units and measurement uncertainty. When the underlying physical processes are complex and not well understood, as the case with hardenability modelling, mixture density networks, which are an extension of neural networks, offer a strong non-linear modelling alternative. Mixture density networks model conditional probability densities, from which it is possible to determine any statistical property. Here the model output is presented in terms of expectation values along with confidence interval. This statistical output facilitates future extension of the model towards optimisation of alloy cost. A good agreement has been obtained between the experimental and the calculated data. In order to ensure the reliability of the model in service, novelty detection of the input data is performed.</p>

In this thesis a mixture density network has been constructed to predict steel hardenability for a given alloy composition. Throughout the work hardenability is expressed in terms of jominy profiles according to the standard jominy test. A piecewise linear description of the jominy profile has been developed to solve the problem of missing data, model identification from data based on different units and measurement uncertainty. When the underlying physical processes are complex and not well understood, as the case with hardenability modelling, mixture density networks, which are an extension of neural networks, offer a strong non-linear modelling alternative. Mixture density networks model conditional probability densities, from which it is possible to determine any statistical property. Here the model output is presented in terms of expectation values along with confidence interval. This statistical output facilitates future extension of the model towards optimisation of alloy cost. A good agreement has been obtained between the experimental and the calculated data. In order to ensure the reliability of the model in service, novelty detection of the input data is performed.

In response to financial crises and opaque practices, governmental entities and financial regulatory bodies have implemented several pieces of legislature and directives meant to protect investors and increase transparency. Such regulations often impose strict liquidity requirements and robust estimations of the risk borne by a financial firm at any given time. Value at Risk (VaR) measures how much an investment can stand to lose with a certain probability over a specified period of time and is ubiquitous in its use by institutional investors and banks alike. In practice, VaR estimations are often computed from simulations of historical data or parameterized distributions.  Inspired by the recent success of Arimond et al. (2020) in using a neural network for VaR estimation, we apply a combination of recurrent neural networks and a mixture density output layer for generating mixture density distributions of future portfolio returns from which VaR estimations are made. As in Arimond et al., we suppose the existence of two regimes stylized as bull and bear markets and employ Monte Carlo simulation to generate predictions of future returns. Rather than use a swappable architecture for the parameters in the mixture density distribution, we here let all parameters be generated endogenously in the neural network. The model's success is then validated through Christoffersen tests and by comparing it to the benchmark VaR estimation models, i.e., the mean-variance approach and historical simulation.  We conclude that recurrent mixture density networks show limited promise for the task of predicting effective VaR estimates if used as is, due to the model consistently overestimating the true portfolio loss. However, for practical use, encouraging results were achieved when manually shifting the predictions based on an average of the overestimation observed in the validation set. Several theories are presented as to why overestimation occurs, while no definitive conclusion could be drawn. As neural networks serve as black box models, their use for conforming to regulatory requirements is thus deemed questionable, likewise the assumption that financial data carries an inherent pattern with potential to be accurately approximated. Still, reactivity in the VaR estimations by the neural network is significantly more pronounced than in the benchmark models, motivating continued experimentation with machine learning methods for risk management purposes. Future research is encouraged to identify the source of overestimation and explore different machine learning techniques to attain more accurate VaR predictions. / I respons till finanskriser och svårfattlig verksamhetsutövning har överstatliga organ och finansmyndigheter implementerat lagstiftning och utfärdat direktiv i syfte att skydda investerare och öka transparens. Sådana regleringar förelägger ofta strikta likviditetskrav och krav på redogörelse av den finansiella risk som en marknadsaktör har vid en given tidpunkt. Value at Risk (VaR) mäter hur mycket en investering kan förlora med en viss sannolikhet över en på förhand bestämd tidsperiod och är allestädes närvarande i dess användning av institutionella investerare såväl som banker. I praktiken beräknas estimeringar av VaR framför allt via simulering av historisk data eller en parametrisering av densamma. Inspirerade av Arimond et als (2020) framgång i användning av neurala nätverk för VaR estimering applicerar vi en kombination av "recurrent" neurala nätverk och ett "mixture density output"-lager i syfte att generera mixture density-fördelningar för framtida portföljavkastning. Likt Arimond et al. förutsätter vi existensen av två regimer stiliserade som "bull" och "bear" marknader och applicerar Monte Carlo simulering för att generera prediktioner av framtida avkastning. Snarare än att använda en utbytbar arkitektur för parametrarna i mixture density-fördelningen låter vi samtliga parametrar genereras endogent i det neurala nätverket. Modellens framgång valideras via Christoffersens tester samt jämförelse med de prevalenta metoderna för att estimera VaR, det vill säga mean-variance-metoden och historisk simulering. Vår slutsats är att recurrent mixture density-nätverk enskilt uppvisar begränsad tillämpbarhet för uppgiften av att uppskatta effektiva VaR estimeringar, eftersom modellen konsekvent överestimerar den sanna portföljförlusten. För praktisk användning visade modellen däremot uppmuntrande resultat när dess prediktioner manuellt växlades ner baserat på ett genomsnitt av överestimeringen observerad i valideringsdatat. Flera teorier presenteras kring varför överestimeringen sker men ingen definitiv slutsats kunde dras. Eftersom neurala nätverksmodeller agerar som svarta lådor är deras potential till att bemöta regulatoriska krav tveksam, likväl antagandet att finansiell data har ett inneboende mönster kapabelt till att approximeras. Med detta sagt uppvisar neurala nätverkets VaR estimeringar betydligt mer reaktivitet än i de prevalenta modellerna, varför fortsatt experimentation med maskininlärningsmetoder för riskhantering ändå kan vara motiverat. Framtida forskning uppmuntras för att identifera källan till överestimeringen, samt utforskningen av andra maskininlärningsmetoder för att erhålla mer precisa VaR prediktioner.

In autonomous driving, scenario generation can play a critical role when it comes to the verification of the autonomous driving software. Since uncertainty is a major component in driving, there cannot be just one right answer to a prediction for the trajectory or the behaviour, and it becomes important to account for and model that uncertainty. Several approaches have been tried for generating the future scenarios for a vehicle and one such pioneering work set out to model the behaviour of the vehicles probabilistically while tackling the challenges of representation, flexibility, and transferability within one system. The proposed system is called the Semantic Graph Network (SGN) which utilizes feedforward neural networks, Gated Recurrent Units (GRU), and a generative model called the Mixed Density Network to serve its purpose. This thesis project set out in the direction of the implementation of this research work in the context of highway merger scenario and consists of three parts. The first part involves basic data analysis for the employed dataset, whereas the second part involves a model that implements certain parts of the SGN including a variation of the context encoding and the Mixture Density Network. The third and the final part is an attempt to recreate the SGN itself. While the first and the second parts were implemented successfully, for the third part, only certain objectives could be achieved. / Vid autonom körning kan scenariegenerering spela en avgörande roll när det gäller verifieringen av programvaran för autonom körning. Eftersom osäkerhet är en viktig komponent i körning kan det inte bara finnas ett rätt svar på en förutsägelse av banan eller beteendet, och det blir viktigt att redogöra för och modellera den osäkerheten. Flera tillvägagångssätt har prövats för att generera framtidsscenarierna för ett fordon och ett sådant banbrytande arbete gick ut på att modellera fordonens beteende sannolikt samtidigt som utmaningarna med representation, flexibilitet och överförbarhet inom ett system hanteras. Det föreslagna systemet kallas Semantic Graph Network (SGN) som använder neurala nätverk, Gated Recurrent Units (GRU) och en generativ modell som kallas Mixed Density Network för att tjäna sitt syfte. Detta examensarbete riktar sig mot genomförandet av detta forskningsarbete i samband med motorvägssammanslagningsscenariot och består av tre delar. Den första delen involverar grundläggande dataanalys för den använda datamängden, medan den andra delen involverar en modell som implementerar vissa delar av SGN inklusive en variation av kontextkodningen och Mixture Density Network. Den tredje och sista delen är ett försök att återskapa själva SGN. Även om den första och den andra delen genomfördes framgångsrikt, kunde endast vissa mål uppnås för den tredje delen.

Prediction of rock and fluid properties such as porosity, clay content, and water saturation is essential for exploration and development of hydrocarbon reservoirs. Rock and fluid property maps obtained from such predictions can be used for optimal selection of well locations for reservoir development and production enhancement. Seismic data are usually the only source of information available throughout a field that can be used to predict the 3D distribution of properties with appropriate spatial resolution. The main challenge in inferring properties from seismic data is the ambiguous nature of geophysical information. Therefore, any estimate of rock and fluid property maps derived from seismic data must also represent its associated uncertainty. In this study we develop a computationally efficient mathematical technique based on neural networks to integrate measured data and a priori information in order to reduce the uncertainty in rock and fluid properties in a reservoir. The post inversion (a posteriori) information about rock and fluid properties are represented by the joint probability density function (PDF) of porosity, clay content, and water saturation. In this technique the a posteriori PDF is modeled by a weighted sum of Gaussian PDF’s. A so-called mixture density network (MDN) estimates the weights, mean vector, and covariance matrix of the Gaussians given any measured data set. We solve several inverse problems with the MDN and compare results with Monte Carlo (MC) sampling solution and show that the MDN inversion technique provides good estimate of the MC sampling solution. However, the computational cost of training and using the neural network is much lower than solution found by MC sampling (more than a factor of 104 in some cases). We also discuss the design, implementation, and training procedure of the MDN, and its limitations in estimating the solution of an inverse problem. In this thesis we focus on data from a deep offshore field in Africa. Our goal is to apply the MDN inversion technique to obtain maps of petrophysical properties (i.e., porosity, clay content, water saturation), and petrophysical facies from 3D seismic data. Petrophysical facies (i.e., non-reservoir, oil- and brine-saturated reservoir facies) are defined probabilistically based on geological information and values of the petrophysical parameters. First, we investigate the relationship (i.e., petrophysical forward function) between compressional- and shear-wave velocity and petrophysical parameters. The petrophysical forward function depends on different properties of rocks and varies from one rock type to another. Therefore, after acquisition of well logs or seismic data from a geological setting the petrophysical forward function must be calibrated with data and observations. The uncertainty of the petrophysical forward function comes from uncertainty in measurements and uncertainty about the type of facies. We present a method to construct the petrophysical forward function with its associated uncertainty from the both sources above. The results show that introducing uncertainty in facies improves the accuracy of the petrophysical forward function predictions. Then, we apply the MDN inversion method to solve four different petrophysical inverse problems. In particular, we invert P- and S-wave impedance logs for the joint PDF of porosity, clay content, and water saturation using a calibrated petrophysical forward function. Results show that posterior PDF of the model parameters provides reasonable estimates of measured well logs. Errors in the posterior PDF are mainly due to errors in the petrophysical forward function. Finally, we apply the MDN inversion method to predict 3D petrophysical properties from attributes of seismic data. In this application, the inversion objective is to estimate the joint PDF of porosity, clay content, and water saturation at each point in the reservoir, from the compressional- and shear-wave-impedance obtained from the inversion of AVO seismic data. Uncertainty in the a posteriori PDF of the model parameters are due to different sources such as variations in effective pressure, bulk modulus and density of hydrocarbon, uncertainty of the petrophysical forward function, and random noise in recorded data. Results show that the standard deviations of all model parameters are reduced after inversion, which shows that the inversion process provides information about all parameters. We also applied the result of the petrophysical inversion to estimate the 3D probability maps of non-reservoir facies, brine- and oil-saturated reservoir facies. The accuracy of the predicted oil-saturated facies at the well location is good, but due to errors in the petrophysical inversion the predicted non-reservoir and brine-saturated facies are ambiguous. Although the accuracy of results may vary due to different sources of error in different applications, the fast, probabilistic method of solving non-linear inverse problems developed in this study can be applied to invert well logs and large seismic data sets for petrophysical parameters in different applications.

Deep learning has gained traction in thermal engineering due to its applications to process simulations, the deeper insights it can provide and its abilities to circumvent the shortcomings of classic thermodynamic simulation approaches by capturing complex inter-dependencies. This works sets out to apply probabilistic deep learning to power plant operations using historic plant data. The first study presented, entails the development of a steady-state mixture density network (MDN) capable of predicting effective heat transfer coefficients (HTC) for the various heat exchanger components inside a utility scale boiler. Selected directly controllable input features, including the excess air ratio, steam temperatures, flow rates and pressures are used to predict the HTCs. In the second case study, an encoder-decoder mixturedensity network (MDN) is developed using recurrent neural networks (RNN) for the prediction of utility-scale air-cooled condenser (ACC) backpressure. The effects of ambient conditions and plant operating parameters, such as extraction flow rate, on ACC performance is investigated. In both case studies, hyperparameter searches are done to determine the best performing architectures for these models. Comparisons are drawn between the MDN model versus standard model architecture in both case studies. The HTC predictor model achieved 90% accuracy which equates to an average error of 4.89 W m2K across all heat exchangers. The resultant time-series ACC model achieved an average error of 3.14 kPa, which translate into a model accuracy of 82%.

In recent years, machine learning has become utilized in external radiation therapy treatment planning. This involves automatic generation of treatment plans based on CT-scans and other spatial information such as the location of tumors and organs. The utility lies in relieving clinical staff from the labor of manually or semi-manually creating such plans. Rather than predicting a deterministic plan, there is great value in modeling it stochastically, i.e. predicting a probability distribution of dose from CT-scans and delineated biological structures. The stochasticity inherent in the RT treatment problem stems from the fact that a range of different plans can be adequate for a patient. The particular distribution can be thought of as the prevalence in preferences among clinicians. Having more information about the range of possible plans represented in one model entails that there is more flexibility in forming a final plan. Additionally, the model will be able to reflect the potentially conflicting clinical trade-offs; these will occur as multimodal distributions of dose in areas where there is a high variance. At RaySearch, the current method for doing this uses probabilistic random forests, an augmentation of the classical random forest algorithm. A current direction of research is learning the probability distribution using deep learning. A novel parametric approach to this is letting a suitable deep neural network approximate the parameters of a Gaussian mixture model in each volume element. Such a neural network is known as a mixture density network. This thesis establishes theoretical results of artificial neural networks, mainly the universal approximation theorem, applied to the activation functions used in the thesis. It will then proceed to investigate the power of deep learning in predicting dose distributions, both deterministically and stochastically. The primary objective is to investigate the feasibility of mixture density networks for stochastic prediction. The research question is the following. U-nets and Mixture Density Networks will be combined to predict stochastic doses. Does there exist such a network, powerful enough to detect and model bimodality? The experiments and investigations performed in this thesis demonstrate that there is indeed such a network. / Under de senaste åren har maskininlärning börjat nyttjas i extern strålbehandlingsplanering. Detta involverar automatisk generering av behandlingsplaner baserade på datortomografibilder och annan rumslig information, såsom placering av tumörer och organ. Nyttan ligger i att avlasta klinisk personal från arbetet med manuellt eller halvmanuellt skapa sådana planer. I stället för att predicera en deterministisk plan finns det stort värde att modellera den stokastiskt, det vill säga predicera en sannolikhetsfördelning av dos utifrån datortomografibilder och konturerade biologiska strukturer. Stokasticiteten som förekommer i strålterapibehandlingsproblemet beror på att en rad olika planer kan vara adekvata för en patient. Den särskilda fördelningen kan betraktas som förekomsten av preferenser bland klinisk personal. Att ha mer information om utbudet av möjliga planer representerat i en modell innebär att det finns mer flexibilitet i utformningen av en slutlig plan. Dessutom kommer modellen att kunna återspegla de potentiellt motstridiga kliniska avvägningarna; dessa kommer påträffas som multimodala fördelningar av dosen i områden där det finns en hög varians. På RaySearch används en probabilistisk random forest för att skapa dessa fördelningar, denna metod är en utökning av den klassiska random forest-algoritmen. En aktuell forskningsriktning är att generera in sannolikhetsfördelningen med hjälp av djupinlärning. Ett oprövat parametriskt tillvägagångssätt för detta är att låta ett lämpligt djupt neuralt nätverk approximera parametrarna för en Gaussisk mixturmodell i varje volymelement. Ett sådant neuralt nätverk är känt som ett mixturdensitetsnätverk. Den här uppsatsen fastställer teoretiska resultat för artificiella neurala nätverk, främst det universella approximationsteoremet, tillämpat på de aktiveringsfunktioner som används i uppsatsen. Den fortsätter sedan att utforska styrkan av djupinlärning i att predicera dosfördelningar, både deterministiskt och stokastiskt. Det primära målet är att undersöka lämpligheten av mixturdensitetsnätverk för stokastisk prediktion. Forskningsfrågan är följande. U-nets och mixturdensitetsnätverk kommer att kombineras för att predicera stokastiska doser. Finns det ett sådant nätverk som är tillräckligt kraftfullt för att upptäcka och modellera bimodalitet? Experimenten och undersökningarna som utförts i denna uppsats visar att det faktiskt finns ett sådant nätverk.