Laminar-turbulent transition has a strong impact on aerodynamic performance in many practical applications. Hence, there is a practical need for developing reliable and efficient transition prediction models, which form a critical element of the CFD process for aerospace vehicles across multiple flow regimes. This dissertation explores machine learning approaches to develop transition models using data from computations based on linear stability theory. Such data provide strong correlation with the underlying physics governed by linearized disturbance equations. In the proposed transition model, a convolutional neural network-based model encodes information from boundary layer profiles into integral quantities. Such automated feature extraction capability enables generalization of the proposed model to multiple instability mechanisms, even for those where physically defined shape factor parameters cannot be defined/determined in a consistent manner. Furthermore, sequence-to-sequence mapping is used to predict the transition location based on the mean boundary layer profiles. Such an end-to-end transition model provides a significantly simplified workflow. Although the proposed model has been analyzed for two-dimensional boundary layer flows, the embedded feature extraction capability enables their generalization to other flows as well. Neural network-based nonlinear functional approximation has also been presented in the context of transport equation-based closure models. Such models have been examined for their computational complexity and invariance properties based on the transport equation of a general scalar quantity. The data-driven approaches explored here demonstrate the potential for improved transition prediction models. / Doctor of Philosophy / Surface skin friction and aerodynamic heating caused by the flow over a body significantly increases due to the transition from laminar to turbulent flow. Hence, efficient and reliable prediction of transition onset location is a critical component of simulating fluid flows in engineering applications. Currently available transition prediction tools do not provide a good balance between computational efficiency and accuracy. This dissertation explores machine learning approach to develop efficient and reliable models for predicting transition in a significantly simplified manner. Convolutional neural network is used to extract features from the state of boundary layer flow at each location along the body. These extracted features are then processed sequentially using recurrent neural network to predict the amplification of instabilities in the flow, which is directly correlated to the onset of transition. Such an automated nature of feature extraction enables the generalization of this model to multiple transition mechanisms associated with different flow conditions and geometries. Furthermore, an end-to-end mapping from flow data to transition prediction requires no user expertise in stability theory and provides a significantly simplified workflow as compared to traditional stability-based computations. Another category of neural network-based models (known as neural operators) is also examined which can learn functional mapping from input variable field to output quantities. Such models can learn directly from data for complex set of problems, without the knowledge of underlying governing equations. Such attribute can be leveraged to develop a transition prediction model which can be integrated seamlessly in flow solvers. While further development is needed, such data-driven models demonstrate the potential for improved transition prediction models.

The automotive industry is inevitably experiencing a paradigm shift from fossil fuels to electric powertrain with significant technological breakthroughs in vehicle electrification. Emerging hybrid electric vehicles were one of the first steps towards cleaner and greener vehicles with a higher fuel economy and lower emission levels. The energy management strategy in hybrid electric vehicles determines the power flow pattern and significantly affects vehicle performance. Therefore, in this thesis, a learning-based strategy is proposed to address the energy management problem of a hybrid electric vehicle in various driving conditions. The idea of a deep recurrent neural network-based energy management strategy is proposed, developed, and evaluated. Initially, a hybrid electric vehicle model with a rule-based supervisory controller is constructed for this case study to obtain training data for the deep recurrent neural network and to evaluate the performance of the proposed energy management strategy. Secondly, due to its capabilities to remember historical data, a long short-term memory recurrent neural network is designed and trained to estimate the powertrain control variables from vehicle parameters. Extensive simulations are conducted to improve the model accuracy and ensure its generalization capability. Also, several hyper-parameters and structures are specifically tuned and debugged for this purpose. The novel proposed energy management strategy takes sequential data as input to capture the characteristics of both driver and controller behaviors and improve the estimation/prediction accuracy. The energy management controller is defined as a time-series problem, and a network predictor module is implemented in the system-level controller of the hybrid electric vehicle model. According to the simulation results, the proposed strategy and prediction model demonstrated lower fuel consumption and higher accuracy compared to other learning-based energy management strategies. / Thesis / Master of Applied Science (MASc)

In this work, we produce several prediction models for aspects of hospital emergency departments. Firstly, we demonstrate the use of a recurrent neural network to predict the rate of patient arrival at a hospital emergency department. The prediction is made on a per hour basis using date, time, calendar, and weather information. Then, we present our comparison of two prediction systems on the task of replicating the human decisions of patient admittance in a typical American emergency department. Again, a recurrent neural network (RNN) was trained to learn the task of selecting the next patient from the waiting room/queue to be admitted for treatment. Lastly, we present our attempt to produce a regression model that can predict the likelihood that a given patient will leave after waiting a specific amount of time in the emergency department’s waiting-room/queue. Such a model could be used to optimize the patient’s waiting-room/queue of an ED to minimize the likelihood of patients leaving without receiving care.

No description available.

<p> This thesis presents a novel hybrid methodology using Recurrent Neural Network and Dynamic Time Warping to solve the mode estimation problem of a radar warning receiver (RWR). The RWR is an electronic support (ES) system with the primary objective to estimate the threat posed by an unfriendly (hostile) radar in an electronic warfare (EW) environment. One such radar is the multi-function radar (MFR), which employs complex signal architecture to perform multiple tasks. As the threat posed by the radar directly depends on its current mode of operation, it is vital to estimate and track the mode of the radar. The proposed method uses a recurrent neural network (echo state network and recurrent multi-layer perceptron) trained in a supervised manner, with the dynamic time warping algorithm as the post processor to estimate the mode of operation. A grid filter in Bayesian framework is then applied to the dynamic time warp estimate to provide an accurate posterior estimate of the operational mode of the MFR. This novel approach is tested on an EW scenario via simulation by employing a hypothetical MFR. Based on the simulation results, we conclude that the hybrid echo state network is more suitable than its recurrent multi-layer perceptron counterpart for the mode estimation problem of a RWR.</p> / Thesis / Master of Applied Science (MASc)

Recurrent neural networks (RNNs) have been widely used for supervised pattern recognition and exploring the underlying spatio-temporal correlation. However, due to the vanishing/exploding gradient problem, training a fully connected RNN in many cases is very difficult or even impossible. The difficulties of training traditional RNNs, led us to reservoir computing (RC) which recently attracted a lot of attention due to its simple training methods and fixed weights at its recurrent layer. There are three different categories of RC systems, namely, echo state networks (ESNs), liquid state machines (LSMs), and delayed feedback reservoirs (DFRs). In this dissertation a novel structure of RNNs which is inspired by dynamic delayed feedback loops is introduced. In the reservoir (recurrent) layer of DFR, only one neuron is required which makes DFRs extremely suitable for hardware implementations. The main motivation of this dissertation is to introduce an energy efficient, and easy to train RNN while this model achieves high performances in different tasks compared to the state-of-the-art. To improve the energy efficiency of our model, we propose to adopt spiking neurons as the information processing unit of DFR. Spiking neural networks (SNNs) are the most biologically plausible and energy efficient class of artificial neural networks (ANNs). The traditional analog ANNs have marginal similarity with the brain-like information processing. It is clear that the biological neurons communicate together through spikes. Therefore, artificial SNNs have been introduced to mimic the biological neurons. On the other hand, the hardware implementation of SNNs have shown to be extremely energy efficient. Towards achieving this overarching goal, this dissertation presents a spiking DFR (SDFR) with novel encoding schemes, and defense mechanisms against adversarial attacks. To verify the effectiveness and performance of the SDFR, it is adopted in three different applications where there exists a significant Spatio-temporal correlations. These three applications are attack detection in smart grids, spectrum sensing of multi-input-multi-output(MIMO)-orthogonal frequency division multiplexing (OFDM) Dynamic Spectrum Sharing (DSS) systems, and video-based face recognition. In this dissertation, the performance of SDFR is first verified in cyber attack detection in Smart grids. Smart grids are a new generation of power grids which guarantee a more reliable and efficient transmission and delivery of power to the costumers. A more reliable and efficient power generation and distribution can be realized through the integration of internet, telecommunication, and energy technologies. The convergence of different technologies, brings up opportunities, but the challenges are also inevitable. One of the major challenges that pose threat to the smart grids is cyber-attacks. A novel method is developed to detect false data injection (FDI) attacks in smart grids. The second novel application of SDFR is the spectrum sensing of MIMO-OFDM DSS systems. DSS is being implemented in the fifth generation of wireless communication systems (5G) to improve the spectrum efficiency. In a MIMO-OFDM system, not all the subcarriers are utilized simultaneously by the primary user (PU). Therefore, it is essential to sense the idle frequency bands and assign them to the secondary user (SU). The effectiveness of SDFR in capturing the spatio-temporal correlation of MIMO-OFDM time-series and predicting the availability of frequency bands in the future time slots is studied as well. In the third application, the SDFR is modified to be adopted in video-based face recognition. In this task, the SDFR is leveraged to recognize the identities of different subjects while they rotate their heads in different angles. Another contribution of this dissertation is to propose a novel encoding scheme of spiking neurons which is inspired by the cognitive studies of rats. For the first time, the multiplexing of multiple neural codes is introduced and it is shown that the robustness and resilience of the spiking neurons is increased against noisy data, and adversarial attacks, respectively. Adversarial attacks are small and imperceptible perturbations of the input data, which have shown to be able to fool deep learning (DL) models. So far, many adversarial attack and defense mechanisms have been introduced for DL models. Compromising the security and reliability of artificial intelligence (AI) systems is a major concern of government, industry and cyber-security researchers, in that insufficient protections can compromise the security and privacy of everyone in society. Finally, a defense mechanism to protect spiking neurons against adversarial attacks is introduced for the first time. In a nutshell, this dissertation presents a novel energy efficient deep spiking recurrent neural network which is inspired by delayed dynamic loops. The effectiveness of the introduced model is verified in several different applications. At the end, novel encoding and defense mechanisms are introduced which improve the robustness of the model against noise and adversarial attacks. / Doctor of Philosophy / The ultimate goal of artificial intelligence (AI) is to mimic the human brain. Artificial neural networks (ANN) are an attempt to realize that goal. However, traditional ANNs are very far from mimicking biological neurons. It is well-known that biological neurons communicate with one another through signals in the format of spikes. Therefore, artificial spiking neural networks (SNNs) have been introduced which behave more similarly to biological neurons. Moreover, SNNs are very energy efficient which makes them a suitable choice for hardware implementation of ANNs (neuromporphic computing). Despite the many benefits that are brought about by SNNs, they are still behind traditional ANNs in terms of performance. Therefore, in this dissertation, a new structure of SNNs is introduced which outperforms the traditional ANNs in three different applications. This new structure is inspired by delayed dynamic loops which exist in biological brains. The main objective of this novel structure is to capture the spatio-temporal correlation which exists in time-series while the training overhead and power consumption is reduced. Another contribution of this dissertation is to introduce novel encoding schemes for spiking neurons. It is clear that biological neurons leverage spikes, but the language that they use to communicate is not clear. Hence, the spikes require to be encoded in a certain language which is called neural spike encoding scheme. Inspired by the cognitive studies of rats, a novel encoding scheme is presented. Lastly, it is shown that the introduced encoding scheme increases the robustness of SNNs against noisy data and adversarial attacks. AI models including SNNs have shown to be vulnerable to adversarial attacks. Adversarial attacks are minor perturbations of the input data that can cause the AI model to misscalassify the data. For the first time, a defense mechanism is introduced which can protect SNNs against such attacks.

A protein is a biopolymer of amino acids that encodes a particular function. Given that there are 20 amino acids possible at each site, even a short protein of 100 amino acids has $20^{100}$ possible variants, making it unrealistic to evaluate all possible sequences in sequence level space. This search space could be reduced by considering the fact that billions of years of evolution exerting a constant pressure has left us with only a small subset of protein sequences that carry out particular cellular functions. The portion of amino acid space occupied by actual proteins found in nature is therefore much smaller than that which is possible cite{kauffman1993origins}. By examining related proteins that share a conserved function and common evolutionary history (heretofore referred to as protein families), it is possible to identify common motifs that are shared. Examination of these motifs allows us to characterize protein families in greater depth and even generate new ``in silico" proteins that are not found in nature, but exhibit properties of a particular protein family. Using novel deep learning approaches and leveraging the large volume of genomic data that is now available due to high-throughput DNA sequencing, it is now possible to examine protein families in a scale and resolution that has never before been possible. By using this abundance of data to learn high dimensional representations of amino acids sequences, in this work, we show that it is possible to generate novel sequences from a particular protein family. Such a deep sequential model-based approach has great value for bioinformatics and biotechnological applications due to its rapid sampling abilities. / Master of Science / Proteins are one of the most important functional biological elements. These are composed of amino acids which link together to form different shapes which might encode a particular function. These proteins may act independently or might form ``complexes" to have a particular function. Therefore, understanding them is of utmost importance. Due to the fact that there are 20 amino acids even a protein sequence fragment of length 5 can have more than 3 million different combinations. Given, that proteins are generally 1000 amino acids long, looking at all the possibilities is next to impossible. In this work, by leveraging the ``deep learning" paradigm and the vast amount of data available, we try to model these proteins and generate new proteins belonging to a specific ``protein family." This approach has great value for bioinformatics and biotechnological applications due to its rapid sampling abilities.

With the inception of music streaming and media content delivery platforms, there has been a tremendous increase in the music available on the internet and the metadata associated with it. In this study, we address the problem of violin artist identification, which tries to classify the performing artist based on the learned features. Even though numerous previous works studied the problem in detail and developed features and deep learning models that can be used, an interesting fact was that most studies focused on artist identification in western popular music and less on Indian classical music. For the same reason, there was no standardized dataset for this purpose. Hence, we curated a new dataset consisting of audio recordings from 6 renowned South Indian Carnatic violin artists. In this study, we explore the use of log-Mel-spectrogram feature and the embeddings generated by a pre-learned VGGish network on a Convolutional Neural Network and Convolutional Recurrent Neural Network Model. From the experiments, we observe that the Convolutional Recurrent Neural Network model trained using the log-Mel-spectrogram feature gave the optimal performance with a classification accuracy of 71.70%. / Med starten av plattformar för musikströmning och leverans av mediainnehåll har det skett en enorm ökning av musiken tillgänglig på internet och den metadata som är associerad med den. I denna studie tar vi upp problemet med fiolkonstnärsidentifikation, som försöker klassificera den utövande konstnären utifrån de inlärda dragen. Även om många tidigare verk studerade problemet i detalj och utvecklade funktioner och modeller för djupinlärning som kan användas, var ett intressant faktum att de flesta studier fokuserade på artistidentifiering i västerländsk populärmusik och mindre på indisk klassisk musik. Av samma anledning fanns det ingen standardiserad datauppsättning för detta ändamål. Därför kurerade vi en ny datauppsättning bestående av ljudinspelningar från 6 kända sydindiska karnatiska violinkonstnärer. I den här studien utforskar vi användningen av log-Melspektrogramfunktionen och inbäddningarna som genereras av ett förinlärt VGGishnätverk på ett Convolutional Neural Network och Convolutional Recurrent Neural Network Model. Från experimenten observerar vi att modellen Convolutional Recurrent Neural Network tränad med hjälp av log-Mel-spektrogramfunktionen gav optimal prestanda med en klassificeringsnoggrannhet på 71,70%.

In this thesis we examine the question whether it is possible to model approach and avoidance behaviour with probabilistic machine learning. The results from this project will primarily aid in our collective understanding of human existence. Secondly, it will extend the knowledge with regards to probabilistic machine learning in the Neuroscience domain. We aid this through building a Variational Recurrent Neural Network (VRNN) that is trained on Electroencephalography (EEG)-data from participants that is subjected to odours with varying pleasantness. The pleasantness of the odours is used to divide the participants into two classes based on their self reported experience. This data is used to train the VRNN. The performance of the VRNN is evaluated by how well we are able to reconstruct the original data from a low dimensional latent representation. In this task the model performs on a similar level as related works. We further investigate how changes in the latent space effects reconstructed data. Despite being disentangled, the latent variables are hard to interpret. Furthermore we try to classify and cluster the latent space as either approach or avoidance behaviour with a Support Vector Machine and Uniform Manifold Approximation. The classification results are only slightly better than random, indicating that the learned latent space is not suitable for the task This is most likely due to the patterns that make up approach and avoidance behaviour is seen as noise by the VRNN. This leads to the patterns not being accurately modelled. This is shown by the evidence that frontal α -asymmetry that exists in the data is not reconstructed by the model. The conclusion is therefore that a VRNN is less suitable for modelling underlying behaviour from raw EEG data due to the low signal to noise ratio. We instead suggests to focus on specific frequency ranges in specific regions when applying machine learning in this domain. / Den här uppsatsen behandlar frågan huruvida det är möjligt att modellera närmande och frånstötande beteendemönster med hjälp av maskininlärning. Resultaten från detta projekt ämnar huvudsakligen att främja vidare förståelse av den mänskliga existensen. Vidare ämnar den även att utvidga förståelsen av hur probabilistisk maskininlärning kan användas för att utforska dylika hänseenden. Vi genomför detta genom att bygga en Variational Recurrent Neural Network-modell (VRNN) som tränas på data från experiment där personer utsätts för olika lukter samtidigt som deras Elektroencefalografi (EEG) spelas in. Deltagarna delas in i två klasser beroende på deras självrapporterade upplevelse av luktens njutbarhet. Maskininlärningsmodellen utvärderas genom att vi analyserar hur väl den lyckas rekonstruera datan. Detta lyckas den väl med. Vidare så undersöker vi hur förändringar i modellens latenta rum påverkar rekonstrueringen av datan. Resultaten från det experimentet är ej tydliga. Vidare så försöker vi klassificera och klustra det latenta rummet med avseende på närmande och frånstötande beteende med hjälp av en Support Vector Machine och Uniform Manifold Approximation. Resultaten från dessa experiment är att vi inte lyckas klassificera eller klustra det latenta rummet med avseende på närmande och frånstötande beteende bättre än slumpen. Vi argumenterar för att detta beror på att de underliggande mönster som skapar dessa beteenden ses som brus av VRNN-modellen och därmed inte modelleras. Detta visas genom att frontal α-asymmetri som existerar i datan ej rekonstrueras av modellen. Slutsaten blir därmed att en VRNN är mindre passande att använda vid modellering av underliggande beteenden av obehandlad EEG data. Detta på grund av det låga signal till brus-förhållandet i EEG-datan. Vi föreslår att istället fokusera på specifika frekvensområden i specifika hjärnregioner när maskininlärning appliceras på EEG.

Under recent years the gambling industry has been moving towards providing their customer the possibility to gamble online instead of visiting a physical location. Aggressive marketing, fast growth and a multitude of actors within the market have resulted in a spike of customers who have developed a gambling problem. Decision makers are trying to fight back by regulating markets in order to make the companies take responsibility and work towards preventing these problems. One method of working proactively in this regards is to identify vulnerable customers before they develop a destructive habit. In this work a novel method of predicting customers that have a higher risk in regards to gambling-related problems is explored. More concretely, a recurrent neural network with long short-term memory cells is created to process raw behaviour data that are aggregated on a daily basis to classify them as high-risk or not. Supervised training is used in order to learn from historical data, where the usage of permanent self-exclusions due to gambling related problems defines problem gamblers. The work consists of: obtain a local optimal configuration of the network which enhances the performance for identifying problem gam- blers who favour the casino section over sports section, and analyze the model to provide insights in the field. This project was carried out together with LeoVegas Mobile Gaming Group. The group offers both online casino games and sports booking in a number of countries in Europe. This collaboration made both data and expertise within the industry accessible to perform this work. The company currently have a model in production to perform these predictions, but want to explore other approaches. The model that has been developed showed a significant increase in performance compared to the one that is currently used at the company. Specifically, the precision and recall which are two metrics important for a two class classification model, increased by 37% and 21% respectively. Using raw time series data, instead of aggregated data increased the responsiveness regarding customers change in behaviour over time. The model also scaled better with more history compared to the current model, which could be a result of the nature of a recurrent network compared to the current model used.