Generative models are broadly used in many subfields of DL. DNNs have recently developed a core approach to solving data-centric problems in image classification, translation, etc. The latest developments in parameterizing these models using DNNs and stochastic optimization algorithms have allowed scalable modeling of complex, high-dimensional data, including speech, text, and image. This dissertation proposal presents our state-the-art probabilistic bases and DL algorithms for generative models, including VAEs, GANs, and RNN-based encoder-decoder. The proposal also discusses application areas that may benefit from deep generative models in both NLP and computer vision. In NLP, we proposed an Arabic poetry generation model with extended phonetic and semantic embeddings (Phonetic CNN_subword embeddings). Extensive quantitative experiments using BLEU scores and Hamming distance show notable enhancements over strong baselines. Additionally, a comprehensive human evaluation confirms that the poems generated by our model outperform the base models in criteria including meaning, coherence, fluency, and poeticness. We proposed a generative video model using a hybrid VAE-GAN model in computer vision. Besides, we integrate two attentional mechanisms with GAN to get the essential regions of interest in a video, focused on enhancing the visual implementation of the human motion in the generated output. We have considered quantitative and qualitative experiments, including comparisons with other state-of-the-arts for evaluation. Our results indicate that our model enhances performance compared with other models and performs favorably under different quantitive metrics PSNR, SSIM, LPIPS, and FVD.Recently, mimicking biologically inspired learning in generative models based on SNNs has been shown their effectiveness in different applications. SNNs are the third generation of neural networks, in which neurons communicate through binary signals known as spikes. Since SNNs are more energy-efficient than DNNs. Moreover, DNN models have been vulnerable to small adversarial perturbations that cause misclassification of legitimate images. This dissertation shows the proposed ``VAE-Sleep'' that combines ideas from VAE and the sleep mechanism leveraging the advantages of deep and spiking neural networks (DNN--SNN).On top of that, we present ``Defense–VAE–Sleep'' that extended work of ``VAE-Sleep'' model used to purge adversarial perturbations from contaminated images. We demonstrate the benefit of sleep in improving the generalization performance of the traditional VAE when the testing data differ in specific ways even by a small amount from the training data. We conduct extensive experiments, including comparisons with the state–of–the–art on different datasets.

The stock market is a non-linear field, but many of the best-known portfolio optimization algorithms are based on linear models. In recent years, the rapid development of machine learning has produced flexible models capable of complex pattern recognition. In this paper, we propose two different methods of portfolio optimization; one based on the development of a multivariate time-dependent neural network,thelongshort-termmemory(LSTM),capable of finding lon gshort-term price trends. The other is the linear Markowitz model, where we add an exponential moving average to the input price data to capture underlying trends. The input data to our neural network are daily prices, volumes and market indicators such as the volatility index (VIX).The output variables are the prices predicted for each asset the following day, which are then further processed to produce metrics such as expected returns, volatilities and prediction error to design a portfolio allocation that optimizes a custom utility function like the Sharpe Ratio. The LSTM model produced a portfolio with a return and risk that was close to the actual market conditions for the date in question, but with a high error value, indicating that our LSTM model is insufficient as a sole forecasting tool. However,the ability to predict upward and downward trends was somewhat better than expected and therefore we conclude that multiple neural network can be used as indicators, each responsible for some specific aspect of what is to be analysed, to draw a conclusion from the result. The findings also suggest that the input data should be more thoroughly considered, as the prediction accuracy is enhanced by the choice of variables and the external information used for training. / Aktiemarknaden är en icke-linjär marknad, men många av de mest kända portföljoptimerings algoritmerna är baserad på linjära modeller. Under de senaste åren har den snabba utvecklingen inom maskininlärning skapat flexibla modeller som kan extrahera information ur komplexa mönster. I det här examensarbetet föreslår vi två sätt att optimera en portfölj, ett där ett neuralt nätverk utvecklas med avseende på multivariata tidsserier och ett annat där vi använder den linjära Markowitz modellen, där vi även lägger ett exponentiellt rörligt medelvärde på prisdatan. Ingångsdatan till vårt neurala nätverk är de dagliga slutpriserna, volymerna och marknadsindikatorer som t.ex. volatilitetsindexet VIX. Utgångsvariablerna kommer vara de predikterade priserna för nästa dag, som sedan bearbetas ytterligare för att producera mätvärden såsom förväntad avkastning, volatilitet och Sharpe ratio. LSTM-modellen producerar en portfölj med avkastning och risk som ligger närmre de verkliga marknadsförhållandena, men däremot gav resultatet ett högt felvärde och det visar att vår LSTM-modell är otillräckligt för att använda som ensamt predikteringssverktyg. Med det sagt så gav det ändå en bättre prediktion när det gäller trender än vad vi antog den skulle göra. Vår slutsats är därför att man bör använda flera neurala nätverk som indikatorer, där var och en är ansvarig för någon specifikt aspekt man vill analysera, och baserat på dessa dra en slutsats. Vårt resultat tyder också på att inmatningsdatan bör övervägas mera noggrant, eftersom predikteringsnoggrannheten.

The present work focuses on the two areas of investigation: system identification of helicopter and design of controller for the helicopter. Helicopter system identification, the first subject of investigation in this thesis, can be described as the extraction of system characteristics/dynamics from measured flight test data. Wind tunnel experimental data suffers from scale effects and model deficiencies. The increasing need for accurate models for the design of high bandwidth control system for helicopters has initiated a renewed interest in and a more active use of system identification. Besides, system identification is likely to become mandatory in the future for model validation of ground based helicopter simulators. Such simulators require accurate models in order to be accepted by pilots and regulatory authorities like Federal Aviation Regulation for realistic complementary helicopter mission training. Two approaches are widely used for system identification, namely, black box and gray box approach. In the black-box approach, the relationship between input-output data is approximated using nonparametric methods such as neural networks and in such a case, internal details of the system and model structure may not be known. In the gray box approach, parameters are estimated after defining the model structure. In this thesis, both black box and gray box approaches are investigated. In the black box approach, in this thesis, a comparative study and analysis of different Recurrent Neural Networks(RNN) for the identification of helicopter dynamics using flight data is investigated. Three different RNN architectures namely, Nonlinear Auto Regressive eXogenous input(NARX) model, neural network with internal memory known as Memory Neuron Networks(MNN)and Recurrent MultiLayer perceptron (RMLP) networks are used to identify dynamics of the helicopter at various flight conditions. Based on the results, the practical utility, advantages and limitations of the three models are critically appraised and it is found that the NARX model is most suitable for the identification of helicopter dynamics. In the gray box approach, helicopter model parameters are estimated after defining the model structure. The identification process becomes more difficult as the number of degrees-of-freedom and model parameters increase. To avoid the drawbacks of conventional methods, neural network based techniques, called the delta method is investigated in this thesis. This method does not require initial estimates of the parameters and the parameters can be directly extracted from the flight data. The Radial Basis Function Network(RBFN)is used for the purpose of estimation of parameters. It is shown that RBFN is able to satisfactorily estimate stability and control derivatives using the delta method. The second area of investigation addressed in this thesis is the control of helicopter in flight. Helicopter requires use of a control system to achieve satisfactory flight. Designing a classical controller involves developing a nonlinear model of the helicopter and extracting linearized state space matrices from the nonlinear model at various flight conditions. After examining the stability characteristics of the helicopter, the desired response is obtained using a feedback control system. The scheduling of controller gains over the entire envelope is used to obtain the desired response. In the present work, a helicopter having a soft inplane four bladed hingeless main rotor and a four-bladed tail rotor with conventional mechanical controls is considered. For this helicopter, a mathematical model and also a model based on neural network (using flight data) has been developed. As a precursor, a feed back controller, the Stability Augmentation System(SAS), is designed using linear quadratic regulator control with full state feedback and LQR with out put feedback approaches. SAS is designed to meet the handling qualities specification known as Aeronautical Design Standard ADS-33E-PRF. The control gains have been tuned with respect to forward speed and gain scheduling has been arrived at. The SAS in the longitudinal axis meets the requirement of the Level1 handling quality specifications in hover and low speed as well as for forward speed flight conditions. The SAS in the lateral axis meets the requirement of the Level2 handling quality specifications in both hover and low speed as well as for forward speed flight conditions. Such conventional design of control has served useful purposes, however, it requires considerable flight testing which is time consuming, to demonstrate and tune these control law gains. In modern helicopters, the stringent requirements and non-linear maneuvers make the controller design further complicated. Hence, new design tools have to be explored to control such helicopters. Among the many approaches in adaptive control, neural networks present a potential alternative for modeling and control of nonlinear dynamical systems due to their approximating capabilities and inherent adaptive features. Furthermore, from a practical perspective, the massive parallelism and fast adaptability of neural network implementations provide more incentive for further investigation in problems involving dynamical systems with unknown non-linearity. Therefore, adaptive control approach based on neural networks is proposed in this thesis. A neural network based Feedback Error Neural adaptive Controller(FENC) is designed for a helicopter. The proposed controller scheme is based on feedback error learning strategy in which the outer loop neural controller enhances the inner loop conventional controller by compensating for unknown non-linearity and parameter un-certainties. Nonlinear Auto Regressive eXogenous input(NARX)neural network architecture is used to approximate the control law and the controller network parameters are adapted using updated rules Lyapunov synthesis. An offline (finite time interval)and on-line adaptation strategy is used to approximate system uncertainties. The results are validated using simulation studies on helicopter undergoing an agile maneuver. The study shows that the neuro-controller meets the requirements of ADS-33 handling quality specifications. Even though the tracking error is less in FENC scheme, the control effort required to follow the command is very high. To overcome these problems, a Direct Adaptive Neural Control(DANC)scheme to track the rate command signal is presented. The neural controller is designed to track rate command signal generated using the reference model. For the simulation study, a linearized helicopter model at different straight and level flight conditions is considered. A neural network with a linear filter architecture trained using back propagation through time is used to approximate the control law. The controller network parameters are adapted using updated rules Lyapunov synthesis. The off-line trained (for finite time interval)network provides the necessary stability and tracking performance. The on-line learning is used to adapt the network under varying flight conditions. The on-line learning ability is demonstrated through parameter uncertainties. The performance of the proposed direct adaptive neural controller is compared with feedback error learning neural controller. The performance of the controller has been validated at various flight conditions. The theoretical results are validated using simulation studies based on a nonlinear six degree-of-freedom helicopter undergoing an agile maneuver. Realistic gust and sensor noise are added to the system to study the disturbance rejection properties of the neural controllers. To investigate the on-line learning ability of the proposed neural controller, different fault scenarios representing large model error and control surface loss are considered. The performances of the proposed DANC scheme is compared with the FENC scheme. The study shows that the neuro-controller meets the requirements of ADS-33 handling quality specifications.

<p>    </p> <p>In order to advance next-generation communication systems, it is critical to enhance the state-of-the-art communication architectures, such as device-to-device (D2D), multiple- input multiple-output (MIMO), and intelligent reflecting surface (IRS), in terms of achieving high data rate, low latency, and high energy efficiency. In the first part of this dissertation, we address joint learning and optimization methodologies on cutting-edge network archi- tectures. First, we consider D2D networks equipped with MIMO systems. In particular, we address the problem of minimizing the network overhead in D2D networks, defined as the sum of time and energy required for processing tasks at devices, through the design for MIMO beamforming and communication/computation resource allocation. Second, we address IRS-assisted communication systems. Specifically, we study an adaptive IRS control scheme considering realistic IRS reflection behavior and channel environments, and propose a novel adaptive codebook-based limited feedback protocol and learning-based solutions for codebook updates. </p> <p><br></p> <p>Furthermore, in order for revolutionary innovations to emerge for future generations of communications, it is crucial to explore and address fundamental, long-standing open problems for communications, such as the design of practical codes for a variety of important channel models. In the later part of this dissertation, we study the design of practical codes for feedback-enabled communication channels, i.e., feedback codes. The existing feedback codes, which have been developed over the past six decades, have been demonstrated to be vulnerable to high forward/feedback noises, due to the non-triviality of the design of feedback codes. We propose a novel recurrent neural network (RNN) autoencoder-based architecture to mitigate the susceptibility to high channel noises by incorporating domain knowledge into the design of the deep learning architecture. Using this architecture, we suggest a new class of non-linear feedback codes that increase robustness to forward/feedback noise in additive White Gaussian noise (AWGN) channels with feedback. </p>

Histogram data are widely used for compressing high-frequency time-series signals due to their ability to capture distributional informa-tion. However, this compression comes at the cost of increased di-mensionality and loss of contextual details from the original features.This study addresses the challenge of effectively capturing changesin distributions over time and their contribution to failure prediction.Specifically, we focus on the task of predicting Time to Event (TTE) forturbocharger failures.In this thesis, we propose a novel approach to improve failure pre-diction by fine-tuning lower-dimensional representations of bi-variatehistograms. The goal is to optimize these representations in a waythat enhances their ability to predict component failure. Moreover, wecompare the performance of our learned representations with hand-crafted histogram features to assess the efficacy of both approaches.We evaluate the different representations using the Weibull Time ToEvent - Recurrent Neural Network (WTTE-RNN) framework, which isa popular choice for TTE prediction tasks. By conducting extensive ex-periments, we demonstrate that the fine-tuning approach yields supe-rior results compared to general lower-dimensional learned features.Notably, our approach achieves performance levels close to state-of-the-art results.This research contributes to the understanding of effective failureprediction from time series histogram data. The findings highlightthe significance of fine-tuning lower-dimensional representations forimproving predictive capabilities in real-world applications. The in-sights gained from this study can potentially impact various indus-tries, where failure prediction is crucial for proactive maintenanceand reliability enhancement.

As an efficient energy storage technology, lithium-ion batteries play a key role in the ongoing electrification of the mobility sector. However, the required modelbased design process, including hardware in the loop solutions, demands precise battery models. In this work, an encoder-decoder model framework based on recurrent neural networks is developed and trained directly on unstructured battery data to replace time consuming characterisation tests and thus simplify the modelling process. A manifold pseudo-random bit stream dataset is used for model training and validation. A mean percentage error (MAPE) of 0.30% for the test dataset attests the proposed encoder-decoder model excellent generalisation capabilities. Instead of the recursive one-step prediction prevalent in the literature, the stage-wise trained encoder-decoder framework can instantaneously predict the battery voltage response for 2000 time steps and proves to be 120 times more time-efficient on the test dataset. Accuracy, generalisation capability and time efficiency of the developed battery model enable a potential online anomaly detection, power or range prediction. The fact that, apart from the initial voltage level, the battery model only relies on the current load as input and thus requires no estimated variables such as the state-of-charge (SOC) to predict the voltage response holds the potential of a battery ageing independent LIB modelling based on raw BMS signals. The intrinsically ageingindependent battery model is thus suitable to be used as a digital battery twin in virtual experiments to estimate the unknown battery SOH on purely BMS data basis.