Transformer neural networks have made a significant impact on natural language processing. The Transformer network self-attention mechanism effectively addresses the vanishing gradient problem that limits a network learning capability, especially when the time series gets longer or the size of the network gets deeper. This dissertation examines the usage of the Transformer model for time-series forecasting and customizes it for a simultaneous multistep-ahead prediction model in a surrogate model predictive control (MPC) application. The proposed method demonstrates enhanced control performance and computation efficiency compared to the Long-short term memory (LSTM)-based MPC and one-step-ahead prediction model structures for both LSTM and Transformer networks. In addition to the Transformer, this research investigates hybrid machine-learning modeling. The machine learning models are known for superior function approximation capability with sufficient data. However, the quantity and quality of data to ensure the prediction precision are usually not readily available. The physics-informed neural network (PINN) is a type of hybrid modeling method using dynamic physics-based equations in training a standard machine learning model as a form of multi-objective optimization. The PINN approach with the state-of-the-art time-series neural networks Transformer is studied in this research providing the standard procedure to develop the Physics-Informed Transformer (PIT) and validating with various case studies. This research also investigates the benefit of nonlinear model-based control and estimation algorithms for managed pressure drilling (MPD). This work presents a new real-time high-fidelity flow model (RT-HFM) for bottom-hole pressure (BHP) regulation in MPD operations. Lastly, this paper presents details of an Arduino microcontroller temperature control lab as a benchmark for modeling and control methods. Standard benchmarks are essential for comparing competing models and control methods, especially when a new method is proposed. A physical benchmark considers real process characteristics such as the requirement to meet a cycle time, discrete sampling intervals, communication overhead with the process, and model mismatch. Novel contributions of this work are (1) a new MPC system built upon a Transformer time-series architecture, (2) a training method for time-series machine learning models that enables multistep-ahead prediction, (3) verification of Transformer MPC solution time performance improvement (15 times) over LSTM networks, (4) physics-informed machine learning to improve extrapolation potential, and (5) two case studies that demonstrate hybrid modeling and benchmark performance criteria.

<p dir="ltr">Several incidents report misuse of synthetic speech for impersonation attacks, spreading misinformation, and supporting financial frauds. To counter such misuse, this dissertation focuses on developing methods for speech forensics. First, we present a method to detect compressed synthetic speech. The method uses comparatively 33 times less information from compressed bit stream than used by existing methods and achieve high performance. Second, we present a transformer neural network method that uses 2D spectral representation of speech signals to detect synthetic speech. The method shows high performance on detecting both compressed and uncompressed synthetic speech. Third, we present a method using an interpretable machine learning approach known as disentangled representation learning for synthetic speech detection. Fourth, we present a method for synthetic speech attribution. It identifies the source of a speech signal. If the speech is spoken by a human, we classify it as authentic/bona fide. If the speech signal is synthetic, we identify the generation method used to create it. We examine both closed-set and open-set attribution scenarios. In a closed-set scenario, we evaluate our approach only on the speech generation methods present in the training set. In an open-set scenario, we also evaluate on methods which are not present in the training set. Fifth, we propose a multi-domain method for synthetic speech localization. It processes multi-domain features obtained from a transformer using a ResNet-style MLP. We show that with relatively less number of parameters, the proposed method performs better than existing methods. Finally, we present a new direction of research in speech forensics <i>i.e.</i>, bias and fairness of synthetic speech detectors. By bias, we refer to an action in which a detector unfairly targets a specific demographic group of individuals and falsely labels their bona fide speech as synthetic. We show that existing synthetic speech detectors are gender, age and accent biased. They also have bias against bona fide speech from people with speech impairments such as stuttering. We propose a set of augmentations that simulate stuttering in speech. We show that synthetic speech detectors trained with proposed augmentation have less bias relative to detector trained without it.</p>