Every day, an overwhelming volume of new information is produced. Information Retrieval systems play a crucial role in managing this deluge and facilitate users to find relevant information. Simultaneously, the volume of scientific literature is also rapidly increasing, requiring powerful retrieval tools in this domain. Current methods of Information Retrieval employ language models based on the Transformer architecture, called Transformer-Encoder models, in this work. These models are generally trained in two phases: initially, a pre-training on general natural language data is performed, then fine-tuning follows, which adapts the model to a specific task such as classification or retrieval. Since Transformer-Encoder models are pre-trained on general natural language corpora, they perform well on these documents. However, scientific documents exhibit different features. The language in these documents is characterized by mathematical notation, such as formulae. Applying Transformer-Encoder models to these documents results in a low retrieval performance (effectiveness). A possible solution is to adapt the model to the new domain by further pre-training on a data set originating in the new domain. This process is called Domain-Adaptive Pre-Training and has been successfully applied to other domains. Mathematical Answer Retrieval involves finding relevant answers from a large corpus for mathematical questions. Both the question and the answers can contain mathematical notation and natural language. To retrieve relevant answers, the model must 'understand' the problem specified in the question and the solution of the answers. This property makes the task of Mathematical Answer Retrieval well suited to evaluate whether Transformer-Encoder models can model mathematical and natural language in conjunction. Transformer-Encoder models showed a low performance on this task compared to traditional retrieval approaches, which is surprising given the success of Transformer-Encoder models in other domains. This thesis, therefore, deals with the domain-adaption of Transformer-Encoder models for the domain of mathematical documents and the development of a retrieval approach using these models for Mathematical Answer Retrieval. We start by presenting a retrieval pipeline using the Cross-Encoder setup, a specific architecture of applying Transformer-Encoder models for retrieval. Then, we enhance the retrieval pipeline by adapting the pre-training schema of the Transformer-Encoder models to capture mathematical language better. Our evaluation demonstrates the strengths of the Cross-Encoder setup using our domain-adapted Transformer-Encoder models. In addition to these contributions, we also present an analysis framework to evaluate what knowledge of mathematics the models have learned. This analysis framework is used to study Transformer-Encoder models before and after fine-tuning for mathematical retrieval. We show that Transformer-Encoder models learn structural features of mathematical formulae during pre-training but rely more on other superficial information for Mathematical Answer Retrieval. These analyses also enable us to improve our fine-tuning setup further. In conclusion, our findings suggest that Transformer-Encoder models can be applied as a suitable and powerful approach for Mathematical Answer Retrieval.

Green light optimal speed advisory (GLOSA) systems require reliable estimates of signal switching times to improve vehicle energy/fuel efficiency. Deployment of successful infrastructure to vehicle communication requires Signal Phase and Timing (SPaT) messages to be populated with most likely estimates of switching times and confidence levels in these estimates. Obtaining these estimates is difficult for actuated signals where the length of each green indication changes to accommodate varying traffic conditions and pedestrian requests. This dissertation explores the different ways in which predictions can be made for the most likely switching times. Data are gathered from six intersections along the Gallows Road corridor in Northern Virginia. The application of long-short term memory neural networks for obtaining predictions is explored for one of the intersections. Different loss functions are tried for the purpose of prediction and a new loss function is devised. Mean absolute percentage error is found to be the best loss function in the short-term predictions. Mean squared error is the best for long-term predictions and the proposed loss function balances both well. The amount of historical data needed to make a single accurate prediction is assessed. The assessment concludes that the short-term prediction is accurate with only a 3 to 10 second time window in the past as long as the training dataset is large enough. Long term prediction, however, is better with a larger past time window. The robustness of LSTM models to different demand levels is then assessed utilizing the unique scenario created by the COVID-19 pandemic stay-at-home order. The study shows that the models are robust to the changing demands and while regularization does not really affect their robustness, L1 and L2 regularization can improve the overall prediction performance. An ensemble approach is used considering the use of transformers for SPaT prediction for the first time across the six intersections. Transformers are shown to outperform other models including LSTM. The ensemble provides a valuable metric to show the certainty level in each of the predictions through the level of consensus of the models. Finally, a hybrid approach integrating deep learning and controller logic is proposed by predicting actuations separately and using a digital twin to replicate SPaT information. The approach is proven to be the best approach with 58% less mean absolute error than other approaches. Overall, this dissertation provides a holistic methodology for predicting SPaT and the certainty level associated with it tailored to the existing technology and communication needs. / Doctor of Philosophy / Automated and connected vehicles waste a lot of fuel and energy to stop and go at traffic signals. The ideal case is for them to be able to know when the traffic signal turns green ahead of time and plan to reach the intersection by the time it is green, so they do not have to stop. Not having to stop can save up to 40 percent of the gas used at the intersection. This is a difficult task because the green time is not fixed. It has a minimum and maximum setting, and it keeps extending the green every time a new vehicle arrives. While this is good for adapting to traffic, it makes it difficult to know exactly when the traffic signal turns green to reach the intersection at that time. In this dissertation, different models to know ahead of time when the traffic signal will change are used. A model is chosen known as long-short term memory neural network (LSTM), which is a way to recognize how the traffic signal is expected to behave in the future from its past behavior. The point is to reduce the errors in the predictions. The first thing is to look at the loss function, which is how the model deals with error. It is found that the best thing is to take the average of the absolute value of the error as a percentage of the prediction if the prediction is that traffic signal will change soon. If it is a longer time until the traffic signal changes, the best way is to take the average of the square of the error. Finally, another function is introduced to balance between both. The second thing explored is how far back in time data was needed to be given to the model to predict accurately. For predictions of less than 20 seconds in the future, only 3 to 10 seconds in the past are needed. For predictions further in the future, looking further back can be useful. The third thing explored was how these models would do after rare events like COVID-19 pandemic. It was found that even though much fewer cars were passing through the intersections, the models still had low errors. Techniques were used to reduce the model reliance on specific data known as regularization techniques. This did not help the models to do better after COVID, but two techniques known as L1 and L2 regularization improved overall performance. The study was then expanded to include 6 intersections and used three additional models in addition to LSTM. One of these models, known as transformers, has never been used before for this problem and was shown to make better predictions than other models. The consensus between the models, which is how many of the models agree on the prediction, was used as a measure for certainty in the prediction. It was proven to be a good indicator. An approach is then introduced that combines the knowledge of the traffic signal controller logic with the powerful predictions of machine learning models. This is done by making a computer program that replicates the logic of the traffic signal controller known as a digital twin. Machine learning models are then used to predict vehicle arrivals. The program is then run using the predicted arrivals to provide a replication of the signal timing. This approach is found to be the best approach with 58 percent less error than the other approaches. Overall, this dissertation provides an end-to-end solution that uses real data generated from intersections to predict the time to green and estimate the certainty in prediction that can help automated and connected vehicles be more fuel efficient.

Maintaining the quality of indoor environments in educational facilities is crucial for student comfort, health, well-being, and learning performance. Amidst the growing recognition of the impact of indoor environmental conditions on occupant comfort, health, and well-being, there has been an increasing focus on the assessment and modeling of Indoor Environmental Quality (IEQ). Despite considerable advancements, current IEQ modeling and assessment methodologies often prioritize and limit to singular comfort metrics, potentially neglect- ing the comprehensive and holistic factors associated with occupant comfort and health. Furthermore, existing indoor environment maintenance practices and building systems for educational facilities often fail to include feedback from occupants (e.g., students and fac- ulty) and exhibit limited adaptability to their needs. This calls for more inclusive and occupant-centric IEQ assessment models that cover a broader spectrum of environmental parameters and occupant needs. To address the gaps, this thesis proposes a novel Artificial Intelligence of Things (AIoT)-based IEQ assessment framework that bridges gaps by uti- lizing multimodal data fusion and deep learning-based prediction and classification models. These models are developed to utilize real-time multidimensional IEQ data, non-intrusive occupant feedback (MFCC features from audio recordings, video/thermal features extracted by Vision Transformer (ViT)), and self-reported comfort and health levels, placing a focus on occupant-centric and data-driven decision-making for intelligent educational facilities. The proposed framework was evaluated and validated at Virginia Tech Blacksburg campus, achieving a 91.9% in R2 score in predicting future IEQ conditions and 97% and 96% accuracy in comfort and health-based IEQ conditions classifications. / Master of Science

Microinverter has become a new trend for photovoltaic (PV) grid-tie systems due to its advantages which include greater energy harvest, simplified system installation, enhanced safety, and flexible expansion. Since an individual microinverter system is typically attached to the back of a PV module, it is desirable that it has a long lifespan that can match PV modules, which routinely warrant 25 years of operation. In order to increase the life expectancy and improve the long-term reliability, electrolytic capacitors must be avoided in microinverters because they have been identified as an unreliable component. One solution to avoid electrolytic capacitors in microinverters is using a two-stage architecture, where the high voltage direct current (DC) bus can work as a double line ripple buffer. For two-stage electrolyte-free microinverters, a high boost ratio dc-dc converter is required to increase the low PV module voltage to a high DC bus voltage required to run the inverter at the second stage. New high boost ratio dc-dc converter topologies using the hybrid transformer concept are presented in this dissertation. The proposed converters have improved magnetic and device utilization. Combine these features with the converter's reduced switching losses which results in a low cost, simple structure system with high efficiency. Using the California Energy Commission (CEC) efficiency standards a 250 W prototype was tested achieving an overall system efficiency of 97.3%. The power inversion stage of electrolyte-free microinverters requires a high efficiency grid-tie inverter. A transformerless inverter topology with low electro-magnetic interference (EMI) and leakage current is presented. It has the ability to use modern superjunction MOSFETs in conjunction with zero-reverse-recovery silicon carbide (SiC) diodes to achieve ultrahigh efficiency. The performance of the topology was experimentally verified with a tested CEC efficiency of 98.6%. Due to the relatively low energy density of film capacitors compared to electrolytic counterparts, less capacitance is used on the DC bus in order to lower the cost and reduce the volume of electrolyte-free microinverters. The reduced capacitance leads to high double line ripple voltage oscillation on DC bus. If the double line oscillation propagates back into the PV module, the maximum power point tracking (MPPT) performance would be compromised. A control method which prevents the double line oscillation from going to the PV modules, thus improving the MPPT performance was proposed. Finally, a control technique using a single microcontroller with low sampling frequency was presented to effectively eliminate electrolyte capacitors in two-stage microinverters without any added penalties. The effectiveness of this control technique was validated both by simulation and experimental results. / Ph. D.

With deregulation, diagnosis and maintenance of power equipment, especially power transformers, become increasingly important to keep power systems in reliable operation. This dissertation systematically studied two kinds of transformer failure and abnormality cases, and then developed a new Internet based Virtual Hospital (VH) for power equipment to help power equipment diagnosis and maintenance. A practical case of generator-step-up (GSU) transformer failures in a pumped storage plant was extensively studied. Abnormal electrical phenomena associated with GSU transformers, including switching transients and very fast transients (VFT), and lightning, were analyzed. Simulation showed that circuit breaker restriking could be a major cause of transformer successive failures, and current surge arrester configuration did not provide enough lightning protection to GSU transformers. Mitigation of abnormal electrical phenomena effects on GSU transformers was proposed and discussed. The study can be a complete reference of troubleshooting of other similar transformer failures. Geomagnetically induced current (GIC) is another possible cause of transformer abnormality. A simplified method based on the equivalent magnetizing curve for transformers with different core design was developed and validated to estimate harmonic currents and MVar drawn by power transformers with a given GIC. An effective indicator was proposed using partial harmonic distortion, PHD, to show when the transformer begins saturating with the input GIC. The developed method has been applied to a real time GIC monitoring system last year for a large power network with thousands of transformers. A new Internet based Virtual Hospital (VH) for Power Equipment was conceptually developed to share experience of power equipment diagnosis and maintenance, and update the existing diagnostic techniques and maintenance strategies, and a comprehensive information model was developed for data organization, access, and archiving related to equipment diagnosis and maintenance. An Internet based interactive fault diagnostic tool has been launched for power transformers based on dissolved gas analysis (DGA). The above results and findings can help improving power equipment diagnosis and utility maintenance strategies. / Ph. D.

With the rapid increase of cloud computing and the high demand for digital content, it is estimated that the power consumption of the IT industry will reach 10 % of the total electric power in the USA by 2020. Multi-core processors (CPUs) and graphics processing units (GPUs) are the key elements in fulfilling all of the digital content requirements, but come with a price of more power-hungry processors, driving the power per server rack to 20 KW levels. The need for more efficient power management solutions on the architecture level, down to the converter level, is inevitable. Recently, data centers have replaced the 12V DC server rack distribution with a 48V DC distribution, producing a significant overall system efficiency improvement. However, 48V rack architecture raises significant challenges for the voltage regulator modules (VRMs) required for powering the processor. The 48V VRM in the vicinity of the CPU needs to be designed with very high efficiency, high power density, high light-load efficiency, as well as meet all transient requirements by the CPU and GPU. Transferring the well-developed multi-phase buck converter used in the 12V VRM to the 48V distribution platform is not that simple. The buck converter operating with 48V, stepping down to sub 2V, will be subjected to significant switching related loss, resulting in lower overall system efficiency. These challenges drive the need to look for more efficient architectures for 48V VRM solutions. Two-stage conversions can help solve the design challenges for 48V VRMs. A first-stage unregulated converter is used to step-down the 48V to a specific intermediate bus voltage. This voltage will feed a multi-phase buck converter that powers the CPU. An unregulated LLC converter is used for the first-stage converter, with zero voltage switching (ZVS) operation for the primary side switches, and zero current switching (ZCS) along with ZVS operation, for the secondary side synchronous rectifiers (SRs). The LLC converter can operate at high frequency, in order to reduce the magnetic components size, while achieving high-efficiency. The high-efficiency first-stage, along with the scalability and high bandwidth control of the second-stage, allows this architecture to achieve high-efficiency and power density. This architecture is simpler to adopt by industry, by plugging the unregulated converter before the existing multi-phase buck converters on today's platforms. The first challenge for this architecture is the transformer design of the first-stage LLC converter. It must avoid all of the loss associated with high frequency operations, and still achieve high power density without scarifying efficiency. In this thesis, the integrated matrix transformer structure is optimized by SR integration with windings, interleaved primary side termination, and a better PCB winding arrangement to achieve high-efficiency and power density, and minimize the losses associated with high-frequency operations. The second challenge is the light load efficiency improvement. In this thesis a light load efficiency improvement is proposed by a dynamic change of the intermediate bus voltage, resulting in more than 8 % light load efficiency improvements. The third challenge is the selection of the optimal bus voltage for the two-stage architecture. The impact of different bus voltages was analyzed in order to maximize the overall conversion efficiency. Multiple 48V unregulated converters were designed with maximum efficiency >98 %, and power densities >1000 W/in3, with different output voltages, to select the optimal bus voltage for the two-stage VRM. Although the two-stage VRM is more scalable and simpler to design and adopt by current industry, the efficiency will reduce as full power flows in two cascaded DC/DC converters. Single-stage conversion can achieve higher-efficiency and power-density. In this thesis, a quasi-parallel Sigma converter is proposed for the 48V VRM application. In this structure, the power is shared between two converters, resulting in higher conversion efficiency. With the aid of an optimized integrated magnetic design, a Sigma converter suitable for narrow voltage range applications was designed with 420 W/in3 and a maximum efficiency of 94 %. Later, another Sigma converter suitable for wide voltage range applications was designed with 700W/in3 and a maximum efficiency of 95 %. Both designs can achieve higher efficiency than the two-stage VRM and all other state-of-art solutions. The challenges associated with the Sigma converter, such as startup and closed loop control were addressed, in order to make it a viable solution for the VRM application. The 48V rack architecture requires regulated 12V output converters for various loads. In this thesis, a regulated LLC is used to design a high-efficiency and power-density 48V bus converter. A novel integration method of the inductor and transformer helps the LLC achieve the required regulation capability with minimum losses, resulting in a converter that can provide 1KW of continuous power with efficiency of 97.8 % and 700 W/in3 power density. This dissertation discusses new power architectures with an optimized design for the 48V rack architectures. With the academic contributions in this dissertation, different conversion architectures can be utilized for 48V VRM solutions that solve all of the challenges associated with it, such as scalability, high-efficiency, high density, and high BW control. / Doctor of Philosophy / With the rapid increase of cloud computing and the high demand for digital content, it is estimated that the power consumption of the IT industry will reach 10 % of the total electric power in the USA by 2020. Multi-core processors (CPUs) and graphics processing units (GPUs) are the key elements in fulfilling all of the digital content requirements but come with a price of more power-hungry processors, driving the power per server rack to 20 KW levels. The need for more efficient power management solutions on the architecture level, down to the converter level, is inevitable. The data center manufacturers have recently adopted a more efficient architecture that supplies a 48V DC server rack distribution instead of a 12V DC distribution to the server motherboard. This helped reduce costs and losses, but as a consequence, raised a challenge in the design of the DC/DC voltage regulator modules (VRM) supplied by the 48V, in order to power the CPU and GPU. In this work, different architectures will be explored for the 48V VRM, and the trade-off between them will be evaluated. The main target is to design the VRM with very high-efficiency and high-power density to reduce the cost and size of the CPU/GPU motherboards. First, a two-stage power conversion structure will be used. The benefit of this structure is that it relies on existing technology using the 12V VRM for powering the CPU. The only modification required is the addition of another converter to step the 48V to the 12V level. This architecture can be easily adopted by industry, with only small modifications required on the system design level. Secondly, a single-stage power conversion structure is proposed that achieves higher efficiency and power density compared to the two-stage approach; however, the structure is very challenging to design and to meet all requirements by the CPU/GPU applications. All of these challenges will be addressed and solved in this work. The proposed architectures will be designed using an optimized magnetic structure. These structures achieve very high efficiency and power density in their designed architectures, compared to state-of-art solutions. In addition, they can be easily manufactured using automated manufacturing processes.

The competitive market of wireless communication devices demands low power and low cost RF solutions. A quadrature local oscillator (LO) is an essential building block for most transceivers. As the CMOS technology scales deeper into the nanometer regime, design of a low-power low-voltage quadrature LO still poses a challenge for RF designers. This dissertation investigates a new quadrature LO topology featuring a transformer-based voltage controlled oscillator (VCO) stacked with a divide-by-two for low-power low-voltage wireless applications. The transformer-based VCO core adopts the Armstrong VCO configuration to mitigate the small voltage headroom and the noise coupling. The LO operating conditions, including the start-up condition, the oscillation frequency, the voltage swing and the current consumption are derived based upon a linearized small-signal model. Both linear time-invariant (LTI) and linear time-variant (LTV) models are utilized to analyze the phase noise of the proposed LO. The results indicate that the quality factor of the primary coil and the mutual inductance between the primary and the secondary coils play an important role in the trade-off between power and noise. The guidelines for determining the parameters of a transformer are developed. The proposed LO was fabricated in 65 nm CMOS technology and its die size is about 0.28 mm2. The measurement results show that the LO can work at 1 V supply voltage, and its operation is robust to process and temperature variations. In high linearity mode, the LO consumes about 2.6 mW of power typically, and the measured phase noise is -140.3 dBc/Hz at 10 MHz offset frequency. The LO frequency is tunable from 1.35 GHz to 1.75 GHz through a combination of a varactor and an 8-bit switched capacitor bank. The proposed LO compares favorably to the existing reported LOs in terms of the figure of merit (FoM). More importantly, high start-up gain, low power consumption and low voltage operation are achieved simultaneously in the proposed topology. However, it also leads to higher design complexity. The contributions of this work can be summarized as 1) proposal of a new quadrature LO topology that is suitable for low-power low-voltage wireless applications, 2) an in-depth circuit analysis as well as design method development, 3) implementation of a fully integrated LO in 65 nm CMOS technology for GPS applications, 4) demonstration of high performance for the design through measurement results. The possible future improvements include the transformer optimization and the method of circuit analysis. / Ph. D.

High voltage power transformer is one of the most important and expensive components in today's power transmission and distribution systems. Any overlooked critical fault generated inside a power transformer may lead to a transformer catastrophic failure which could not only cause a disruption to the power system but also significant equipment damage. Accurate and prompt information on the health state of a transformer is thus the critical prerequisite for an asset manager to make a vital decision on a transformer with suspicious conditions. Partial discharge (PD) is not only a precursor of insulation degradation, but also a primary factor to accelerate the deterioration of the insulation system in a transformer. Monitoring of PD activities and the concentration of PD generated combustible gases dissolved in the transformer oil has been proven to be an effective procedure for transformer health state estimation. However current commercially available sensors can only be installed outside of transformers and offer indirect or delayed information. This research is aimed to investigate and develop several sensor techniques for transformer health monitoring. The first work is an optical fiber extrinsic Fabry-Perot interferometric sensor for PD detection. By filling SF6 into the sensor air cavity of the extrinsic Fabry-Perot interferometer sensor, the last potential obstacle that prevents this kind of sensors from being installed inside transformers has been removed. The proposed acoustic sensor multiplexing system is stable and more economical than the other sensor multiplexing methods that usually require the use of a tunable laser or filters. Two dissolved gas analysis (DGA) methods for dissolved hydrogen or acetylene measurement are also proposed and demonstrated. The dissolved hydrogen detection is based on hydrogen induced fiber loss and the dissolved acetylene detection is by direct oil transmission measurement. / Ph. D.

The power industry relies on high voltage transformers as the backbone of power distribution networks. High voltage transformers are designed to handle immense electrical loads in hostile environments. Long term placement is desired, however by being under constant heavy load transformers face mechanical, thermal, and electrical stresses which lead to failures of the protection systems in place. The service life of a transformer is often limited by the life time of its insulation system. Insulation failures most often develop from thermal faults, or hotspots, and electrical faults, or partial discharges. Detecting hotspots and partial discharges to predict transformer life times is imperative and much research is focused towards these topics. As these protection systems fail they often generate gas or acoustic signals signifying a problem. Research has already been performed discovering new ways integrate optical fiber sensors into high voltage transformers. This thesis is a continuation of that research by attempting to improve sensor sensitivity for hydrogen and acetylene gasses. Of note is the fabrication of new hydrogen sensing fiber for operation around a larger absorption peak and also the improvement of the acetylene sensor's light source stability. Also detailed is the manufacturing of a field testable prototype and the non-sensitivity testing of several other gasses. The developed sensors are capable but still could be improved with the use of more powerful and stable light sources. / Master of Science

This thesis presents an artificial neural network (ANN) approach to diagnose and detect faults in oil-filled power transformers based on dissolved gas-in-oil analysis. The goal of the research is to investigate the available transformer incipient fault diagnosis methods and then develop an ANN approach for this purpose. This ANN classifier should not only be able to detect the fault type, but also should be able to judge the cellulosic material breakdown. This classifier should also be able to accommodate more than one type of fault. This thesis describes a two-step ANN method that is used to detect faults with or without cellulose involved. Utilizing a feedforward artificial neural network, the classifier was trained with back-propagation, using training samples collected from different failed transformers. It is shown in the thesis that such a neural-net based approach can yield a high diagnosis accuracy. Several possible design alternatives and comparisons are also addressed in the thesis. The final system has been successfully tested, exhibiting a classification accuracy of 95% for major fault type and 90% for cellulose breakdown. / Master of Science