In recent years, metasurfaces have shown promising abilities to control and manipulate electromagnetic (EM) waves through modified surface boundary conditions. These surfaces are electrically thin and comprise an array of spatially varying sub-wavelength scattering elements (or meta-atoms). Metasurfaces can transform an incident EM wave into an arbitrarily tailored transmitted or reflected wavefront through carefully engineering each meta-atom. Recent developments in metasurfaces have opened exciting new opportunities in antenna design, sensing, and communications systems. In particular, reconfigurable metasurfaces - wherein meta-atoms are embedded with active components - lead to the development of low-cost, lightweight, and compact systems capable of producing programmable radiation patterns and jointly performing multi-function communications, and enable advanced sensors for next-generation platforms. This research introduces reconfigurable metasurfaces and their various applications in designing simplified communications systems, wherein the RF aperture and transceiver are integrated within the metasurface. Finally, we will present our recent work on reconfigurable metasurfaces control, metasurface-enabled direct signal modulation, and deep learning-based metasurface design. / Doctor of Philosophy / Metasurfaces are a promising new technology to enhance the capacity and coverage of wireless communication networks by dynamically reconfiguring the wireless propagation environment. These low-profile artificial electromagnetic surfaces, consisting of subwavelength resonant elements, are capable of tailoring electromagnetic waves controllably. In this dissertation, we control the transmission or reflection properties of the surface using digital codes by embedding tunable elements within each subwavelength element. Furthermore, metasurface antennas are a promising candidate for reducing the cost and hardware footprint of wireless sensor systems, such as radar or imaging. Using a digital microcontroller, we program the metasurface to steer the antenna beam in the direction of interest, modulate the radio wave, or change the polarization of an incoming signal. In addition to dynamic beamforming capabilities, we program the metasurface to reduce the scattering of an incoming signal, thereby reducing its perturbations on the radio environment. Still, the design of metasurfaces for specific applications remains complex and technically challenging. Lastly, we present innovative deep learning techniques to simplify metasurface design.

A protein is a biopolymer of amino acids that encodes a particular function. Given that there are 20 amino acids possible at each site, even a short protein of 100 amino acids has $20^{100}$ possible variants, making it unrealistic to evaluate all possible sequences in sequence level space. This search space could be reduced by considering the fact that billions of years of evolution exerting a constant pressure has left us with only a small subset of protein sequences that carry out particular cellular functions. The portion of amino acid space occupied by actual proteins found in nature is therefore much smaller than that which is possible cite{kauffman1993origins}. By examining related proteins that share a conserved function and common evolutionary history (heretofore referred to as protein families), it is possible to identify common motifs that are shared. Examination of these motifs allows us to characterize protein families in greater depth and even generate new ``in silico" proteins that are not found in nature, but exhibit properties of a particular protein family. Using novel deep learning approaches and leveraging the large volume of genomic data that is now available due to high-throughput DNA sequencing, it is now possible to examine protein families in a scale and resolution that has never before been possible. By using this abundance of data to learn high dimensional representations of amino acids sequences, in this work, we show that it is possible to generate novel sequences from a particular protein family. Such a deep sequential model-based approach has great value for bioinformatics and biotechnological applications due to its rapid sampling abilities. / Master of Science / Proteins are one of the most important functional biological elements. These are composed of amino acids which link together to form different shapes which might encode a particular function. These proteins may act independently or might form ``complexes" to have a particular function. Therefore, understanding them is of utmost importance. Due to the fact that there are 20 amino acids even a protein sequence fragment of length 5 can have more than 3 million different combinations. Given, that proteins are generally 1000 amino acids long, looking at all the possibilities is next to impossible. In this work, by leveraging the ``deep learning" paradigm and the vast amount of data available, we try to model these proteins and generate new proteins belonging to a specific ``protein family." This approach has great value for bioinformatics and biotechnological applications due to its rapid sampling abilities.

The thesis focuses on applying deep learning and reinforcement learning techniques on human keypoint detection and robot arm manipulation. Inspired by Semi-Autonomous Victim Extraction Robot (SAVER), an autonomous first-aid airway-management robotic system designed to perform Cricothyrotomy on patients is proposed. Perception, decision-making, and control are embedded in the system. In this system, first, the location of the cricothyroid membrane (CTM)-the incision site of Cricothyrotomy- is detected; then, the robot arm is controlled to reach the detected position on a medical manikin. A hybrid neural network (HNNet) that can balance both speed and accuracy is proposed. HNNet is an ensemble-based network architecture that consists of two ensembles: the region proposal ensemble and the keypoint detection ensemble. This architecture can maintain the original high resolution of the input image without heavy computation and can meet the high-precision and real-time requirements at the same time. A dataset containing more than 16,000 images from 13 people, with a clear view of the neck area, and with CTM position labeled by a medical expert was built to train and validate the proposed model. It achieved a success rate of $99.6%$ to detect the position of the CTM with an error of less than 5mm. The robot arm manipulator was trained with the reinforcement learning model to reach the detected location. Finally, the detection neural network and the manipulation process are combined as an integrated system. The system was validated in real-life experiments on a human-sized medical manikin using a Kinect V2 camera and a MICO robot arm manipulator. / Master of Science / The thesis focuses on applying deep learning and reinforcement learning techniques on human keypoint detection and robot arm manipulation. Inspired by Semi-Autonomous Victim Extraction Robot (SAVER), an autonomous first-aid airway-management robotic system designed to perform Cricothyrotomy on patients is proposed. Perception, decision-making, and control are embedded in the system. In this system, first, the location of the cricothyroid membrane(CTM)-the incision site of Cricothyrotomy- is detected; then, the robot arm is controlled to reach the detected position on a medical manikin. A hybrid neural network (HNNet) that can balance both speed and accuracy is proposed. HNNet is an ensemble-based network architecture that consists of two ensembles: the region proposal ensemble and the keypoint detection ensemble. This architecture can maintain the original high resolution of the input image without heavy computation and can meet the high-precision and real-time requirements at the same time. Finally, the detection neural network and the manipulation process are combined as an integrated system. The robot arm manipulator was trained with the reinforcement learning model to reach the detected location. The system was validated in real-life experiments on a human-sized medical manikin using an RGB-D camera and a robot arm manipulator.

Metal additive manufacturing (AM) provides a platform for microstructure optimization via process control, the ability to model the evolution of microstructures from changes in processing condition or even predict the microstructures from given processing condition would greatly reduce the time frame and the cost of the optimization process. In 1, we present a deep learning framework to quantitatively analyze the microstructural variations of metals fabricated by AM under different processing conditions. We also demonstrate the capability of predicting new microstructures from the representation with deep learning and we can explore the physical insights of the implicitly expressed microstructure representations. We validate our framework using samples fabricated by a solid-state AM technology, additive friction stir deposition, which typically results in equiaxed microstructures. In 2, we further improve and generalize the generating framework, a set of metrics is used to quantitatively analyze the effectiveness of the generation by comparing the microstructure characteristics between the generated samples and the originals. We also take advantage of image processing techniques to aid the calculation of metrics that require grain segmentation. / Master of Science / Different from the traditional manufacturing technique which removes material to form the desired shape, additive manufacturing (AM) adds material together to form the shapes usually layer by layer. AM which is sometimes also referred to as 3-D printing enables the optimization of material property through changing the processing conditions. The microstructure is structures formed by materials on a microscopic scale. Crystals like metal usually form a crystalline structure composed of grains where atoms have the same orientation. Especially for metal AM, changes in the processing condition will usually result in changes in microstructures and material properties. To better optimize for the desired material properties, in 1 we present a microstructure representation method that allows projection of microstructure onto the representation space and prediction from an arbitrary point from the representation space. This representation method allows us to better analyze the changes in microstructure in relation to the changes in processing conditions. In 2, we validate the representation and prediction using EBSD data collected from copper samples manufactured with AM under different processing conditions.

Visual recognition tasks have witnessed vast improvements in performance since the advent of deep learning. Despite the gains in performance, image understanding algorithms are still not completely robust to partial occlusion. In this work, we propose a novel object classification method based on compositional modeling and explore its effect in the context of the newly introduced panoptic segmentation task. The panoptic segmentation task combines both semantic and instance segmentation to perform labelling of the entire image. The novel classification method replaces the object detection pipeline in UPSNet, a Mask R-CNN based design for panoptic segmentation. We also discuss an issue with the segmentation mask prediction of Mask R-CNN that affects overlapping instances. We perform extensive experiments and showcase results on the complex COCO and Cityscapes datasets. The novel classification method shows promising results for object classification on occluded instances in complex scenes. / Master of Science / Visual recognition tasks have witnessed vast improvements in performance since the advent of deep learning. Despite making significant improvements, algorithms for these tasks still do not perform well at recognizing partially visible objects in the scene. In this work, we propose a novel object classification method that uses compositional models to perform part based detection. The method first looks at individual parts of an object in the scene and then makes a decision about its identity. We test the proposed method in the context of the recently introduced panoptic segmentation task. The panoptic segmentation task combines both semantic and instance segmentation to perform labelling of the entire image. The novel classification method replaces the object detection module in UPSNet, a Mask R-CNN based algorithm for panoptic segmentation. We also discuss an issue with the segmentation mask prediction of Mask R-CNN that affects overlapping instances. After performing extensive experiments and evaluation, it can be seen that the novel classification method shows promising results for object classification on occluded instances in complex scenes.

This thesis proposes a deep learning approach to predict stress at various stages of mechanical loading in 2-D representations of fiber-reinforced composites. More specifically, the full-field stress distribution at elastic and at an early stage of damage initiation is predicted based on the microstructural geometry. The required data set for the purposes of training and validation are generated via high-fidelity simulations of several randomly generated microstructural representations with complex geometries. Two deep learning approaches are employed and their performances are compared: fully convolutional generator and Pix2Pix translation. It is shown that both the utilized approaches can well predict the stress distributions at the designated loading stages with high accuracy. / M.S. / Fiber-reinforced composites are material types with excellent mechanical performance. They form the major material in the construction of space shuttles, aircraft, fancy cars, etc., the structures that are designed to be lightweight and at the same time extremely stiff and strong. Due to the broad application, especially in the sensitives industries, fiber-reinforced composites have always been a subject of meticulous research studies. The research studies to better understand the mechanical behavior of these composites has to be conducted on the micro-scale. Since the experimental studies on micro-scale are expensive and extremely limited, numerical simulations are normally adopted. Numerical simulations, however, are complex, time-consuming, and highly computationally expensive even when run on powerful supercomputers. Hence, this research aims to leverage artificial intelligence to reduce the complexity and computational cost associated with the existing high-fidelity simulation techniques. We propose a robust deep learning framework that can be used as a replacement for the conventional numerical simulations to predict important mechanical attributes of the fiber-reinforced composite materials on the micro-scale. The proposed framework is shown to have high accuracy in predicting complex phenomena including stress distributions at various stages of mechanical loading.

Image classification has been a pivotal area of research in Deep Learning, with a vast body of literature working to tackle the problem, constantly striving to achieve higher accuracies. This push to reach achieve greater prediction accuracy however, has further exacerbated the black box phenomenon which is inherent of neural networks, and more for so CNN style deep architectures. Likewise, it has lead to the development of highly tuned methods, suitable only for a specific data sets, requiring significant work to alter given new data. Although these models are capable of producing highly accurate predictions, we have little to no ability to understand the decision process taken by a network to reach a conclusion. This factor poses a difficulty in use cases such as medical diagnostics tools or autonomous vehicles, which require insight into prediction reasoning to validate a conclusion or to debug a system. In essence, modern applications which utilize deep networks are able to learn to produce predictions, but lack interpretability and a deeper understanding of the data. Given this key point, we look to decision trees, opposite in nature to deep networks, with a high level of interpretability but a low capacity for learning. In our work we strive to merge these two techniques as a means to maintain the capacity for learning while providing insight into the decision process. More importantly, we look to expand the understanding of class relationships through a tree architecture. Our ultimate goal in this work is to create a technique able to automatically create a visual feature based knowledge hierarchy for class relations, applicable broadly to any data set or combination thereof. We maintain these goals in an effort to move away from specific systems and instead toward artificial general intelligence (AGI). AGI requires a deeper understanding over a broad range of information, and more so the ability to learn new information over time. In our work we embed networks of varying sizes and complexity within decision trees on a node level, where each node network is responsible for selecting the next branch path in the tree. Each leaf node represents a single class and all parent and ancestor nodes represent groups of classes. We designed the method such that classes are reasonably grouped by their visual features, where parent and ancestor nodes represent hidden super classes. Our work aims to introduce this method as a small step towards AGI, where class relations are understood through an automatically generated decision tree (representing a class hierarchy), capable of accurate image classification. / Master of Science / Many modern day applications make use of deep networks for image classification. Often these networks are incredibly complex in architecture, and applicable only for specific tasks and data. Standard approaches use just a neural network to produce predictions. However, the internal decision process of the network remains a black box due to the nature of the technique. As more complex human related applications, such as medical image diagnostic tools or autonomous driving software, are being created, they require an understanding of reasoning behind a prediction. To provide this insight into the prediction reasoning, we propose a technique which merges decision trees and deep networks. Tested on the MNIST image data set we were able to achieve an accuracy over 99.0%. We were also able to achieve an accuracy over 73.0% on the CIFAR-10 image data set. Our method is found to create decision trees that are easily understood and are reasonably capable of image classification.

X-ray Computed Tomography (CT) imaging has been playing a central role in clinical practice since it was invented in 1972. However, the traditional x-ray CT technique fails to distinguish different materials with similar density, especially for biological tissues. The lack of a quantitative imaging representation has constrained the application of CT technique from a broadening application such as personal or precision medicine. Therefore, my major thesis statement is to develop novel material-specific CT imaging techniques for molecular imaging in biological bodies. To achieve the goal, comprehensive studies were conducted to investigate three different techniques: x-ray fluorescence molecular imaging, material identification (specification) from photon counting CT, and photon counting CT data distortion correction approach based on deep learning. X-ray fluorescence molecular imaging (XFMI) has shown great promise as a low-cost molecular imaging modality for clinical and pre-clinical applications with high sensitivity. In this study, the effects of excitation beam spectrum on the molecular sensitivity of XFMI were experimentally investigated, by quantitatively deriving minimum detectable concentration (MDC) under a fixed surface entrance dose of 200 mR at three different excitation beam spectra. The result shows that the MDC can be readily increased by a factor of 5.26 via excitation spectrum optimization. Furthermore, a numerical model was developed and validated by the experimental data (≥0.976). The numerical model can be used to optimize XFMI system configurations to further improve the molecular sensitivity. Findings from this investigation could find applications for in vivo pre-clinical small-animal XFMI in the future. PCCT is an emerging technique that has the ability to distinguish photon energy and generate much richer image data that contains x-ray spectral information compared to conventional CT. In this study, a physics model was developed based on x-ray matter interaction physics to calculate the effective atomic number () and effective electron density () from PCCT image data for material identification. As the validation of the physics model, the and were calculated under various energy conditions for many materials. The relative standard deviations are mostly less than 1% (161 out of 168) shows that the developed model obtains good accuracy and robustness to energy conditions. To study the feasibility of applying the model with PCCT image data for material identification, both PCCT system numerical simulation and physical experiment were conducted. The result shows different materials can be clearly identified in the − map (with relative error ≤8.8%). The model has the value to serve as a material identification scheme for PCCT system for practical use in the future. As PCCT appears to be a significant breakthrough in CT imaging field, there exists severe data distortion problem in PCCT, which greatly limits the application of PCCT in practice. Lately, deep learning (DL) neural network has demonstrated tremendous success in medical imaging field. In this study, a deep learning neural network based PCCT data distortion correction method was proposed. When applying the algorithm to process the test dataset data, the accuracy of the PCCT data can be greatly improved (RMSE improved 73.7%). Compared with traditional data correction approaches such as maximum likelihood, the deep learning approach demonstrate superiority in terms of RMSE, SSIM, PSNR, and most importantly, runtime (4053.21 sec vs. 1.98 sec). The proposed method has the potential to facilitate the PCCT studies and applications in practice. / Doctor of Philosophy / X-ray Computed Tomography (CT) has played a central role in clinical imaging since it was invented in 1972. It has distinguishing characteristics of being able to generate three dimensional images with comprehensive inner structural information in fast speed (less than one second). However, traditional CT imaging lacks of material-specific capability due to the mechanism of image formation, which makes it cannot be used for molecular imaging. Molecular imaging plays a central role in present and future biomedical research and clinical diagnosis and treatment. For example, imaging of biological processes and molecular markers can provide unprecedented rich information, which has huge potentials for individualized therapies, novel drug design, earlier diagnosis, and personalized medicine. Therefore there exists a pressing need to enable the traditional CT imaging technique with material-specific capability for molecular imaging purpose. This dissertation conducted comprehensive study to separately investigate three different techniques: x-ray fluorescence molecular imaging, material identification (specification) from photon counting CT, and photon counting CT data distortion correction approach based on deep learning. X-ray fluorescence molecular imaging utilizes fluorescence signal to achieve molecular imaging in CT; Material identification can be achieved based on the rich image data from PCCT; The deep learning based correction method is an efficient approach for PCCT data distortion correction, and furthermore can boost its performance on material identification. With those techniques, the material-specific capability of CT can be greatly enhanced and the molecular imaging can be approached in biological bodies.

Today's news agencies are moving from traditional journalism, where publishing just a few news articles per day was sufficient, to modern content generation mechanisms, which create more than thousands of news pieces every day. With the growth of these modern news agencies comes the arduous task of properly handling this massive amount of data that is generated for each news article. Therefore, news agencies are constantly seeking solutions to facilitate and automate some of the tasks that have been previously done by humans. In this dissertation, we focus on some of these problems and provide solutions for two broad problems which help a news agency to not only have a wider view of the behaviour of readers around the article but also to provide an automated tools to ease the job of editors in summarizing news articles. These two disjoint problems are aiming at improving the users' reading experience by helping the content generator to monitor and focus on poorly performing content while allow them to promote the good-performing ones. We first focus on the task of popularity prediction of news articles via a combination of regression, classification, and clustering models. We next focus on the problem of generating automated text summaries for a long news article using deep learning models. The first problem aims at helping the content developer in understanding of how a news article is performing over the long run while the second problem provides automated tools for the content developers to generate summaries for each news article. / Doctor of Philosophy / Nowadays, each person is exposed to an immense amount of information from social media, blog posts, and online news portals. Among these sources, news agencies are one of the main content providers for each person around the world. Contemporary news agencies are moving from traditional journalism to modern techniques from different angles. This is achieved either by building smart tools to track the behaviour of readers’ reaction around a specific news article or providing automated tools to facilitate the editor’s job in providing higher quality content to readers. These systems should not only be able to scale well with the growth of readers but also they have to be able to process ad-hoc requests, precisely since most of the policies and decisions in these agencies are taken around the result of these analytical tools. As part of this new movement towards adapting new technologies for smart journalism, we have worked on various problems with The Washington Post news agency on building tools for predicting the popularity of a news article and automated text summarization model. We develop a model that monitors each news article after its publication and provide prediction over the number of views that this article will receive within the next 24 hours. This model will help the content creator to not only promote potential viral article in the main page of the web portal or social media, but also provide intuition for editors on potential poorly performing articles so that they can edit the content of those articles for better exposure. On the other hand, current news agencies are generating more than a thousands news articles per day and generating three to four summary sentences for each of these news pieces not only become infeasible in the near future but also very expensive and time-consuming. Therefore, we also develop a separate model for automated text summarization which generates summary sentences for a news article. Our model will generate summaries by selecting the most salient sentence in the news article and paraphrase them to shorter sentences that could represent as a summary sentence for the entire document.

Computer vision techniques are becoming increasingly popular. For example, face recognition is used to help police find criminals, vehicle detection is used to prevent drivers from serious traffic accidents, and written word recognition is used to convert written words into printed words. With the rapid development of vehicle detection given the use of deep learning techniques, there are still concerns about the performance of state-of-the-art vehicle detection techniques. For example, state-of-the-art vehicle detectors are restricted by the large variation of scales. People working on vehicle detection are developing techniques to solve this problem. This thesis proposes an advanced vehicle detection model, adopting one of the classical neural networks, which are the residual neural network and the region proposal network. The model utilizes the residual neural network as a feature extractor and the region proposal network to detect the potential objects' information. / Master of Science / Computer vision techniques are becoming increasingly popular. For example, face recognition is used to help police find criminals, vehicle detection is used to prevent drivers from serious traffic accidents, and written word recognition is used to convert written words into printed words. With the rapid development of vehicle detection given the use of deep learning techniques, there are still concerns about the performance of state-of-the art vehicle detection techniques. For example, state-of-the-art vehicle detectors are restricted by the large variation of scales. People working on vehicle detection are developing techniques to solve this problem. This thesis proposes an advanced vehicle detection model, utilizing deep learning techniques to detect the potential objects’ information.