Capsule Networks are the next generation of image classifiers. Although they have several advantages over conventional Convolutional Neural Networks (CNNs), they remain computationally heavy. Since inference on Capsule Networks is timeconsuming, thier usage becomes limited to tasks in which latency is not essential. Approximation methods in Deep Learning help networks lose redundant parameters to increase speed and lower energy consumption. In the first part of this work, we go through an algorithm called zero-skipping. More than 50% of trained CNNs consist of zeros or values small enough to be considered zero. Since multiplication by zero is a trivial operation, the zero-skipping algorithm can play a massive role in speed increase throughout the network. We investigate the eligibility of Capsule Networks for this algorithm on two different datasets. Our results suggest that Capsule Networks contain enough zeros in their Primary Capsules to benefit from this algorithm. In the second part of this thesis, we investigate pruning as one of the most popular Neural Network approximation methods. Pruning is the act of finding and removing neurons which have low or no impact on the output. We run experiments on four different datasets. Pruning Capsule Networks results in the loss of redundant Primary Capsules. The results show a significant increase in speed with a minimal drop in accuracy. We also, discuss how dataset complexity affects the pruning strategy. / Graduate

3-monochloropropane-1,2-diol or 3-chloropropane-1,2-diol (3-MCPD) and glycidol are the most commonly occurring group of thermal process contaminants which are considered as “possible human carcinogen” and “probably carcinogenic to humans”, respectively. Potato strips prepared from three different potatoes cultivars (Russet Burbank, Ranger Russet, and Umatilla Russet) grown in North Dakota from the crop year 2018 were fried with vegetable oil at 190 ºC, respectively, for five consecutive days (8 h/day). The dynamic changes of 3-MCPD and glycidol equivalents were investigated during deep-fat frying. 3-MCPD equivalent in oil and potato strips decreased with increased frying time. Meanwhile, the content of glycidol equivalent increased with increased frying time. The major 3-MCPD and glycidol equivalents that were detected in the fried potato strips were those that migrated from the oils during frying. The application of absorbents, i.e., Magnesol and Celite, achieved the mitigation of 3-MCPD and glycidol in frying oil.

Deep water corals are an understudied yet biologically important and fragile ecosystem under threat from recent increasing temperatures and high carbon dioxide emissions. Using 454 sequencing, we develop 14 new microsatellite markers for the deep water coral Eguchipsammia fistula, collected from the Red Sea but found in deep water coral ecosystems globally. We tested these microsatellite primers on 26 samples of this coral collected from a single population. Results show that these corals are highly clonal within this population stemming from a high level of asexual reproduction. Mitochondrial studies back up microsatellite findings of high levels of genetic similarity. CO1, ND1 and ATP6 mitochondrial sequences of E. fistula and 11 other coral species were used to build phylogenetic trees which grouped E. fistula with shallow water coral Porites rather than deep sea L. Petusa.

Many well-studied elasmobranch populations have recently exhibited significant decline. The limited data related to fisheries and sightings for many unstudied or poorly understood populations indicate that these are also suffering. Directed fisheries and more cryptic threats such as entanglement and vessel strike represent significant risk to mobulid rays, arguably one of the most vulnerable elasmobranch groups. Very little information currently exists describing the basic ecology of manta rays or quantifying anthropogenic threats and impacts; however, recent efforts have drastically improved the body of knowledge available for these species, including oceanographic influences on movement, seasonal migration, and mating behaviors. Nevertheless, Red Sea mantas remain completely enigmatic. In this thesis, Chapter 1 details results from tagging 18 reef manta rays Manta alfredi in the eastern Red Sea using satellite and acoustic tag technology and demonstrates that mantas occupy areas with high human traffic. The combined satellite and acoustic techniques define both regional movements and ‘hotspots’ of habitat use where there is significant potential for manta-human interaction. I also present opportunistic sighting data that corroborate anthropogenic impacts on this population. Chapter 2 explores the vertical component of the nine satellite tags that were deployed on Manta alfredi as described in the previous chapter. Seven tags returned adequate data for analysis. Three of the seven were physically recovered yielding full archival datasets of depth, temperature, and light levels every 10-15 seconds for over 2.6 5 million cumulative data points. Mantas frequented the upper 10 m during the day and occupied deeper water through nocturnal periods. Individuals also exhibited deep diving behavior as deep as 432 m, extending the known depth range of the species. An investigation of 76 high-resolution deep dives suggests gliding is a significant behavioral component of these dives and may provide an efficient mechanism for travel compared to continuous horizontal swimming. This study is the first to employ satellite telemetry techniques on Manta alfredi and is the only study directed at mobulids in the Red Sea. A holistic understanding of these behaviors is essential for developing and implementing appropriate management techniques, and this work is particularly timely in light of recent international trade regulation as mantas were listed on Appendix II of the Convention on International Trade in Endangered Species.

Image degradation arises from various environmental conditions due to the exis tence of aerosols such as fog, haze, and dust. These phenomena mitigate image vis ibility by creating color distortion, reducing contrast, and fainting object surfaces. Although the end-to-end deep learning approach has made significant progress in the field of homogeneous dehazing, the image quality of these algorithms in the context of non-homogeneous real-world images has not yet been satisfactory. We argue two main reasons that are responsible for the problem: 1) First, due to the unbalanced information processing of the high-level and low-level information in conventional dehazing algorithms, 2) due to lack of trainable data pairs. To ad dress the above two problems, we propose a parallel dual-branch design that aims to balance the processing of high-level and low-level information, and through a method of transfer learning, utilize the small data sets to their full potential. The results from the two parallel branches are aggregated in a simple fusion tail, in which the high-level and low-level information are fused, and the final result is generated. To demonstrate the effectiveness of our proposed method, we present extensive experimental results in the thesis. / Thesis / Master of Applied Science (MASc)

In aviation, safety is a big concern. Knowing the position of an aircraft at all times is of high importance. Today most aircraft utilize Global Navigation Satellite Systems (GNSS) for localization and precision navigation because of the small position error which do not increase over time. However, recent research show that GNSS can easily be jammed or spoofed. An alternative navigation method is Visual Odometry (VO). VO is navigation through visual input and is a key-part in development of fully autonomous vehicles. This thesis investigates the Deep Learning-based Visual Odometry (DL-VO) for aircraft at altitudes over 100 m. DL-VO deployed at high altitude is almost none existing. Therefore, this thesis investigates the deployments of ground developed DL-VO in the aerial domain. DeepVO is a Frame-To-Frame optical flow estimation method which is trained supervised and end-to-end. The domain change, from ground to high altitude aerial, brought bigger issues and had larger impact on the performance than first though. The use of full 6 Degrees of Freedom (DoF) pose estimation increases the complexity and was much harder than 2D estimation (x, y, yaw). A good angle representation was of higher importance during training and testing in the aerial domain. Since in the aerial domain the full 3D rotation is not unique in all representations of the orientation and issues with Gimbal lock can occur. Results on simulated data show that the propose method fails to estimate 6 DoF poses. However, the reduced 2D estimations shows that a trajectory can be maintained even is drift is present. The result on real world dataset shows that it easier to recover scale at lower speeds and with a less down angled camera. The difference between simulated and non-simulated data has not been investigated to the extent that a fair assessment on how the method’s performance is affected by the data character. / Flygsäkerhet är av stor vikt inom flygindustin. Att som pilot alltid veta var planet befinner sig är av stor vikt. Global Navigation Satellite Systems (GNSS) är idag den mest använda metoden för lokalisering och precisionsnavigering då GNSS har liten felmarginal som inte förvärras över tid. Nyligen har forskare visat att GNSS kan lätt störas och alternativa lokaliseringsmetoder behövs. En av dem är Visual Odometry (VO). VO metoder försöker navigera sig i olika miljöer genom att estimera kamerors rörelser i sekvens av bilder. Det pågår mycket forsking på området då det är ett nyckelkoncept för autonoma fordon. Detta arbete undersöker användadet av Deep Learning-based Visual Odometry (DL-VO) för flygfarkoster på höjder över 100 m. Det är väldigt få som har testat DL-VO på annat än små drönare vilket skiljer från flygplan på högre höjd som stöter på andra problem där alla obejekt är väldigt små. Då forskingen på DL-VO för flygplan på högre höjd är minimal undersöker arbetet ett domän byte genom att ta en metod utveklad för markfordon och flytta den till flygdomänen. För att undersöka bytet av domän avändes en anpassad version av DeepVO nätverket. DeepVO använder sig av realtiv Frame-To-Frame optiskt flödes estimeringar och är tränad end-to-end enligt supervised learning metoden. Domän bytet, från mark till luft, medförde större problem än först trott och det ökade komplexiteten på problemet. Estimeringar med 6 frihetgrader är mer komplexa och en bra vinkel representation är av mycket större vikt. Minimering av vinklar under träningen skapade andra problem i flygdomänen än vad det gjorde på ursrungliga datasetet. Resultaten på simiulerad data visar att den framtagna metoden inte klarar estimeringar med 6 frihetgrader. Men om problemet reduceras så kan metoden estimaera 2D banor på en fixerad höjd i luften även om viss drift över tid existerar. Kameravinkeln och hastighet påverkar metodens förmåga att hålla en korrekt skala. Resultat på verklig data visar att det är lättare att uppnå korrekt skala vid lägre hastighet och mindre nervinklad kamera. Skillnaderna mellan simulerad och verklig data har inte undersökts i den utsträktning som behövs för att göra en korrekt slutsats om dess efftekter på resultatet.

Using machine learning, especially deep learning, to facilitate biological research is a fascinating research direction. However, in addition to the standard classi cation or regression problems, whose outputs are simple vectors or scalars, in bioinformatics, we often need to predict more complex structured targets, such as 2D images and 3D molecular structures. The above complex prediction tasks are referred to as structured prediction. Structured prediction is more complicated than the traditional classi cation but has much broader applications, especially in bioinformatics, considering the fact that most of the original bioinformatics problems have complex output objects. Due to the properties of those structured prediction problems, such as having problem-speci c constraints and dependency within the labeling space, the straightforward application of existing deep learning models on the problems can lead to unsatisfactory results. In this dissertation, we argue that the following two ideas can help resolve a wide range of structured prediction problems in bioinformatics. Firstly, we can combine deep learning with other classic algorithms, such as probabilistic graphical models, which model the problem structure explicitly. Secondly, we can design and train problem-speci c deep learning architectures or methods by considering the structured labeling space and problem constraints, either explicitly or implicitly. We demonstrate our ideas with six projects from four bioinformatics sub elds, including sequencing analysis, structure prediction, function annotation, and network analysis. The structured outputs cover 1D electrical signals, 2D images, 3D structures, hierarchical labeling, and heterogeneous networks. With the help of the above ideas, all of our methods can achieve state-of-the-art performance on the corresponding problems. The success of these projects motivates us to extend our work towards other more challenging but important problems, such as health-care problems, which can directly bene t people's health and wellness. We thus conclude this thesis by discussing such future works, and the potential challenges and opportunities.

SARS-CoV-2 has plagued the world since 2019 with continuously emergence of new variants, resulting in repeated waves of outbreak. Although the countermeasures like vaccination campaign has taken worldwide, the sophisticated virus mutated to escape immune system, threatening the public health. To win the race with the virus and ultimately end the pandemic, we have to take one step ahead to predict how the SARSCoV-2 might evolve and defeat it at the beginning of a new wave. Hence, we proposed a deep learning based framework to first build a deep learning model to shape the fitness landscape of the virus and then use genetic algorithm to predict the high-risk variants that might appear in the future. By combining pre-trained protein language model and structure modeling, the model is trained in a supervised way, predicting the viral transmissibility and antibodies escape ability to eight antibodies simultaneously. The prevenient virus evolution trajectory can be largely recovered by our model with high correlation to their sampling time. Novel mutations predicted by our model show high antibody escape through in silico simulation and overlapped with the mutations developed in prevenient infected patients. Overall, our scheme can provide insights into the evolution of SARS-CoV-2 and hopefully guide the development of vaccination and increase the preparedness.

With the desired deployment of Artificial Intelligence (AI), concerns over whether AI can “communicate” why it has made its decisions is of particular importance. In this thesis, we utilize predictive entropy (PE) as an surrogate for predictive uncertainty and report it for various test-time conditions that alter the testing distribution. This is done to evaluate the potential for PE to indicate when users should trust or dis- trust model predictions under dataset shift or out-of-distribution (OOD) conditions, two scenarios that are prevalent in real-world settings. Specifically, we trained an ensemble of three 2D-UNet architectures to segment synthetically damaged regions in fractional anisotropy scalar maps, a widely used diffusion metric to indicate mi- crostructural white-matter damage. Baseline ensemble statistics report that the true positive rate, false negative rate, false positive rate, true negative rate, Dice score, and precision are 0.91, 0.091, 0.23, 0.77, 0.85, and 0.80, respectively. Test-time PE was reported before and after the ensemble was exposed to increasing geometric distortions (OOD), adversarial examples (OOD), and decreasing signal-to-noise ratios (dataset shift). We observed that even though PE shows a strong negative correlation with model performance for increasing adversarial severity (ρAE = −1), this correlation is not seen under distortion or SNR conditions (ρD = −0.26, ρSNR = −0.30). However, the PE variability (PE-Std) between individual model predictions was shown to be a better indicator of uncertainty as strong negative correlations between model performance and PE-Std were seen during geometric distortions and adversarial ex- amples (ρD = −0.83, ρAE = −1). Unfortunately, PE fails to report large absolute uncertainties during these conditions, thus restricting the analysis to correlative relationships. Finally, determining an uncertainty threshold between “certain” and “uncertain” model predictions was seen to be heavily dependant on model calibra- tion. For augmentation conditions close to the training distribution, a single threshold could be hypothesized. However, caution must be taken if such a technique is clinically applied, as model miscalibration could nullify such a threshold for samples far from the distribution. To ensure that PE or PE-Std could be used more broadly for uncertainty estimation, further work must be completed. / Thesis / Master of Applied Science (MASc)

With the rapid growth of seismic data volumes, traditional automated processing methods, which have been in use for decades, face increasing challenges in handling these data, especially in noisy environments. Deep learning (DL) methods, due to their ability to handle large datasets and perform well in complex scenarios, offer promising solutions to these challenges. When I started my Ph.D. degree, although a sizeable number of researchers were beginning to explore the application of deep learning in seismology, almost no one was involved in the development of much-needed automated data annotation tools and deep learning training platforms for this field. In other rapidly evolving fields of artificial intelligence, such automated tools and platforms are often a prerequisite and critical to advancing the development of deep learning. Motivated by this gap, my Ph.D. research focuses on creating these essential tools and conducting critical investigations in the field of earthquake detection and phase picking using DL methods. The first research chapter introduces QuakeLabeler, an open-source Python toolbox that facilitates the efficient creation and management of seismic training datasets. This tool aims to address the laborious process of producing training labels in the vast amount of seismic data available today. Building on this foundational tool, the second research chapter presents Blockly Earthquake Transformer (BET), a deep learning platform that provides an interactive dashboard for efficient customization of deep learning phase pickers. BET aims to optimize the performance of seismic event detection and phase picking by allowing easy customization of model parameters and providing extensions for transfer learning and fine-tuning. The third and final research chapter investigates the performance of DL pickers by examining the effect of training data size and deployment settings on phase picking accuracy. This investigation provides insight into the optimal size of training datasets, the suitability of DL pickers for new target regions, and the impact of various factors on training and on model performance. Through the development of these tools and investigations, this thesis contributes to the application of DL in seismology, paving the way for more efficient seismic data processing, customizable model creation, and a better understanding of DL model performance in earthquake detection and phase-picking tasks.