<p>Though modern deep learning based approaches have achieved remarkable progress in computer vision community such as image classification using a static image dataset, it suf- fers from catastrophic forgetting when learning new classes incrementally in a phase-by-phase fashion, in which only data for new classes are provided at each learning phase. In this work we focus on continual learning with the objective of learning new tasks from sequentially available data without forgetting the learned knowledge. We study this problem from three perspectives including (1) continual learning in online scenario where each data is used only once for training (2) continual learning in unsupervised scenario where no class label is pro- vided and (3) continual learning in real world applications. Specifically, for problem (1), we proposed a variant of knowledge distillation loss together with a two-step learning technique to efficiently maintain the learned knowledge and a novel candidates selection algorithm to reduce the prediction bias towards new classes. For problem (2), we introduced a new framework for unsupervised continual learning by using pseudo labels obtained from cluster assignments and an efficient out-of-distribution detector is designed to identify whether each new data belongs to new or learned classes in unsupervised scenario. For problem (3), we proposed a novel training regime targeted on food images using balanced training batch and a more efficient exemplar selection algorithm. Besides, we further proposed an exemplar-free continual learning approach to address the memory issue and privacy concerns caused by storing part of old data as exemplars.</p> <p>In addition to the work related to continual learning, we study the image-based dietary assessment with the objective of determining what someone eats and how much energy is consumed during the course of a day by using food or eating scene images. Specifically, we proposed a multi-task framework for simultaneously classification and portion size estima- tion by future fusion and soft-parameter sharing between backbone networks. Besides, we introduce RGB-Distribution image by concatenating the RGB image with the energy distri- bution map as the fourth channel, which is then used for end-to-end multi-food recognition and portion size estimation.</p>

Facade parsing is a fundamental problem in urban modeling that forms the back- bone of a variety of tasks including procedural modeling, architectural analysis, urban reconstruction and quite often relies on semantic segmentation as the first step. With the shift to deep learning based approaches, existing small-scale datasets are the bot- tleneck for making further progress in fa ̧cade segmentation and consequently fa ̧cade parsing. In this thesis, we propose a new fa ̧cade image dataset for semantic segmenta- tion called PSV-22, which is the largest such dataset. We show that PSV-22 captures semantics of fa ̧cades better than existing datasets. Additionally, we propose three architectural modifications to current state of the art deep-learning based semantic segmentation architectures and show that these modifications improve performance on our dataset and already existing datasets. Our modifications are generalizable to a large variety of semantic segmentation nets, but are fa ̧cade-specific and employ heuris- tics which arise from the regular grid-like nature of fac ̧ades. Furthermore, results show that our proposed architecture modifications improve the performance compared to baseline models as well as specialized segmentation approaches on fa ̧cade datasets and are either close in, or improve performance on existing datasets. We show that deep models trained on existing data have a substantial performance reduction on our data, whereas models trained only on our data actually improve when evaluated on existing datasets. We intend to release the dataset publically in the future.

As self-awareness is important to human higher level cognition so too is the ability to self-model important to performing complex behaviors. The power of these self-models is one that I demonstrate grows with the complexity of problems being solved, and thus provides the framework for higher level cognition. I demonstrate that self-models can be used to effectively control and improve on existing control algorithms to allow agents to perform complex tasks. I further investigate new ways in which these self-models can be learned and applied to increase their efficacy and improve the ability of these models to generalize across tasks and bodies. Finally, I demonstrate the overall power of these self-models to allow for complex tasks to be completed with little data across a variety of bodies and using a number of algorithms.

京都大学 / 新制・課程博士 / 博士(農学) / 甲第23944号 / 農博第2493号 / 新制||農||1090(附属図書館) / 学位論文||R4||N5379(農学部図書室) / 京都大学大学院農学研究科森林科学専攻 / (主査)教授 德地 直子, 教授 北山 兼弘, 教授 神﨑 護, 准教授 伊勢 武史 / 学位規則第4条第1項該当 / Doctor of Agricultural Science / Kyoto University / DFAM

<p>Every optimization problem shares the common objective of finding a minima/maxima, but its application spans over a wide variety of fields ranging from solving NP-hard problems to training a neural network. This thesis addresses two crucial aspects of the above-mentioned fields. The first project is concerned with designing a hardware-system for efficiently solving Traveling Salesman Problem (TSP). It involves encoding  the solution to the ground state of an Ising Hamiltonian and finding the minima of the energy landscape. To that end, we i) designed a stochastic nanomagnet-based device as a building block for the system, ii) developed a unique approach to encode any TSP into an array of these blocks, and finally, iii) established the operating principle to make the system converge to an optimal solution. We used this method to solve TSPs having more than 600 nodes.</p> <p>  </p> <p>The next parts of the thesis deal with another genre of optimization problems involving deep neural networks (DNN) in image-classification tasks. DNNs are trained by finding the minima of a loss landscape aimed at mapping input images to a set of discrete labels. Adversarial attacks tend to disrupt this mapping by corrupting the inputs with subtle perturbations, imperceptible to human eyes. Although it is imperative to deploy some external defense mechanisms to guard against these attacks, the defense procedure can be aided by some intrinsic robust properties of the network. In the quest for an inherently resilient neural network, we explored the robustness of biologically-inspired Spiking Neural Networks (SNN) in the second part of the thesis. We demonstrated that accuracy degradation is less severe in SNNs than in their non-spiking counterparts. We attribute this robustness to two fundamental characteristics of SNNs: (i) input discretization  and (ii) leak rate in Leaky-Integrate-Fire neurons and analyze their effects.</p> <p><br></p> <p>As mentioned beforehand, this intrinsic robustness is merely an aiding tool to external defense mechanisms. Adversarial training has been established as the stat-of-the-art defense to provide significant robustness against existing attack techniques. This method redefines the boundary of the neural network by augmenting the training dataset with adversarial samples. In the process of achieving robustness, we are faced with a trade-off: a decrease in the prediction accuracy of clean or unperturbed data. The goal of the last section of my thesis is to understand this setback by using Gradient Projection-based sequential learning as an analysis tool. We systematically analyze the interplay between clean training and adversarial training on parameter subspace. In this technique, adversarial training follows clean training task where the parameter update is performed in the orthogonal direction of the previous task (clean training). It is possible to track down the principal component directions responsible for adversarial training by restricting clean and adversarial parameter update to two orthogonal subspaces. By varying the partition of subspace, we showed that the low-variance principal components are not capable of learning adversarial data, rather it is necessary to perform parameter update in a common subspace consisting of higher variance principal components to obtain significant adversarial accuracy. However, disturbing these higher variance components causes the decrease in standard clean accuracy, hence the accuracy-robustness trade-off. Further, we showed that this trade-off is worsened</p> <p>when the network capacity is smaller due to under-parameterization effect.</p>

The image segmentation task divides an image into regions of similar pixels based on brightness, color, and texture, in which every pixel in the image is as- signed to a label. Segmentation is vital in numerous medical imaging applications, such as quantifying the size of tissues, the localization of diseases, treatment plan- ning, and surgery guidance. This thesis focuses on two medical image segmentation tasks: retinal vessel segmentation in fundus images and brain segmentation in 3D MRI images. Finally, we introduce LEON, a lightweight neural network for edge detection. The first part of this thesis proposes a lightweight neural network for retinal blood vessel segmentation. Our model achieves cutting-edge outcomes with fewer parameters. We obtained the most outstanding performance results on CHASEDB1 and DRIVE datasets with an F1 measure of 0.8351 and 0.8242, respectively. Our model has few parameters (0.34 million) compared to other networks such as ladder net with 1.5 million parameters and DCU-net with 1 million parameters. The second part of this thesis investigates the association between whole and re- gional volumetric alterations with increasing age in a large group of healthy subjects (n=6739, age range: 30–80). We used a deep learning model for brain segmentation for volumetric analysis to extract quantified whole and regional brain volumes in 95 classes. Segmentation methods are called edge or boundary-based methods based on finding abrupt changes and discontinuities in the intensity value. The third part of the thesis introduces a new Lightweight Edge Detection Network (LEON). The proposed approach is designed to integrate the advantages of the deformable unit and DepthWise Separable convolutions architecture to create a lightweight back- bone employed for efficient feature extraction. Our experiments on BSDS500 and NYUDv2 show that LEON, while requiring only 500000 parameters, outperforms the current lightweight edge detectors without using pre-trained weights. / Graduate / 2022-10-12

This paper examines the use of convolutional neural networks to classify Covid-19 in chest radiographs. Three network architectures are compared: VGG16, ResNet-50, and DenseNet-121 along with preprocessing methods which include contrast limited adaptive histogram equalization and non-local means denoising. Chest radiographs from patients with healthy lungs, lung cancer, non-Covid pneumonia, tuberculosis, and Covid-19 were used for training and testing. Networks trained using radiographs that were preprocessed using contrast limited adaptive histogram equalization and non-local means denoising performed better than those trained on the original radiographs. DenseNet-121 performed slightly better in terms of accuracy, performance, and F1 score than all other networks but was not found to be statistically better performing than VGG16.

Gait recognition is important for identifying suspects in criminal investigations. This study will study the potential of using models based on transfer learning for this purpose. Both supervised and unsupervised learning will be examined. For the supervised learning part, the data is labeled and we investigate how accurate the models can be, and the impact of different walking conditions. Unsupervised learning is when the data is unlabeled and this part will determine if clustering can be used to identify groups of individuals without knowing who it is. Two deep learning models, the InceptionV3 model and the ResNet50V2, model are utilized, and the Gait Energy image method is used as gait representation. After optimization analysis, the models achieved the highest prediction accuracy of 100 percent when only including normal walking conditions and 99.25 percent when including different walking conditions such as carrying a backpack and wearing a coat, making them applicable for use in real-world investigations, provided that the data is labeled. Due to the apparent sensitivity of the models to varying camera angles, the clustering part resulted in an accuracy of approximately 30 percent. For unsupervised learning on gait recognition to be applicable in the real world, additional enhancements are required.

Chemosynthetic ecosystems are habitats whose food webs rely on chemosynthesis, a process by which bacteria fix carbon using energy from chemicals, rather than sunlight-driven photosynthesis for primary production, and they are found all over the world on the ocean floor. Although these deep-sea habitats are remote, they are increasingly being impacted by human activities such as oil and gas exploration and the imminent threat of deep-sea mining. My dissertation examines deep-sea chemosynthetic ecosystems at several ecological scales to answer basic biology questions and lay a foundation for future researchers studying these habitats. There are two major varieties of chemosynthetic ecosystems, hydrothermal vents and cold seeps, and my dissertation studies both. My first chapter begins at cold seeps and at the population level by modeling the population dynamics and lifespan of a single species of tubeworm, Escarpia laminata, found in the Gulf of Mexico. I found that this tubeworm, a foundation species that forms biogenic habitat for other seep animals, can reach ages over 300 years old, making it one of the longest-lived animals known to science. According to longevity theory, its extreme lifespan is made possible by the stable seep environment and lack of extrinsic mortality threats such as predation. My second chapter expands the scope of my research from this single species to the entire cold seep community and surrounding deep-sea animals common to the Gulf of Mexico. The chemicals released at cold seeps are necessary for chemosynthesis but toxic to non-adapted species such as cold-water corals. Community studies in this area have previously shown that seeps shape community assembly through niche processes. Using fine-scale water chemistry samples and photographic mapping of the seafloor, I found that depressed dissolved oxygen levels and the presence of hydrogen sulfide from seepage affect foundation taxa distributions, but the concentrations of hydrocarbons released from these seeps did not predict the distributions of corals or seep species. In my third chapter I examine seep community assembly drivers in the Costa Rica Margin and compare the macrofaunal composition at the family level to both hydrothermal vents and methane seeps around the world. The Costa Rica seep communities have not previously been described, and I found that depth was the primary driver behind community composition in this region. Although this margin is also home to a hybrid “hydrothermal seep” feature, this localized habitat did not have any discernible influence on the community samples analyzed. When vent and seep communities worldwide were compared at the family-level, geographic region was the greatest determinant of community similarity, accounting for more variation than depth and habitat type. Hydrothermal vent and methane seeps are two chemosynthetic ecosystems are created through completely different geological processes, leading to extremely different habitat conditions and distinct sets of related species. However, at the broadest spatial scale and family-level taxonomic resolution, neutral processes and dispersal limitation are the primary drivers behind community structure, moreso than whether the habitat is a seep or a vent. At more local spatial scales, the abiotic environment of seeps still has a significant influence on the ecology of deep-sea organisms. The millennial scale persistence of seeps in the Gulf of Mexico shapes the life history of vestimentiferan tubeworms, and the sulfide and oxygen concentrations at those seeps determine seep and non-seep species’ distributions across the deep seafloor. / Biology

Traditional Magnetic Resonance Imaging (MRI) reconstruction methods, which may be highly time-consuming and sensitive to noise, heavily depend on solving nonlinear optimization problems. By contrast, deep learning (DL)-based reconstruction methods do not need any explicit analytical data model and are robust to noise due to its large data-based training, which both make DL a versatile tool for fast and high-fidelity MR image reconstruction. While DL can be performed completely independently of traditional methods, it can, in fact, benefit from incorporating these established methods to achieve better results. To test this hypothesis, we proposed a hybrid DL-based MR image reconstruction method, which combines two state-of-the-art deep learning networks, U-Net and Generative Adversarial Network with Cycle loss (CycleGAN), with a traditional data reconstruction method: Projection Onto Convex Sets (POCS). Experiments were then performed to evaluate the method by comparing it to several existing state-of-the-art methods. Our results demonstrate that the proposed method outperformed the current state-of-the-art in terms of higher peak signal-to-noise ratio (PSNR) and higher Structural Similarity Index (SSIM). / Electrical and Computer Engineering