This project represents an important first step in determining the fundamental aesthetically relevant features of sound. Though there has been much effort in revealing the features learned by a deep neural network (DNN) trained on visual data, little effort in applying these techniques to a network trained on audio data has been performed. Importantly, these efforts in the audio domain often impose strong biases about relevant features (e.g., musical structure). In this project, a DNN is trained to mimic the acoustic aesthetic judgment of a professional composer. A unique corpus of sounds and corresponding professional aesthetic judgments is leveraged for this purpose. By applying a variation of Google's "DeepDream" algorithm to this trained DNN, and limiting the assumptions introduced, we can begin to listen to and examine the features of sound fundamental for aesthetic judgment. / Master of Science / The question of what makes a sound aesthetically “interesting” is of great importance to many, including biologists, philosophers of aesthetics, and musicians. This project serves as an important first step in determining the fundamental aesthetically relevant features of sound. First, a computer is trained to mimic the aesthetic judgments of a professional composer; if the composer would deem a sound “interesting,” then so would the computer. During this training, the computer learns for itself what features of sound are important for this classification. Then, a variation of Google’s “DeepDream” algorithm is applied to allow these learned features to be heard. By carefully considering the manner in which the computer is trained, this algorithmic “dreaming” allows us to begin to hear aesthetically salient features of sound.

Grünflächen stellen einen der wichtigsten Umwelteinflüsse in der Wohnumwelt der Menschen dar. Einerseits wirken sie sich positiv auf die physische und mentale Gesundheit der Menschen aus, andererseits können Grünflächen auch negative Wirkungen anderer Faktoren abmildern, wie beispielsweise die im Laufe des Klimawandels zunehmenden Hitzeereignisse. Dennoch sind Grünflächen nicht für die gesamte Bevölkerung gleichermaßen zugänglich. Bestehende Forschung im Kontext der Umweltgerechtigkeit (UG) konnte bereits aufzeigen, dass unterschiedliche sozio-ökonomische und demographische Gruppen der deutschen Bevölkerung unterschiedlichen Zugriff auf Grünflächen haben. An bestehenden Analysen von Umwelteinflüssen im Kontext der UG wird kritisiert, dass die Auswertung geographischer Daten häufig auf zu stark aggregiertem Level geschieht, wodurch lokal spezifische Expositionen nicht mehr genau abgebildet werden. Dies trifft insbesondere für großflächig angelegte Studien zu. So werden wichtige räumliche Informationen verloren. Doch moderne Erdbeobachtungs- und Geodaten sind so detailliert wie nie und Methoden des maschinellen Lernens ermöglichen die effiziente Verarbeitung zur Ableitung höherwertiger Informationen. Das übergeordnete Ziel dieser Arbeit besteht darin, am Beispiel von Grünflächen in Deutschland methodische Schritte der systematischen Umwandlung umfassender Geodaten in relevante Geoinformationen für die großflächige und hochaufgelöste Analyse von Umwelteigenschaften aufzuzeigen und durchzuführen. An der Schnittstelle der Disziplinen Fernerkundung, Geoinformatik, Sozialgeographie und Umweltgerechtigkeitsforschung sollen Potenziale moderner Methoden für die Verbesserung der räumlichen und semantischen Auflösung von Geoinformationen erforscht werden. Hierfür werden Methoden des maschinellen Lernens eingesetzt, um Landbedeckung und -nutzung auf nationaler Ebene zu erfassen. Diese Entwicklungen sollen dazu beitragen bestehende Datenlücken zu schließen und Aufschluss über die Verteilungsgerechtigkeit von Grünflächen zu bieten. Diese Dissertation gliedert sich in drei konzeptionelle Teilschritte. Im ersten Studienteil werden Erdbeobachtungsdaten der Sentinel-2 Satelliten zur deutschlandweiten Klassifikation von Landbedeckungsinformationen verwendet. In Kombination mit punktuellen Referenzdaten der europaweiten Erfassung für Landbedeckungs- und Landnutzungsinformationen des Land Use and Coverage Area Frame Survey (LUCAS) wird ein maschinelles Lernverfahren trainiert. In diesem Kontext werden verschiedene Vorverarbeitungsschritte der LUCAS-Daten und deren Einfluss auf die Klassifikationsgenauigkeit beleuchtet. Das Klassifikationsverfahren ist in der Lage Landbedeckungsinformationen auch in komplexen urbanen Gebieten mit hoher Genauigkeit abzuleiten. Ein Ergebnis des Studienteils ist eine deutschlandweite Landbedeckungsklassifikation mit einer Gesamtgenauigkeit von 93,07 %, welche im weiteren Verlauf der Arbeit genutzt wird, um grüne Landbedeckung (GLC) räumlich zu quantifizieren. Im zweiten konzeptionellen Teil der Arbeit steht die differenzierte Betrachtung von Grünflächen anhand des Beispiels öffentlicher Grünflächen (PGS), die häufig Gegenstand der UG-Forschung ist, im Vordergrund. Doch eine häufig verwendete Quelle für räumliche Daten zu öffentlichen Grünflächen, der European Urban Atlas (EUA), wird bisher nicht flächendeckend für Deutschland erhoben. Dieser Studienteil verfolgt einen datengetriebenen Ansatz, die Verfügbarkeit von öffentlichem Grün auf der räumlichen Ebene von Nachbarschaften für ganz Deutschland zu ermitteln. Hierfür dienen bereits vom EUA erfasste Gebiete als Referenz. Mithilfe einer Kombination von Erdbeobachtungsdaten und Informationen aus dem OpenStreetMap-Projekt wird ein Deep Learning -basiertes Fusionsnetzwerk erstellt, welche die verfügbare Fläche von öffentlichem Grün quantifiziert. Das Ergebnis dieses Schrittes ist ein Modell, welches genutzt wird, um die Menge öffentlicher Grünflächen in der Nachbarschaft zu schätzen (𝑅 2 = 0.952). Der dritte Studienteil greift die Ergebnisse der ersten beiden Studienteile auf und betrachtet die Verteilung von Grünflächen in Deutschland unter Hinzunahme von georeferenzierten Bevölkerungsdaten. Diese exemplarische Analyse unterscheidet dabei Grünflächen nach zwei Typen: GLC und PGS. Zunächst wird mithilfe deskriptiver Statistiken die generelle Grünflächenverteilung in der Bevölkerung Deutschlands beleuchtet. Daraufhin wird die Verteilungsgerechtigkeit anhand gängiger Gerechtigkeitsmetriken bestimmt. Abschließend werden die Zusammenhänge zwischen der demographischen Komposition der Nachbarschaft und der verfügbaren Menge von Grünflächen anhand dreier exemplarischer soziodemographischer Gesellschaftsgruppen untersucht. Die Analyse zeigt starke Unterschiede der Verfügbarkeit von PGS zwischen städtischen und ländlichen Gebieten. Ein höherer Prozentsatz der Stadtbevölkerung hat Zugriff das Mindestmaß von PGS gemessen an der Vorgabe der Weltgesundheitsorganisation. Die Ergebnisse zeigen auch einen deutlichen Unterschied bezüglich der Verteilungsgerechtigkeit zwischen GLC und PGS und verdeutlichen die Relevanz der Unterscheidung von Grünflächentypen für derartige Untersuchungen. Die abschließende Betrachtung verschiedener Bevölkerungsgruppen arbeitet Unterschiede auf soziodemographischer Ebene auf. In der Zusammenschau demonstriert diese Arbeit wie moderne Geodaten und Methoden des maschinellen Lernens genutzt werden können bisherige Limitierungen räumlicher Datensätze zu überwinden. Am Beispiel von Grünflächen in der Wohnumgebung der Bevölkerung Deutschlands wird gezeigt, dass landesweite Analysen zur Umweltgerechtigkeit durch hochaufgelöste und lokal feingliedrige geographische Informationen bereichert werden können. Diese Arbeit verdeutlicht, wie die Methoden der Erdbeobachtung und Geoinformatik einen wichtigen Beitrag leisten können, die Ungleichheit der Wohnumwelt der Menschen zu identifizieren und schlussendlich den nachhaltigen Siedlungsbau in Form von objektiven Informationen zu unterstützen und überwachen. / Green spaces are one of the most important environmental factors for humans in the living environment. On the one hand they provide benefits to people’s physical and mental health, on the other hand they allow for the mitigation of negative impacts of environmental stressors like heat waves which are increasing as a result of climate change. Yet, green spaces are not equally accessible to all people. Existing literature in the context of Environmental Justice (EJ) research has shown that the access to green space varies among different socio-economic and demographic groups in Germany. However, previous studies in the context of EJ were criticized for using strongly spatially aggregated data for their analyses resulting in a loss of spatial detail on local environmental exposure metrics. This is especially true for large-scale studies where important spatial information often get lost. In this context, modern earth observation and geospatial data are more detailed than ever, and machine learning methods enable efficient processing to derive higher value information for diverse applications. The overall objective of this work is to demonstrate and implement methodological steps that allow for the transformation of vast geodata into relevant geoinformation for the large-scale and high-resolution analysis of environmental characteristics using the example of green spaces in Germany. By bridging the disciplines remote sensing, geoinformatics, social geography and environmental justice research, potentials of modern methods for the improvement of spatial and semantic resolution of geoinformation are explored. For this purpose, machine learning methods are used to map land cover and land use on a national scale. These developments will help to close existing data gaps and provide information on the distributional equity of green spaces. This dissertation comprises three conceptual steps. In the first part of the study, earth observation data from the Sentinel-2 satellites are used to derive land cover information across Germany. In combination with point reference data on land cover and land use from the paneuropean Land Use and Coverage Area Frame Survey (LUCAS) a machine learning model is trained. Therein, different preprocessing steps of the LUCAS data and their influence on the classification accuracy are highlighted. The classification model derives land cover information with high accuracy even in complex urban areas. One result of the study is a Germany-wide land cover classification with an overall accuracy of 93.07 % which is used in the further course of the dissertation to spatially quantify green land cover (GLC). The second conceptual part of this study focuses on the semantic differentiation of green spaces using the example of public green spaces (PGS), which is often the subject of EJ research. A frequently used source of spatial data on public green spaces, the European Urban Atlas (EUA),however, is not available for all of Germany. This part of the study takes a data-driven approach to determine the availability of public green space at the spatial level of neighborhoods for all of Germany. For this purpose, areas already covered by the EUA serve as a reference. Using a combination of earth observation data and information from the OpenStreetMap project, a Deep Learning -based fusion network is created that quantifies the available area of public green space. The result of this step is a model that is utilized to estimate the amount of public green space in the neighborhood (𝑅 2 = 0.952). The third part of this dissertation builds upon the results of the first two parts and integrates georeferenced population data to study the socio-spatial distribution of green spaces in Germany. This exemplary analysis distinguishes green spaces according to two types: GLC and PGS. In this,first, descriptive statistics are used to examine the overall distribution of green spaces available to the German population. Then, the distributional equality is determined using established equality metrics. Finally, the relationships between the demographic composition of the neighborhood and the available amount of green space are examined using three exemplary sociodemographic groups. The analysis reveals strong differences in PGS availability between urban and rural areas. Compared to the rural population, a higher percentage of the urban population has access to the minimum level of PGS defined as a target by the World Health Organization (WHO). The results also show a clear deviation in terms of distributive equality between GLC and PGS, highlighting the relevance of distinguishing green space types for such studies. The final analysis of certain population groups addresses differences at the sociodemographic level. In summary, this dissertation demonstrates how previous limitations of spatial datasets can be overcome through a combination of modern geospatial data and machine learning methods. Using the example of green spaces in the residential environment of the population in Germany,it is shown that nationwide analyses of environmental justice can be enriched by high-resolution and locally fine-grained geographic information. This study illustrates how earth observation and methods of geoinformatics can make an important contribution to identifying inequalities in people’s living environment. Such objective information can ultimately be deployed to support and monitor sustainable urban development.

It has been a while that computers have replaced our record books, and medical records are no exception. Electronic Health Records (EHR) are digital version of a patient's medical records. EHRs are available to authorized users, and they contain the medical records of the patient, which should help doctors understand a patient's condition quickly. In recent years, Deep Learning models have proved their value and have become state-of-the-art in computer vision, natural language processing, speech and other areas. The private nature of EHR data has prevented public access to EHR datasets. There are many obstacles to create a deep learning model with EHR data. Because EHR data are primarily consisting of huge sparse matrices, these challenges are mostly unique to this field. Due to this, research in this area is limited, and we can improve existing research substantially. In this study, we focus on high-performance synthetic data generation in EHR datasets. Artificial data generation can help reduce privacy leakage for dataset owners as it is proven that de-identification methods are prone to re-identification attacks. We propose a novel approach we call Improved Correlation Capturing Wasserstein Generative Adversarial Network (SCorGAN) to create EHR data. This work, leverages Deep Convolutional Neural Networks to extract and understand spatial dependencies in EHR data. To improve our model's performance, we focus on our Deep Convolutional AutoEncoder to better map our real EHR data to our latent space where we train the Generator. To assess our model's performance, we demonstrate that our generative model can create excellent data that are statistically close to the input dataset. Additionally, we evaluate our synthetic dataset against the original data using our previous work that focused on GAN Performance Evaluation. This work is publicly available at https://github.com/mohibeyki/SCorGAN / Master of Science / Artificial Intelligence (AI) systems have improved greatly in recent years. They are being used to understand all kinds of data. A practical use case for AI systems is to leverage their power to identify illnesses and find correlations between different conditions. To train AI and Machine Learning systems, we need to feed them huge datasets, and in the training process, we need to guide them so that they learn different features in our data. The more data an intelligent system has seen, the better it performs. However, health records are private, and we cannot share real people's health records with the public, whether they are a researcher or not. This study provides a novel approach to synthetic data generation that others can use with intelligent systems. Then these systems can work with actual health records can give us accurate feedback on people's health conditions. We then show that our synthetic dataset is a good substitute for real datasets to train intelligent systems. Lastly, we present an intelligent system that we have trained using synthetic datasets to identify illnesses in a real dataset with high accuracy and precision.

This work addresses the problem of automatically detecting human skin in images without reliance on color information. Unlike previous methods, we present a new approach that performs well in the absence of such information. A key aspect of the work is that color-space augmentation is applied strategically during the training, with the goal of reducing the influence of features that are based entirely on color and increasing more semantic understanding. The resulting system exhibits a dramatic improvement in performance for images in which color details are diminished. We have demonstrated the concept using the U-Net architecture, and experimental results show improvements in evaluations for all Fitzpatrick skin tones in the ECU dataset. We further tested the system with RFW dataset to show that the proposed method is consistent across different ethnicities and reduces bias to any skin tones. Therefore, this work has strong potential to aid in mitigating bias in automated systems that can be applied to many applications including surveillance and biometrics. / Master of Science / Skin segmentation deals with the classification of skin and non-skin pixels and regions in a image containing these information. Although most previous skin-detection methods have used color cues almost exclusively, they are vulnerable to external factors (e.g., poor or unnatural illumination and skin tones). In this work, we present a new approach based on U-Net that performs well in the absence of color information. To be specific, we apply a new color space augmentation into the training stage to improve the performance of skin segmentation system over the illumination and skin tone diverse. The system was trained and tested with both original and color changed ECU dataset. We also test our system with RFW dataset, a larger dataset with four human races with different skin tones. The experimental results show improvements in evaluations for skin tones and complex illuminations.

This Ph.D. thesis presents advancements in the analysis of radar sounder data. Radar sounders (RSs) are remote sensors that transmit an electromagnetic (EM) wave at the nadir direction that penetrates the subsurface. The backscattered echoes captured by the RS antenna are coherently summed to generate an image of the subsurface profile known as a radargram. The first focus of this work is to automate the segmentation of radargrams using deep learning methodologies while minimizing the need for labeled training data. The surge in radar sounding data volume necessitates efficient automated methods. However, the amount of training labeled data in this field is strongly limited. This first work introduces a transfer learning framework based on deep learning tailored for radar sounder data that minimizes the training data requirements. This method automatically identifies and segments geological units within radargrams acquired in the cryosphere. With the cryosphere being a critical indicator of climate change, understanding its dynamics is paramount. Geological details within radargrams, such as the basal interface or the inland and floating ice, are key to this understanding. Our work shifts the focus to uncharted territory: the coastal areas of Antarctica. Novel targets such as floating ice and crevasses add complexity to the data, but the transfer learning framework minimizes the need for extensive labeled training data. The results, based on data from Antarctica, confirm the effectiveness of the approach, promising adaptability to other targets and radar data from existing and future planetary missions like RIME and SRS. The second focus of this thesis explores the generation of novel and improved geological data products by harnessing the unique characteristics of radar sounder data, including subsurface information and so-called “unwanted” clutter. The thesis introduces two methods that use RS data to generate geological products. The first contribution proposes a global high-frequency radar image of Mars. This product delivers a novel, comprehensive global radar image of Mars, capturing both surface and shallow subsurface structures. The method unlocks the potential to explore concealed Martian geology and further understand Martian geological features like dust, revealing possible candidate large dust deposits that were unknown until now. Furthermore, this method can potentially offer insights into celestial bodies beyond Mars, such as the detection of new lunar facets and Venusian geological formations. The third contribution aims to generate Digital Elevation Models (DEM) from single swath radargrams. The activity addresses the challenge of precise bed DEM estimations in Antarctica. Bed topography is critical in ice modeling and mass balance calculations, yet existing methods face limitations. To overcome these, we employ a generative adversarial network (GAN) approach that utilizes clutter information from single radargrams. This innovative technique promises to refine bed DEMs and enhance our understanding of glacier erosion and ice dynamics. The proposed methodologies were validated with data acquired on both Earth and Mars, showing promising results and confirming their effectiveness.

Deep learning models demonstrated good performance in various domains such as ComputerVision and Natural Language Processing. However, the utilization of data-driven methods in healthcare raises privacy concerns, which creates limitations for collaborative research. A remedy to this problem is to generate and employ synthetic data to address privacy concerns. Existing methods for artificial data generation suffer from different limitations, such as being bound to particular use cases. Furthermore, their generalizability to real-world problems is controversial regarding the uncertainties in defining and measuring key realistic characteristics. Hence, there is a need to establish insightful metrics (and to measure the validity of synthetic data), as well as quantitative criteria regarding privacy restrictions. We propose the use of Generative Adversarial Networks to help satisfy requirements for realistic characteristics and acceptable values of privacy metrics, simultaneously. The present study makes several unique contributions to synthetic data generation in the healthcare domain. First, we propose a novel domain-agnostic metric to evaluate the quality of synthetic data. Second, by utilizing 1-D Convolutional Neural Networks, we devise a new approach to capturing the correlation between adjacent diagnosis records. Third, we employ ConvolutionalAutoencoders for creating a robust and compact feature space to handle the mixture of discrete and continuous data. Finally, we devise a privacy-preserving framework that enforcesRényi differential privacy as a new notion of differential privacy. / Doctor of Philosophy / Computers programs have been widely used for clinical diagnosis but are often designed with assumptions limiting their scalability and interoperability. The recent proliferation of abundant health data, significant increases in computer processing power, and superior performance of data-driven methods enable a trending paradigm shift in healthcare technology. This involves the adoption of artificial intelligence methods, such as deep learning, to improve healthcare knowledge and practice. Despite the success in using deep learning in many different domains, in the healthcare field, privacy challenges make collaborative research difficult, as working with data-driven methods may jeopardize patients' privacy. To overcome these challenges, researchers propose to generate and utilize realistic synthetic data that can be used instead of real private data. Existing methods for artificial data generation are limited by being bound to special use cases. Furthermore, their generalizability to real-world problems is questionable. There is a need to establish valid synthetic data that overcomes privacy restrictions and functions as a real-world analog for healthcare deep learning data training. We propose the use of Generative Adversarial Networks to simultaneously overcome the realism and privacy challenges associated with healthcare data.

Society's newfound dependence on wireless transmission systems has driven demand for access to the electromagnetic (EM) spectrum to an all-time high. In particular, wireless applications related to the fifth generation (5G) of cellular technology along with statically allocated radar systems have contributed to the increasing scarcity of the sub 6 GHz frequency bands. As a result, development of Dynamic Spectrum Access (DSA) techniques for sharing these frequencies has become a critical research area for the greater wireless community. Since among incumbent systems, radars are the largest consumers of spectrum in the sub 6 GHz regime, and are being used increasingly for civilian applications such as traffic control, adaptive cruise control, and collision avoidance, the need for radars which can adaptively tune specific transmission parameters in an intelligent manner to promote coexistence with other systems has arisen. Thus, fully-aware, dynamic, cognitive radar has been proposed as target for radars to evolve towards. In this thesis, we extend current research thrusts towards cognitive radar to utilize Reinforcement Learning (RL) techniques which allow a radar system to learn desired behavior using information obtained from past transmissions. Since radar systems inherently interact with their electromagnetic environment, it is natural to view the use of reinforcement learning techniques as a straightforward extension to previous adaptive techniques. However, in designing learning algorithms for radar systems, we must carefully define goal-driven rewards, formalize the learning process, and consider an appropriate amount of environmental information. In this thesis, we apply well-established and emerging reinforcement learning approaches to meet the demands of modern radar coexistence problems. In particular, function estimation using deep neural networks is examined, as Deep RL presents a scalable learning framework which allows many environmental states to be considered in the decision-making process. We then show how these techniques can be used to improve traditional radar performance metrics, such as interference avoidance, spectral efficiency, and target detectibility with simulated and experimental results. We also compare the learning techniques to each other and naive approaches, such as fixed bandwidth radar and avoiding interference reactively. Finally, online learning strategies are considered which aim to balance the fundamental learning trade-off between exploration and exploitation. We show that online learning techniques can be used to select individual waveforms or applied as a high-level controller in a hierarchical learning scheme based on the biologically inspired concept of metacognition. The general use of RL techniques provides a robust framework for decision making under uncertainty that is more flexible than previously proposed cognitive radar strategies. Further, the wide array of RL models and algorithms allow the underlying structure to be applied to both small and large-scale radar scenarios. / Master of Science / Society's newfound dependence on wireless transmission systems has driven demand for control of the electromagnetic (EM) spectrum to an all-time high. In particular, federal spectrum auctions and the fifth generation of wireless technologies have contributed to the scarcity of frequency bands below 6GHz. These frequencies are widely used by both radar and communications systems due to favorable propagation characteristics. However, current radar systems typically occupy a fixed bandwidth and are tend to be poorly equipped to share their allocated spectrum with other users, which has become a necessity given the growth of wireless traffic. In this thesis, we study learning algorithms which enable a radar to optimize its electromagnetic pulses based on feedback from received signals. In particular, we are interested in reinforcement learning algorithms which allow a radar to learn optimal behavior based on rewards defined by a human. Using these algorithms, radar system designers can choose which metrics may be most important for a given radar application which can then be optimized for the given setting. However, scaling reinforcement learning to real-world problems such as radar optimization is often difficult due to the massive scope of the problem. Here we attempt to identify potential issues with implementation of each algorithm and narrow in on algorithms that are well-suited for real-time radar operation.

Building energy load forecasting is becoming an increasingly important task with the rapid deployment of smart homes, integration of renewables into the grid and the advent of decentralized energy systems. Residential load forecasting has been a challenging task since the residential load is highly stochastic. Deep learning models have showed tremendous promise in the fields of time-series and sequential data and have been successfully used in the field of short-term load forecasting at the building level. Although, other studies have looked at using deep learning models for building energy forecasting, most of those studies have looked at limited number of homes or an aggregate load of a collection of homes. This study aims to address this gap and serve as an investigation on selecting the better deep learning model architecture for short term load forecasting on 3 communities of residential buildings. The deep learning models CNN and LSTM have been used in the study. For 15-min ahead forecasting for a collection of homes it was found that homes with a higher variance were better predicted by using CNN models and LSTM showed better performance for homes with lower variances. The effect of adding weather variables on 24-hour ahead forecasting was studied and it was observed that adding weather parameters did not show an improvement in forecasting performance. In all the homes, deep learning models are shown to outperform the simple ANN model. / Master of Science / Building energy load forecasting is becoming an increasingly important task with the rapid deployment of smart homes, integration of renewables into the grid and the advent of decentralized energy systems. Residential load forecasting has been a challenging task since residential load is highly stochastic. Deep learning models have showed tremendous promise in the fields of time-series and sequential data and have been successfully used in the field of short-term load forecasting. Although, other studies have looked at using deep learning models for building energy forecasting, most of those studies have looked at only a single home or an aggregate load of a collection of homes. This study aims to address this gap and serve as an analysis on short term load forecasting on 3 communities of residential buildings. Detailed analysis on the model performances across all homes have been studied. Deep learning models have been used in this study and their efficacy is measured compared to a simple ANN model.

Melanoma is the most deadly form of skin cancer worldwide, which causes the 75% of deaths related to skin cancer. National Cancer Institute estimated that 91,270 new case and 9,320 deaths are expected in 2018 caused by melanoma. Early detection of melanoma is the key for the treatment. The image technique to diagnose skin cancer is dermoscopy, which leads to improved diagnose accuracy compared to traditional ABCD criteria. But reading and examining dermoscopic images is a time-consuming and complex process. Therefore, computerized analysis methods of dermoscopic images have been developed to assist the visual interpretation of dermoscopic images. The automatic segmentation of skin lesion attributes is a key step in computerized analysis of dermoscopic images. The International Skin Imaging Collaboration (ISIC) hosted the 2018 Challenges to help the diagnosis of melanoma based on dermoscopic images. In this thesis, I develop a deep learning based approach to automatically segment the attributes from dermoscopic skin lesion images. The approach described in the thesis achieved the Jaccard index of 0.477 on the official test dataset, which ranked 5th place in the challenge. / Master of Science / Melanoma is the most deadly form of skin cancer worldwide, which causes the 75% of deaths related to skin cancer. Early detection of melanoma is the key for the treatment. The image technique to diagnose skin cancer is called dermoscopy. It has become increasingly conveniently to use dermoscopic device to image the skin in recent years. Dermoscopic lens are available in the market for individual customer. When coupling the dermoscopic lens with smartphones, people are be able to take dermoscopic images of their skin even at home. However, reading and examining dermoscopic images is a time-consuming and complex process. It requires specialists to examine the image, extract the features, and compare with criteria to make clinical diagnosis. The time-consuming image examination process becomes the bottleneck of fast diagnosis of melanoma. Therefore, computerized analysis methods of dermoscopic images have been developed to promote the melanoma diagnosis and to increase the survival rate and save lives eventually. The automatic segmentation of skin lesion attributes is a key step in computerized analysis of dermoscopic images. In this thesis, I developed a deep learning based approach to automatically segment the attributes from dermoscopic skin lesion images. The segmentation result from this approach won 5th place in a public competition. It has the potential to be utilized in clinic application in the future.

The ability to understand occlusion and disocclusion is critical in analyzing motion and forecasting changes. For example, when we see a car gradually blocks our view of a human figure, we know that either the car or the human is moving. We also know that the human behind the car will be visible again if we move to other positions. As many vision-based intelligent systems need to handle and react to visual data with potentially intensive motions, it is therefore beneficial to incorporate the occlusion reasoning into such systems. In this thesis, we study how we can improve the performance of vision-based deep learning models by harnessing the power of occlusion handling. We first visit the problem of optical flow estimation for motion analysis. We present a deep learning module that builds upon occlusion handling methods in classic Computer Vision literature. Our results show performance improvement in occluded regions on standard benchmarks, as well as real-world applications. We then examine the problem of view synthesis for 3D photography. We propose an inpainting method that leverages local color and depth context for novel view synthesis. We validate the proposed inpainting approach with a series of quantitative and qualitative experiments, and demonstrate promising results in predicting plausible content in occluded regions. / Master of Science / Human has the ability to understand occlusion, and make use of such knowledge to make predictions about motions and occluded contents. For example, when we see a car gradually blocks our view of a human figure, we know that either the car or the human is moving. We also know that the human behind the car will be visible again if we move to other positions. In this thesis, we study how we can replicate such an ability to artificial intelligence systems. We first investigate the effect of occlusion reasoning in the task of predicting motion. Our experimental results show that a system equipped with our occlusion reasoning module can better capture the motions happening in image sequences. Next, we examine the problem of hallucinating visual contents that are blocked in an image. We develop a model that can produce plausible content in occluded regions. In our experiments, we show that given one single RGB image with an estimated depth map, our model can produce a corresponding 3D photo by hallucinating the structures that are not visible in the image.