Les activités basées sur l'exploitation de données vidéo se sont développées de manière fulgurante ces dernières années : nous assisté à une démocratisation de certaines de ces activités (vidéo-surveillance) mais également à une diversification importante des applications opérationnelles (suivi de ressources naturelles, reconnaissance etc). Cependant, le volume de données vidéo généré est aujourd'hui astronomique et l'efficacité de ces activités est limitée par le coût et la durée nécessaire à l'interprétation humaine des données vidéo. L'analyse automatique de flux vidéos est donc devenue une problématique cruciale pour de nombreuses applications. L'approche semi-automatique développée dans le cadre de cette thèse se concentre plus spécifiquement sur l'analyse de vidéos aériennes, et permet d'assister l'analyste image dans sa tâche en suggérant des zones d'intérêt potentiel par détection de changements. Pour cela, nous effectuons une modélisation tridimensionnelle des apparences observées dans les vidéos de référence. Cette modélisation permet ensuite d'effectuer une détection en ligne des changements significatifs dans une nouvelle vidéo, en identifiant les déviations d'apparence par rapport aux modèles de référence. Des techniques spécifiques ont également été proposées pour effectuer l'estimation des paramètres d'acquisition ainsi que l'atténuation des effets de l'illumination. De plus, nous avons développé plusieurs techniques de consolidation permettant d'exploiter la connaissance a priori relative aux changements à détecter. L'intérêt et les bonnes performances de notre approche a été minutieusement démontré à l'aide de données réelles et synthétiques. / Business activities based on the use of video data have developed at a dazzling speed these last few years: not only has the market of some of these activities widely expanded (video-surveillance) but the operational applications have also greatly diversified (natural resources monitoring, intelligence etc). However, nowadays, the volume of generated data has become overwhelming and the efficiency of these activities is now limited by the cost and the time required by the human interpretation of this video data. Automatic analysis of video streams has hence become a critical problem for numerous applications. The semi-autmoatic approach developed in this thesis focuses more specifically on the automatic analysis of aerial videos and enables assisting the image analyst in his task by suggesting areas of potential interest identified using change detection. For that purpose, our approach proceeds to a tridimensional modeling of the appearances observed in the reference videos. Such a modeling then enables the online detection of significant changes in a new video, by identifying appearance deviations with respect to the reference models. Specific techniques have also been developed to estimate the acquisition parameters and to attenuate illumination effects. Moreover, we developed several consolidation techniques making use of a priori knowledge related to targeted changes, in order to improve detection accuracy. The interest and good performance of our change detection approach has been carefully demonstrated using both real and synthetical data.

Nous étudions ici l’intérêt des descripteurs locaux pour les images satellites optiques et radar. Ces descripteurs, par leurs invariances et leur représentation compacte, offrent un intérêt pour la comparaison d’images acquises dans des conditions différentes. Facilement applicables aux images optiques, ils offrent des performances limitées sur les images radar, en raison de leur fort bruit multiplicatif. Nous proposons ici un descripteur original pour la comparaison d’images radar. Cet algorithme, appelé SAR-SIFT, repose sur la même structure que l’algorithme SIFT (détection de points-clés et extraction de descripteurs) et offre des performances supérieures pour les images radar. Pour adapter ces étapes au bruit multiplicatif, nous avons développé un opérateur différentiel, le Gradient par Ratio, permettant de calculer une norme et une orientation du gradient robustes à ce type de bruit. Cet opérateur nous a permis de modifier les étapes de l’algorithme SIFT. Nous présentons aussi deux applications pour la télédétection basées sur les descripteurs. En premier, nous estimons une transformation globale entre deux images radar à l’aide de SAR-SIFT. L’estimation est réalisée à l’aide d’un algorithme RANSAC et en utilisant comme points homologues les points-clés mis en correspondance. Enfin nous avons mené une étude prospective sur l’utilisation des descripteurs pour la détection de changements en télédétection. La méthode proposée compare les densités de points-clés mis en correspondance aux densités de points-clés détectés pour mettre en évidence les zones de changement. / We study here the interest of local features for optical and SAR images. These features, because of their invariances and their dense representation, offer a real interest for the comparison of satellite images acquired under different conditions. While it is easy to apply them to optical images, they offer limited performances on SAR images, because of their multiplicative noise. We propose here an original feature for the comparison of SAR images. This algorithm, called SAR-SIFT, relies on the same structure as the SIFT algorithm (detection of keypoints and extraction of features) and offers better performances for SAR images. To adapt these steps to multiplicative noise, we have developed a differential operator, the Gradient by Ratio, allowing to compute a magnitude and an orientation of the gradient robust to this type of noise. This operator allows us to modify the steps of the SIFT algorithm. We present also two applications for remote sensing based on local features. First, we estimate a global transformation between two SAR images with help of SAR-SIFT. The estimation is realized with help of a RANSAC algorithm and by using the matched keypoints as tie points. Finally, we have led a prospective study on the use of local features for change detection in remote sensing. The proposed method consists in comparing the densities of matched keypoints to the densities of detected keypoints, in order to point out changed areas.

La tâche de segmentation et de regroupement en locuteurs (speaker diarization) consiste à identifier "qui parle quand" dans un flux audio sans connaissance a priori du nombre de locuteurs ou de leur temps de parole respectifs. Les systèmes de segmentation et de regroupement en locuteurs sont généralement construits en combinant quatre étapes principales. Premièrement, les régions ne contenant pas de parole telles que les silences, la musique et le bruit sont supprimées par la détection d'activité vocale (VAD). Ensuite, les régions de parole sont divisées en segments homogènes en locuteur par détection des changements de locuteurs, puis regroupées en fonction de l'identité du locuteur. Enfin, les frontières des tours de parole et leurs étiquettes sont affinées avec une étape de re-segmentation. Dans cette thèse, nous proposons d'aborder ces quatre étapes avec des approches fondées sur les réseaux de neurones. Nous formulons d’abord le problème de la segmentation initiale (détection de l’activité vocale et des changements entre locuteurs) et de la re-segmentation finale sous la forme d’un ensemble de problèmes d’étiquetage de séquence, puis nous les résolvons avec des réseaux neuronaux récurrents de type Bi-LSTM (Bidirectional Long Short-Term Memory). Au stade du regroupement des régions de parole, nous proposons d’utiliser l'algorithme de propagation d'affinité à partir de plongements neuronaux de ces tours de parole dans l'espace vectoriel des locuteurs. Des expériences sur un jeu de données télévisées montrent que le regroupement par propagation d'affinité est plus approprié que le regroupement hiérarchique agglomératif lorsqu'il est appliqué à des plongements neuronaux de locuteurs. La segmentation basée sur les réseaux récurrents et la propagation d'affinité sont également combinées et optimisées conjointement pour former une chaîne de regroupement en locuteurs. Comparé à un système dont les modules sont optimisés indépendamment, la nouvelle chaîne de traitements apporte une amélioration significative. De plus, nous proposons d’améliorer l'estimation de la matrice de similarité par des réseaux neuronaux récurrents, puis d’appliquer un partitionnement spectral à partir de cette matrice de similarité améliorée. Le système proposé atteint des performances à l'état de l'art sur la base de données de conversation téléphonique CALLHOME. Enfin, nous formulons le regroupement des tours de parole en mode séquentiel sous la forme d'une tâche supervisée d’étiquetage de séquence et abordons ce problème avec des réseaux récurrents empilés. Pour mieux comprendre le comportement du système, une analyse basée sur une architecture de codeur-décodeur est proposée. Sur des exemples synthétiques, nos systèmes apportent une amélioration significative par rapport aux méthodes de regroupement traditionnelles. / Speaker diarization is the task of determining "who speaks when" in an audio stream that usually contains an unknown amount of speech from an unknown number of speakers. Speaker diarization systems are usually built as the combination of four main stages. First, non-speech regions such as silence, music, and noise are removed by Voice Activity Detection (VAD). Next, speech regions are split into speaker-homogeneous segments by Speaker Change Detection (SCD), later grouped according to the identity of the speaker thanks to unsupervised clustering approaches. Finally, speech turn boundaries and labels are (optionally) refined with a re-segmentation stage. In this thesis, we propose to address these four stages with neural network approaches. We first formulate both the initial segmentation (voice activity detection and speaker change detection) and the final re-segmentation as a set of sequence labeling problems and then address them with Bidirectional Long Short-Term Memory (Bi-LSTM) networks. In the speech turn clustering stage, we propose to use affinity propagation on top of neural speaker embeddings. Experiments on a broadcast TV dataset show that affinity propagation clustering is more suitable than hierarchical agglomerative clustering when applied to neural speaker embeddings. The LSTM-based segmentation and affinity propagation clustering are also combined and jointly optimized to form a speaker diarization pipeline. Compared to the pipeline with independently optimized modules, the new pipeline brings a significant improvement. In addition, we propose to improve the similarity matrix by bidirectional LSTM and then apply spectral clustering on top of the improved similarity matrix. The proposed system achieves state-of-the-art performance in the CALLHOME telephone conversation dataset. Finally, we formulate sequential clustering as a supervised sequence labeling task and address it with stacked RNNs. To better understand its behavior, the analysis is based on a proposed encoder-decoder architecture. Our proposed systems bring a significant improvement compared with traditional clustering methods on toy examples.

The problem of whether Landsat 7 data could be used to delineate areas prone to mass movement, particularly debris flows and landslides, was examined using three techniques: change detection in NDVI (Normalized Difference Vegetation Index), change detection in band 5, and the tasseled cap transformation. These techniques were applied to areas that had recently experienced mass movement: Layton, Davis County and Alpine, Spanish Fork Canyon and Santaquin, Utah County. No distinctive spectral characteristics were found with any of these techniques with two possible explanations: 1. That despite improved spatial resolution in Landat 7 over its predecessors and improved digital image processing capabilities, the resolution is still too low to detect these characteristics or 2. That the aspects of a slope that make it prone to mass movement are undetectable at any resolution by remote sensing. Change detection in NDVI examined if areas that remained unchanged (defined as < 5% change) between August 14, 1999 and October 17, 1999 correlated to areas that are prone to mass movement. There was no correlation. Change detection in band 5 was examined between August 14, 1999 and October 17, 1999, October 17, 1999 and May 28, 2000, and August 14, 1999 and May 28, 2000. An interesting result is that the Shurtz Lake and Thistle landslides (Spanish Fork Canyon) showed changes of greater than 30% during August 14, 1999 - October 17, 1999 and October 17, 1999 - May 28, 2000. These changes were limited to these landslides and not seen in abundance in surrounding areas. A similar localization of 30% change was seen in the Cedar Bench landslide (Layton) for the same time periods. There were no other correlations. The tasseled cap ransformation shows areas of dominate greenness, soil brightness or wetness. None of these factors had distinctive patterns in the areas studied when compared to surrounding, mass movement-prone areas so no conclusions can be drawn about the utility of the tasseled cap transformation as it relates to areas of potential mass movement.

Interferometry and coherent change detection (CCD) utilize phase differences between complex SAR images to find terrain height and to detect small changes between images, respectively. A new method for improving interferometry and CCD using multiple sub-apertures is proposed. Using backprojection processing, multiple sub-aperture images are created for a pair of flights. An interferogram and coherence map is made from each sub-aperture. For CCD, each sub-aperture coherence map offers an independent estimate of the coherence over the same area. By combining coherence maps, low coherence areas associated with residual motion errors are reduced, shadowed areas are minimized, and the overall coherence of stationary objects between images is increased. For interferometry, combining independent estimates of a scene's height offers a more accurate height estimate. For repeat-pass interferometry, multiple apertures are shown to increase the coverage of valid height estimates. The benefits of multi-aperture interferometry and CCD are shown using examples with real data.

[pt] Os dados de observação da Terra são freqüentemente afetados pelo fenômeno de mudança de domínio. Mudanças nas condições ambientais, variabilidade geográfica e diferentes propriedades de sensores geralmente tornam quase impossível empregar classificadores previamente treinados para novos dados sem experimentar uma queda significativa na precisão da classificação. As técnicas de adaptação de domínio baseadas em modelos de aprendizado profundo têm se mostrado úteis para aliviar o problema da mudança de domínio. Trabalhos recentes nesta área fundamentam-se no treinamento adversárial para alinhar os atributos extraídos de imagens de diferentes domínios em um espaço latente comum. Outra forma de tratar o problema é empregar técnicas de translação de imagens e adaptá-las de um domínio para outro de forma que as imagens transformadas contenham características semelhantes às imagens do outro domínio. Neste trabalho, propõem-se abordagens de adaptação de domínio para tarefas de detecção de mudanças, baseadas em primeiro lugar numa técnica de traslação de imagens, Cycle-Consistent Generative Adversarial Network (CycleGAN), e em segundo lugar, num modelo de alinhamento de atributos: a Domain Adversarial Neural Network (DANN). Particularmente, tais técnicas foram estendidas, introduzindo-se restrições adicionais na fase de treinamento dos componentes do modelo CycleGAN, bem como um procedimento de pseudo-rotulagem não supervisionado para mitigar o impacto negativo do desequilíbrio de classes no DANN. As abordagens propostas foram avaliadas numa aplicação de detecção de desmatamento, considerando diferentes regiões na floresta amazônica e no Cerrado brasileiro (savana). Nos experimentos, cada região corresponde a um domínio, e a precisão de um classificador treinado com imagens e referências de um dos domínio (fonte) é medida na classificação de outro domínio (destino). Os resultados demonstram que as abordagens propostas foram bem sucedidas em amenizar o problema de desvio de domínio no contexto da aplicação alvo. / [en] Earth observation data are frequently affected by the domain shift phenomenon. Changes in environmental conditions, geographical variability and different sensor properties typically make it almost impossible to employ previously trained classifiers for new data without a significant drop in classification accuracy. Domain adaptation (DA) techniques based on Deep Learning models have been proven useful to alleviate domain shift. Recent improvements in DA technology rely on adversarial training to align features extracted from images of the different domains in a common latent space. Another way to face the problem is to employ image translation techniques, and adapt images from one domain in such a way that the transformed images contain characteristics that are similar to the images from the other domain. In this work two different DA approaches for change detection tasks are proposed, which are based on a particular image translation technique, the Cycle-Consistent Generative Adversarial Network (CycleGAN), and on a representation matching strategy, the Domain Adversarial Neural Network (DANN). In particular, additional constraints in the training phase of the original CycleGAN model components are proposed, as well as an unsupervised pseudo-labeling procedure, to mitigate the negative impact of class imbalance in the DANN-based approach. The proposed approaches were evaluated on a deforestation detection application, considering different sites in the Amazon rain-forest and in the Brazilian Cerrado (savanna) biomes. In the experiments each site corresponds to a domain, and the accuracy of a classifier trained with images and references from one (source) domain is measured in the classification of another (target) domain. The experimental results show that the proposed approaches are successful in alleviating the domain shift problem.

As the global population continues to grow, urbanization is one of the most significant anthropogenic processes linked to ecological change. But even in countries where the overall population is stagnating, migratory movements toward urban centres will continue to place pressure on the finite resource of land. Therefore, it is particularly important to determine and describe the development of settlement areas as precisely as possible in order to inform spatial planning decisions. For this reason, this dissertation presents vector-based methods to analyse, quantify and evaluate small-scale changes in settlement area. In this work, which constitutes a cumulative dissertation, novel methods are described that can be used to determine not only areal change in settlement and traffic areas (SuV), but also the type of building change and urban densification. This is of particular interest for the spatial planning of expanding metropolitan areas, where the question arises: Where, how and to which extent can built-up areas be further densified in order to reduce the consumption of land for new settlement areas? The methods presented here can facilitate spatially detailed analyses and already form the basis for a nationwide monitoring of settlement and open space development. This work shows how geometric deviations and changes in the underlying data model can be taken into account when determining SuV growth from data of the Authoritative Topographic-Cartographic Information System (ATKIS). In this context, positional inaccuracies of linearly and arealy modelled geometries are each treated in a special way so that minor positional offsets no longer affect the SuV increase. In addition, changes in the data model are accommodated by disregarding specific object reallocations when determining the SuV increase. To test these methods, the SuV increase was determined and analysed for Germany using national ATKIS data sets that feature geometric positional inaccuracies and data model changes. It could be shown that a considerable share of the calculated SuV increase is not due to real-world changes but to modelling issues. Furthermore, a novel method for the detection of building changes is presented, which focuses on the differentiation between modified and replaced buildings. It could be shown that this new approach is more accurate than other investigated methods. Furthermore, an algorithm was developed in this work to generate defined location deviations. This could be used to show how position deviations affect the accuracy of the examined procedures. The threshold values determined in this work can form the basis for similar investigations. In addition, an indicator was developed to track changes in building density. This indicator not only reflects the extent of building change but also the size of the existing building stock. Moreover, the indicator was designed in such a way as to allow comparison of the densification of developed and undeveloped areas, and thus also inner and outer urban areas. Furthermore, the indicator can be used to symmetrically calculate a decrease in the building stock, enabling a comparison of densification and de-densification processes.:1. Introduction 1.1 Motivation 1.2 Problem description 1.3 Aims 1.4 Structure 2. Dissertation main articles 2.1 Measuring land take in Germany 2.2 Detecting building change 2.3 Indicator for building densification 3. Methods for measuring settlement changes 3.1 Measuring changes through land use data 3.2 Detection of building changes 3.3 Measuring changes in building density 4. Main findings 4.1 Effects of non-real changes on land take 4.2 Distinguishing building modification and replacement 4.3 Impact of building changes on building density 4.4 How the articles are connected 4.5 Additional relevant publications 5. Conclusion and Outlook References Abbreviations List of figures List of author’s publications Articles Conference Papers Acknowledgments Appendix with publications

Anthropogenic climate and land use change is causing rapid shifts in the distribution and composition of habitats with profound impacts on ecosystem biodiversity. The sustainable management of ecosystems requires monitoring programmes capable of detecting shifts in habitat distribution and composition at large spatial scales. Remote sensing observations facilitate such efforts as they enable cost-efficient modelling approaches that utilize publicly available datasets and can assess the status of habitats over extended periods of time. In this study, we introduce a modelling framework for habitat monitoring in Germany using readily available MODIS surface reflectance data. We developed supervised classification models that allocate (semi-)natural areas to one of 18 classes based on their similarity to Natura 2000 habitat types. Three machine learning classifiers, i.e., Support Vector Machines (SVM), Random Forests (RF), and C5.0, and an ensemble approach were employed to predict habitat type using spectral signatures from MODIS in the visible-to-near-infrared and short-wave infrared. The models were trained on homogenous Special Areas of Conservation that are predominantly covered by a single habitat type with reference data from 2013, 2014, and 2016 and tested against ground truth data from 2010 and 2019 for independent model validation. Individually, the SVM and RF methods achieved better overall classification accuracies (SVM: 0.72–0.93%, RF: 0.72–0.94%) than the C5.0 algorithm (0.66–0.93%), while the ensemble classifier developed from the individual models gave the best performance with overall accuracies of 94.23% for 2010 and 80.34% for 2019 and also allowed a robust detection of non-classifiable pixels. We detected strong variability in the cover of individual habitat types, which were reduced when aggregated based on their similarity. Our methodology is capable to provide quantitative information on the spatial distribution of habitats, differentiate between disturbance events and gradual shifts in ecosystem composition, and could successfully allocate natural areas to Natura 2000 habitat types.

Change detection using satellite imagery has applications in urban development, disaster response and precision agriculture. Current deep learning models show promising results. However, on-board computers are typically highly constrained which poses a challenge for deployment. On-board processing is desirable for saving bandwidth by downlinking only novel and valuable data. The goal of this work is to determine what change detection models are most technically feasible for on-board use in satellites. The novel patch based model MobileGoNogo is evaluated along current state-of-the-art models. Technical feasibility was determined by observing accuracy, inference time, storage buildup, memory usage and resolution on a satellite computer tasked with detecting changes in buildings from the SpaceNet 7 dataset. Three high level approaches were taken; direct classification, post classification and patch-based change detection. None of the models compared in the study fulfilled all requirements for general technical feasibility. Direct classification models were highly resource intensive and slow. Post classification model had critically low accuracy but desirable storage characteristics. Patch based MobileGoNogo performed better by all metrics except in resolution where it is significantly lower than any other model. We conclude that the novel model offers a feasible solution for low resolution, noncritical applications. / Upptäckt av förändringar med hjälp av satellitbilder har tillämpningar inom bl.a. stadsutveckling, katastrofinsatser och precisionsjordbruk. De nuvarande modellerna för djupinlärning visar lovande resultat. Datorerna ombord satelliter är dock vanligtvis mycket begränsade, vilket innebär en utmaning för användningen av dessa modeller. Databehandling ombord är önskvärd för att spara bandbredd genom att endast skicka ner nya och värdefulla data. Målet med detta arbete är att fastställa vilka modeller för upptäckt av förändringar som är mest tekniskt genomförbara för användning ombord på satelliter. Den nya bildfältbaserade modellen MobileGoNogo utvärderas tillsammans med de senaste modellerna. Den tekniska genomförbarheten fastställdes genom att observera träffsäkerhet, inferenstid, lagring, minnesanvändning och upplösning på en satellitdator med uppgift att upptäcka förändringar i byggnader från SpaceNet 7dataset. Tre tillvägagångssätt på hög nivå användes: direkt klassificering, postklassificering och fältbaserad klassificering. Ingen av de modeller som jämfördes i studien uppfyllde alla krav på allmän teknisk genomförbarhet. Direkta klassificeringsmodeller var mycket resurskrävande och långsamma. Postklassificeringsmodellen hade kritiskt låg träffsäkerhet men önskvärda lagringsegenskaper. Den bildfältbaserade MobileGoNogo-modellen var bättre i alla mätvärden utom i upplösningen, där den var betydligt lägre än någon annan modell. Vi drar slutsatsen att den nya modellen erbjuder en genomförbar lösning för icke-kritiska tillämpningar med låg upplösning.

No description available.