The advancement of computer architecture and chip design has set the stage for the deep learning revolution by supplying enormous computational power. In general, deep learning is built upon neural networks that can be regarded as a universal approximator of any mathematical function. In contrast to model-based machine learning where the representative features are designed by human engineers, deep learning enables the automatic discovery of desirable feature representations based on a data-driven manner. In this thesis, the applications of deep learning to visual content processing and analysis are discussed. For visual content processing, two novel approaches, named LCVSR and RawVSR, are proposed to address the common issues in the filed of Video Super-Resolution (VSR). In LCVSR, a new mechanism based on local dynamic filters via Locally Connected (LC) layers is proposed to implicitly estimate and compensate motions. It avoids the errors caused by the inaccurate explicit estimation of flow maps. Moreover, a global refinement network is proposed to exploit non-local correlations and enhance the spatial consistency of super-resolved frames. In RawVSR, the superiority of camera raw data (where the primitive radiance information is recorded) is harnessed to benefit the reconstruction of High-Resolution (HR) frames. The developed network is in line with the real imaging pipeline, where the super-resolution process serves as a pre-processing unit of ISP. Moreover, a Successive Deep Inference (SDI) module is designed in accordance with the architectural principle suggested by a canonical decomposition result for Hidden Markov Model (HMM) inference, and a reconstruction module is built with elaborately designed Attention based Residual Dense Blocks (ARDBs). For visual content analysis, a new approach, named PSCC-Net, is proposed to detect and localize image manipulations. It consists of two paths: a top-down path that extracts the local and global features from an input image, and a bottom-up path that first distinguishes manipulated images from pristine ones via a detection head, and then localizes forged regions via a progressive mechanism, where manipulation masks are estimated from small scales to large ones, each serving as a prior of the next-scale estimation. Moreover, a Spatio-Channel Correlation Module (SCCM) is proposed to capture both spatial and channel-wise correlations among extracted features, enabling the network to cope with a wide range of manipulation attacks. Extensive experiments validate that the proposed methods in this thesis have achieved the SOTA results and partially addressed the existing issues in previous works. / Dissertation / Doctor of Philosophy (PhD)

In this thesis, a computationally efficient method for detecting a whistle and capturing it using a 4 microphone array is proposed. Furthermore, methods are developed to efficiently process the data captured from all the microphones to estimate the direction of the sound source. The accuracy, the shortcoming and the constraints of the method proposed are also discussed. There is an emphasis placed on being computationally efficient so that the methods may be implemented on a low cost microcontroller and be used to provide a heading to an Unmanned Ground Vehicle. / MS / As humans, we rely on our sense of hearing to help us interact with the outside world. It helps us to listen not just to other people but also for sounds that maybe a warning for us. It can often be the first warning we get of an impending danger as we might hear a predator before we see it or we might hear a car brake and slip before we turn to look at it. However, it is not merely the ability to hear a sound that makes hearing so useful. It is the fact that we can tell which direction the sound is coming from that makes it so important. That is what allows us to know which direction to turn towards to respond to someone or from which direction the sound warning us of danger is coming. We may not be able to pinpoint the location of the source with complete accuracy but we can discern the general heading. It was this idea that inspired this research work. We wanted to be capable of estimating where a sound is coming from while being computationally efficient so that it may be implemented in real time with the help of a low cost microcontroller. This would then be used to provide a heading to an Unmanned Ground Vehicle while keeping the costs down.

A massive amount of data is being generated at an unprecedented level from a diversity of sources, including social media, internet services, biological studies, physical infrastructure monitoring and many others. The necessity of analyzing such complex data has led to the birth of an emerging framework, graph signal processing. This framework offers an unified and mathematically rigorous paradigm for the analysis of high-dimensional data with complex and irregular structure. It extends fundamental signal processing concepts such as signals, Fourier transform, frequency response and filtering, from signals residing on regular lattices, which have been studied by the classical signal processing theory, to data residing on general graphs, which are called graph signals. In this thesis, we consider five fundamental tasks on graphs from the perspective of graph signal processing: representation, sampling, recovery, detection and localization. Representation, aiming to concisely model shapes of graph signals, is at the heart of the proposed techniques. Sampling followed by recovery, aiming to reconstruct an original graph signal from a few selected samples, is applicable in semi-supervised learning and user profiling in online social networks. Detection followed by localization, aiming to identify and localize targeted patterns in noisy graph signals, is related to many real-world applications, such as localizing virus attacks in cyber-physical systems, localizing stimuli in brain connectivity networks, and mining traffic events in city street networks, to name just a few. We illustrate the power of the proposed tools on two real-world problems: fast resampling of 3D point clouds and mining of urban traffic data.

Tato diplomová práce se zabývá využitím neuronových sítí pro detekci obličeje v obraze. Práce poskytuje základní informace nezbytné pro pochopení detekce obličejů a neuronových sítí. Dále se věnuje současným nejúspěšnějším detektorům, především detektorům založeným na neuronových sítích. Detailně je pak popsán detektor, který navrhl Rowley. Z tohoto detektoru moje práce ve velké míře čerpá. Dále je popsána implementace tohoto detektoru společně s navrženými zlepšeními a jsou prezentovány výsledky provedených testů.

This project focuses on wavelength division multiplexing passive optical network (WDM-PON) supervision using optical time domain reflectometer (OTDR) for detection and localization of any fault occurred in optical distribution network. The objective is to investigate the impact of OTDR monitoring signal on the data transmission in the WDM-PON based on wavelength re-use system, where the same wavelength is assigned for both upstream and downstream to each end user. Experimental validation has been carried out to measure three different schemes, i.e. back-to-back, WDM-PON with and without OTDR connection by using 1xN and NxN arrayed waveguide gratings. Furthermore, a comprehensive comparison has been made to trace out the effect of the monitoring signal which is transmitted together with the data through the implemented setup. Finally, the result has confirmed that the OTDR supervision signal does not affect the data transmission. The experiment has been carried out at Ericsson AB, Kista.

Anomaly event detection has become increasingly important and is of great significance for real-time monitoring systems. However, developing a reliable anomaly detection and localization model still requires overcoming many challenging problems considering the ambiguity in the definition of an abnormal event and the lack of ground truth datasets for training. In this thesis, we propose a Two-way Multi-input Generative Neural Network (TMGNN), which is an unsupervised anomaly events detection and localization method based on Generative Adversarial Network (GAN). TMGNN is composed of two neural networks, an appearance generation neural network and a motion generation neural network. These two networks are trained on normal frames and their corresponding motion and mosaic frames respectively. In the testing steps, the trained model cannot properly reconstruct the anomalous objects since the network is trained only on normal frames and has not learned patterns of anomalous cases. With the help of our new patch-based evaluation method, we utilize the reconstruction error to detect and localize possible anomalous objects. Our experiments show that on the UCSD Pedestrain2 dataset, our approach achieves 96.5% Area Under Curve (AUC) and 94.1% AUC for the frame-level and pixel-level criteria, respectively, reaching the best classification results compared to other traditional and deep learning methods. / Thesis / Master of Applied Science (MASc) / Recently, abnormal event detection has attracted increasing attention in the field of surveillance video. However, it is still a big challenge to build an automatic and reliable abnormal event detection system to review a surveillance video containing hundreds of frames and mask the frames with abnormal objects or events. In this thesis, we build a model and teach it to memorize the structure of normal frames. Then the model is able to tell which frames are normal. Any other frames that appear in the surveillance video will be classified as abnormal frames. Moreover, we design a new method to evaluate the performance of our model and compare it with other models’ results.

Most engineering structures include nonlinearity to some degree. Depending on the dynamic conditions and level of external forcing, sometimes a linear structure assumption may be justified. However, design requirements of sophisticated structures such as satellites, stabilized weapon systems and radars may require nonlinear behavior to be considered for better performance. Therefore, it is very important to successfully detect, localize and parametrically identify nonlinearity in such cases. In engineering applications, the location of nonlinearity and its type may not be always known in advance. Furthermore, as the structure will be excited from only a few coordinates, the frequency response function matrices will not be complete. In order to parametrically identify more than one type of nonlinearity which may co-exist at the same location with the above mentioned limitations, a method is proposed where restoring force surface plots are used which are evaluated by describing function inversion. Then, by reformulating this method, a second method is proposed which can directly evaluate the total describing function of more than one type of nonlinearity which may co-exist at the same location without using any linear frequency response function matrix. It is also aimed in this study to use the nonlinearity localization formulations for damage localization purposes. The validation of the methods developed in this study is demonstrated with case studies based on simulated experiments, as well as real experiments with nonlinear structures and it is concluded that the methods are very promising to be used in engineering structures.

This study presents two computer vision based algorithms for automated 2D detection of construction workers and equipment from site video streams. The state-of-the-art research proposes semi-automated detection methods for tracking of construction workers and equipment. Considering the number of active equipment and workers on jobsites and their frequency of appearance in a camera's field of view, application of semi-automated techniques can be time-consuming. To address this limitation, two new algorithms based on Histograms of Oriented Gradients and Colors (HOG+C), 1) HOG+C sliding detection window technique, and 2) HOG+C deformable part-based model are proposed and their performance are compared to the state-of-the-art algorithm in computer vision community. Furthermore, a new comprehensive benchmark dataset containing over 8,000 annotated video frames including equipment and workers from different construction projects is introduced. This dataset contains a large range of pose, scale, background, illumination, and occlusion variation. The preliminary results with average performance accuracies of 100%, 92.02%, and 89.69% for workers, excavators, and dump trucks respectively, indicate the applicability of the proposed methods for automated activity analysis of workers and equipment from single video cameras. Unlike other state-of-the-art algorithms in automated resource tracking, these methods particularly detects idle resources and does not need manual or semi-automated initialization of the resource locations in 2D video frames. / Master of Science

Ce travail s’inscrit dans le cadre de la mise en place d’une stratégie de SHM (Structural Health Monitoring) dédiée à la surveillance des structures en génie civil. Il a porté, d’une part, sur l’étude des méthodes de détection et de localisation de l’endommagement du bâti existant et, d’autre part, sur l’élaboration du cahier des charges d’un capteur « intégré » capable de délivrer des informations par transmission compacte des données pour les communiquer à une chaîne SHM. Des études numériques et expérimentales ont été réalisées dans cet objectif. L’état de l’art a clairement mis en évidence plusieurs points faibles des méthodes de détection et de localisation d’endommagements usuelles comme, par exemple, le manque de précision et/ou la complexité de mise en place. On observe aussi que la sensibilité de ces méthodes par rapport à plusieurs paramètres, essentiellement la direction de mesure, le positionnement des capteurs et la sévérité des endommagements, ne permet pas à ce jour de dresser un diagnostic précis de l’état de santé des structures. Pour répondre au cahier des charges d’une chaîne SHM, un Algorithme de Détection et de Localisation (ADL) a été élaboré. Cet algorithme fait appel à des méthodes utilisant les paramètres modaux, essentiellement les fréquences propres et les déformées modales. Leurs mises en œuvre séquentielles et itératives, judicieusement structurées et pilotées,a permis de répondre aux objectifs fixés. Les paramètres modaux requis sont identifiés à l’aide des techniques d’Analyse Modale Opérationnelle (AMO) et à partir de la réponse en accélérations des structures. Deux algorithmes d’AMO ont été utilisés pour leur efficacité et pour leur aptitude à l’automatisation: la méthode stochastique par sous ensemble (SSI), et la méthode de décomposition dans le domaine fréquentiel (FDD). En fusionnant les algorithmes d’AMO avec l’ADL, une chaîne complète de surveillance a été créée. La validation des algorithmes et de la chaîne de surveillance s’est faite à plusieurs niveaux. Tout d’abord, basés sur la théorie des éléments finis, des modèles numériques de la tour de l'Ophite et du pont canadien de la Rivière aux-Mulets ont permis d'évaluer l'ADL. Ces modèles sont endommagés par des signaux sismiques et fournissent les données accélérométriques, données d’entrée du logiciel que nous avons développé. Les résultats obtenus sont tout à fait satisfaisants voire meilleurs que ceux issus des méthodes usuelles. Dans un second temps, nous avons traité des données expérimentales «réelles », issues des mesures accélérométriques sur la tour de l’Ophite. La confrontation entre les résultats d’identification des fréquences propres et des déformées modales issus des algorithmes d’AMO et ceux reportés par la bibliographie, a révélé l’efficacité des algorithmes développés.Enfin, une maquette d’un bâtiment à échelle réduite a également été élaborée et instrumentée.L’application de la chaine de surveillance a permis, d’une part, de détecter et localiser l’endommagement introduit dans la structure et, d’autre part, de mettre en évidence l’intérêt de la surveillance automatique. Finalement, une étude a été menée dans le but de réduire la quantité d’informations enregistrées sur les structures et de faciliter le transfert des données servant comme entrées de la chaîne de surveillance. Les résultats de ces études ont contribué à la spécification d’un nouveau système de surveillance / The work presented in this thesis is part of the development of a Structural Health Monitoring(SHM) system dedicated to civil engineering applications. First, it studies the methods of damagedetection and localization. Furthermore, it helps elaborate the specifications of an integratedsensor capable of delivering information by compact transmission of data to an SHM chain.Numerical and experimental studies have been carried out for this purpose. The study of theliterature clearly highlighted several weak points of the traditional damage detection andlocalization methods, such as the lack of precision and the complexity of implementation. Thesensitivity of these methods with respect to several parameters, essentially the measurementdirection, the positioning of the sensors and the severity of the damage, makes it impossible todraw up an accurate diagnosis of the structures. In order to overcome these limitations, a damageDetection and Localization Algorithm (DLA) was developed. By applying Vibration-Based Damage Detection Methods, following a precise order and taking into account the sensitivity, the simplicityand the SHM level of each method, this algorithm made it possible to meet the objectives set at the beginning of this work. The required modal parameters, namely eigen-frequencies and modeshapes, were identified from the structure’s output-only response using Operational ModalAnalysis techniques (OMA). Two OMA algorithms were used for their efficiency and automationability: the Stochastic Subspace Identification method (SSI) and the Frequency DomainDecomposition method (FDD). By merging the OMA algorithms with the DLA, a complete SHMchain was created. The algorithms validation was made at several levels. First, the DLA wasevaluated using a Finite Element Model (FEM) of the Ophite tower and the Rivière aux Muletsbridge. The results obtained were quite satisfactory. Secondly, experimental data were processed,from accelerometric measurements on the Ophite tower. The confrontation between the results ofeigen-frequencies and mode shapes identification using OMA algorithms and those reported in theliterature revealed the efficiency of the developed algorithms. Finally, a scale model of a buildingwas developed, instrumented and damaged. The use of the surveillance chain allowed thedetection and localization of the damage. Moreover, it showed all the interest of using automatic surveillance. The last step of this work dealt with a study carried out to reduce the amount of datarecorded on structures in order to facilitate their transfer to the SHM chain. As a conclusion, the results of these studies contributed to the specification of a new monitoring system

Ce travail de thèse concerne la surveillance de l’état de santé de structures complexes en service. Elle est appliquée à des éléments d’une nacelle d’avion gros porteur. Ce travail est original et s’inscrit dans le cadre d’un projet, coordonné par AIRBUS Operations SAS et porté par AIRCELLE (Groupe SAFRAN). Les principales parties de la nacelle visées par notre démarche sont le capot de soufflante (fan cowl, composite monolithique) et la structure interne fixe du capot coulissant de l’inverseur de poussée (IFS, sandwich nid d’abeille). Ces structures réalisées en matériaux composites sont sujettes à de nombreux modes de dégradation(rupture de fibres, délaminage, fissures, etc…), qui peuvent impacter la durée de vie de la nacelle. De plus elles sont exposées à de nombreuses sollicitations environnementales dont des variations thermiques importantes (de -55 °C à +120°C). L’objectif de ce travail est la mise en place d’un système SHM visant à suivre l’état de santé de ces structures afin de détecter l’apparition de tels endommagements et de les localiser avant qu’ils ne conduisent à une dégradation de la structure; ceci de manière à permettre une maintenance prédictive. Des capteurs et actionneurs piézoélectriques (PZT) sont collés sur la structure et sont utilisés pour générer des ondes de Lamb et effectuer des mesures. La démarche SHM proposée s’appuie sur des mesures successives en partant d’un état initial considéré comme sain, puis en réalisant régulièrement des mesures de suivi. La différence entre des signaux mesurés pour deux états est analysée afin d’en extraire des caractéristiquessensibles à l’apparition de dommages. Après validation, des PZT ont été collés sur le fan cowl et l’IFS ainsi que sur des coupons et un banc d’essai approprié a été conçu afin de valider notre démarche. Du fait que l’on est amené à travailler sur des différences de signaux, des algorithmes de détection, basés sur les testsd’hypothèses statistiques et l’Analyse en Composantes Principales (ACP), ont dû être développés et validés. Ceci a d’abord été testé pour la détection de dommages contrôlés introduits d’abord dans des coupons, puis dans le fan cowl et dans l’IFS. Des algorithmes robustes (y compris aux variations de température) de localisation de ces dommages, basés sur l’extraction des temps de vol des ondes de Lamb, ont été développés et validés sur les structures étudiées. Une approche de quantification des incertitudes sur la localisation par inférence Bayésienne a été proposée en complément de la démarche déterministe implémentée. / This work aims at designing a Structural Health Monitoring (SHM) system for complex composite structures, with an application to elements of aeronautical nacelles. This work is original and is in the framework of a project, coordinated by AIRBUS Operations SAS and headed by AIRCELLE (SAFRAN Group). The main parts of the nacelle concerned with our approach are the fan cowl (composite monolithic) and the inner fixed structure (IFS, sandwich structure with honeycomb core) of the thrust reverser. These structures made from composite materials are subjected to many damages types which can affect nacelle’s useful life (fiber breaking, delamination, crack, etc…). Furthermore these structures are exposed to many environmental constraints which are for instance important thermal variations (from -55°C to +120°C). The objective of this work is to develop a SHM system aimed at detecting and localizing these damages, before the degradation of the whole structureoccurs. Piezoelectric (PZT) actuators and sensors are bonded on the structure and they are used to generate Lamb wave signals and perform measurements. The proposed SHM approach is based on successive measurements starting from an initial state, considered as healthy and regularly conducting follow-up. The difference in signals measured between two states is analyzed in order to extract some damages-sensitivesfeatures. After validation, PZT elements were glued to the fan cowl and to the IFS as well as on representative coupons and a suitable test bench is designed in order to validate our approach. Since one has to work on difference in signals, damage detection algorithms based on statistical hypothesis testing and PrincipalComponent Analysis (PCA) have been developed and validated. This was first tested for the detection of controlled damages introduced in coupons, and thereafter on the fan cowl and IFS. Robust damage localization algorithms (including with temperature variations) based on Time-of-flight (ToF) extraction from difference in signals, were developed and validated for these structures. A Bayesian approach for uncertainties quantification in the damage localization is also developed, leading to more accuracy in the damage localization results.