This dissertation introduces work on face recognition using a novel technique based on Hidden Markov Models (HMMs). Through the integration of a priori structural knowledge with statistical information, HMMs can be used successfully to encode face features. The results reported are obtained using a database of images of 40 subjects, with 5 training images and 5 test images for each. It is shown how standard one-dimensional HMMs in the shape of top-bottom models can be parameterised, yielding successful recognition rates of up to around 85%. The insights gained from top-bottom models are extended to pseudo two-dimensional HMMs, which offer a better and more flexible model, that describes some of the twodimensional dependencies missed by the standard one-dimensional model. It is shown how pseudo two-dimensional HMMs can be implemented, yielding successful recognition rates of up to around 95%. The performance of the HMMs is compared with the Eigenface approach and various domain and resolution experiments are also carried out. Finally, the performance of the HMM is evaluated in a fully automated system, where database images are cropped automatically.

Highway assets, including traffic signs, traffic signals, light poles, and guardrails, are important components of transportation networks. They guide, warn and protect drivers, and regulate traffic. To manage and maintain the regular operation of the highway system, state departments of transportation (DOTs) need reliable and up-to-date information about the location and condition of highway assets. Different methodologies have been employed to collect road inventory data. Currently, ground-based technologies are widely used to help DOTs to continually update their road database, while air-based methods are not commonly used. One possible reason is that the initial investment for air-based methods is relatively high; another is the lack of a systematic and effective approach to extract road features from raw airborne light detection and ranging (LiDAR) data and aerial image data. However, for large-area inventories (e.g., a whole state highway inventory), the total cost of using aerial mapping is actually much lower than other methods considering the time and personnel needed. Moreover, unmanned aerial vehicles (UAVs) are easily accessible and inexpensive, which makes it possible to reduce costs for aerial mapping. The focus of this project is to analyze the capability and strengths of airborne data collection system in highway inventory data collection. In this research, a field experiment was conducted by the Remote Sensing Service Laboratory (RSSL), Utah State University (USU), to collect airborne data. Two kinds of methodologies were proposed for data processing, namely ArcGIS-based algorithm for airborne LiDAR data, and MATLAB-based procedure for aerial photography. The results proved the feasibility and high efficiency of airborne data collection method for updating highway inventory database.

L’un des défis de la science planétaire est la détermination de l’âge des surfaces des différents corps célestes du système solaire, pour comprendre leurs processus de formation et d’évolution. Une approche repose sur l’analyse de la densité et de la taille des cratères d’impact. En raison de l’énorme quantité de données à traiter, des approches automatiques ont été proposées pour détecter les cratères d’impact afin de faciliter ce processus de datation. Ils utilisent généralement les valeurs de couleur des images ou les valeurs d’altitude de "modèles numériques d’élévation" (DEM). Dans cette thèse, nous proposons une nouvelle approche pour détecter les bords des cratères. L’idée principale est de combiner l’analyse de la courbure avec une classification basée sur un réseau de neurones. Cette approche comporte deux étapes principales : premièrement, chaque sommet du maillage est étiqueté avec la valeur de la courbure minimale; deuxièmement, cette carte de courbure est injectée dans un réseau de neurones pour détecter automatiquement les formes d’intérêt. Les résultats montrent que la détection des formes est plus efficace en utilisant une carte en deux dimensions s’appuyant sur le calcul d’estimateurs différentiels discrets, plutôt qu’en utilisant la valeur de l’élévation en chaque sommet. Cette approche réduit significativement le nombre de faux négatifs par rapport aux approches précédentes basées uniquement sur une information topographique. La validation de la méthode est effectuée sur des DEM de Mars, acquis par un altimètre laser à bord de la sonde spatiale "Mars Global Surveyor" de la NASA et combinés avec une base de données de cratères identifiés manuellement. / One of the challenges of planetary science is the age determination of the surfaces of the different celestial bodies in the solar system, to understand their formation and evolution processes. An approach relies on the analysis of the crater impact density and size. Due to the huge quantity of data to process, automatic approaches have been proposed for automatically detecting impact craters in order to facilitate this dating process. They generally use the color values from images or the elevation values from Digital Elevation Model (DEM). In this PhD thesis, we propose a new approach for detecting craters rims. The main idea is to combine curvature analysis with Neural Network based classification. This approach contains two main steps: first, each vertex of the mesh is labeled with the value of the minimal curvature; second, this curvature map is injected into a neural network to automatically detect the shapes of interest. The results show that detecting forms are more efficient using a two-dimensional map based on the computation of discrete differential estimators, than by the value of the elevation at each vertex. This approach significantly reduces the number of false negatives compared to previous approaches based on topographic information only. The validation of the method is performed on DEMs of Mars, acquired by a laser altimeter aboard NASA’s Mars Global Surveyor spacecraft and combined with a database of manually identified craters.

This thesis evaluates two methods for automatic feature extraction to classify the accelerometer data of periodic and sporadic human activities. The first method selects features using individual hypothesis tests and the second one is using a random forest classifier as an embedded feature selector. The hypothesis test was combined with a correlation filter in this study. Both methods used the same initial pool of automatically generated time series features. A decision tree classifier was used to perform the human activity recognition task for both methods.The possibility of running the developed model on a processor with limited computing power was taken into consideration when selecting methods for evaluation. The classification results showed that the random forest method was good at prioritizing among features. With 23 features selected it had a macro average F1 score of 0.84 and a weighted average F1 score of 0.93. The first method, however, only had a macro average F1 score of 0.40 and a weighted average F1 score of 0.63 when using the same number of features. In addition to the classification performance this thesis studies the potential business benefits that automation of feature extractioncan result in. / Denna studie utvärderar två metoder som automatiskt extraherar features för att klassificera accelerometerdata från periodiska och sporadiska mänskliga aktiviteter. Den första metoden väljer features genom att använda individuella hypotestester och den andra metoden använder en random forest-klassificerare som en inbäddad feature-väljare. Hypotestestmetoden kombinerades med ett korrelationsfilter i denna studie. Båda metoderna använde samma initiala samling av automatiskt genererade features. En decision tree-klassificerare användes för att utföra klassificeringen av de mänskliga aktiviteterna för båda metoderna. Möjligheten att använda den slutliga modellen på en processor med begränsad hårdvarukapacitet togs i beaktning då studiens metoder valdes. Klassificeringsresultaten visade att random forest-metoden hade god förmåga att prioritera bland features. Med 23 utvalda features erhölls ett makromedelvärde av F1 score på 0,84 och ett viktat medelvärde av F1 score på 0,93. Hypotestestmetoden resulterade i ett makromedelvärde av F1 score på 0,40 och ett viktat medelvärde av F1 score på 0,63 då lika många features valdes ut. Utöver resultat kopplade till klassificeringsproblemet undersöker denna studie även potentiella affärsmässiga fördelar kopplade till automatisk extrahering av features.

Cette thèse de doctorat traite de la reconnaissance automatique du Langage français Parlé Complété (LPC), version française du Cued Speech (CS), à partir de l’image vidéo et sans marquage de l’information préalable à l’enregistrement vidéo. Afin de réaliser cet objectif, nous cherchons à extraire les caractéristiques de haut niveau de trois flux d’information (lèvres, positions de la main et formes), et fusionner ces trois modalités dans une approche optimale pour un système de reconnaissance de LPC robuste. Dans ce travail, nous avons introduit une méthode d’apprentissage profond avec les réseaux neurono convolutifs (CNN)pour extraire les formes de main et de lèvres à partir d’images brutes. Un modèle de mélange de fond adaptatif (ABMM) est proposé pour obtenir la position de la main. De plus, deux nouvelles méthodes nommées Modified Constraint Local Neural Fields (CLNF Modifié) et le model Adaptive Ellipse Model ont été proposées pour extraire les paramètres du contour interne des lèvres (étirement et ouverture aux lèvres). Le premier s’appuie sur une méthode avancée d’apprentissage automatique (CLNF) en vision par ordinateur. Toutes ces méthodes constituent des contributions significatives pour l’extraction de caractéristiques du LPC. En outre, en raison de l’asynchronie des trois flux caractéristiques du LPC, leur fusion est un enjeu important dans cette thèse. Afin de le résoudre, nous avons proposé plusieurs approches, y compris les stratégies de fusion au niveau données et modèle avec une modélisation HMM dépendant du contexte. Pour obtenir le décodage, nous avons proposé trois architectures CNNs-HMMs. Toutes ces architectures sont évaluées sur un corpus de phrases codées en LPC en parole continue sans aucun artifice, et la performance de reconnaissance CS confirme l’efficacité de nos méthodes proposées. Le résultat est comparable à l’état de l’art qui utilisait des bases de données où l’information pertinente était préalablement repérée. En même temps, nous avons réalisé une étude spécifique concernant l’organisation temporelle des mouvements de la main, révélant une avance de la main en relation avec l’emplacement dans la phrase. En résumé, ce travail de doctorat propose les méthodes avancées d’apprentissage automatique issues du domaine de la vision par ordinateur et les méthodologies d’apprentissage en profondeur dans le travail de reconnaissance CS, qui constituent un pas important vers le problème général de conversion automatique de CS en parole audio. / This PhD thesis deals with the automatic continuous Cued Speech (CS) recognition basedon the images of subjects without marking any artificial landmark. In order to realize thisobjective, we extract high level features of three information flows (lips, hand positions andshapes), and find an optimal approach to merging them for a robust CS recognition system.We first introduce a novel and powerful deep learning method based on the ConvolutionalNeural Networks (CNNs) for extracting the hand shape/lips features from raw images. Theadaptive background mixture models (ABMMs) are also applied to obtain the hand positionfeatures for the first time. Meanwhile, based on an advanced machine learning method Modi-fied Constrained Local Neural Fields (CLNF), we propose the Modified CLNF to extract theinner lips parameters (A and B ), as well as another method named adaptive ellipse model. Allthese methods make significant contributions to the feature extraction in CS. Then, due tothe asynchrony problem of three feature flows (i.e., lips, hand shape and hand position) in CS,the fusion of them is a challenging issue. In order to resolve it, we propose several approachesincluding feature-level and model-level fusion strategies combined with the context-dependentHMM. To achieve the CS recognition, we propose three tandem CNNs-HMM architectureswith different fusion types. All these architectures are evaluated on the corpus without anyartifice, and the CS recognition performance confirms the efficiency of our proposed methods.The result is comparable with the state of the art using the corpus with artifices. In parallel,we investigate a specific study about the temporal organization of hand movements in CS,especially about its temporal segmentation, and the evaluations confirm the superior perfor-mance of our methods. In summary, this PhD thesis applies the advanced machine learningmethods to computer vision, and the deep learning methodologies to CS recognition work,which make a significant step to the general automatic conversion problem of CS to sound.The future work will mainly focus on an end-to-end CNN-RNN system which incorporates alanguage model, and an attention mechanism for the multi-modal fusion.

This paper describes a unique method for the semi-automatic detection of historic charcoal production sites in LiDAR-derived digital elevation models. Intensified iron production in the early 17th century has remarkably influenced ways of how the land in Sweden was managed. Today, the abundance of charcoal kilns embedded in the landscape survives as cultural heritage monuments that testify about the scale forest management for charcoal production has contributed to the uprising iron manufacturing industry. An arbitrary selected study area (54 km2) south west of Gävle city served as an ideal testing ground, which is known to consist of already registered as well as unsurveyed charcoal kiln sites. The proposed approach encompasses combined morphometric classification methods being subjected to analytical image processing, where an image that represents refined terrain morphology was segmented and further followed by Hough Circle transfer function applied in seeking to detect circular shapes that represent charcoal kilns. Sites that have been identified manually and using the proposed method were only verified within an additionally established smaller validation area (6 km2). The resulting outcome accuracy was measured by calculating harmonic mean of precision and recall (F1-Score). Along with indication of previously undiscovered site locations, the proposed method showed relatively high score in recognising already registered sites after post-processing filtering. In spite of required continual fine-tuning, the described method can considerably facilitate mapping and overall management of cultural resources.