The advent of multimedia technology has offered a new dimension in computerised applications. Art-based applications are among those which have and will continue to benefit from this advancement. Content-based image retrieval (CBIR) and analysis is attracting attention from museums and art institutions. One of the image-based requirements from museums is to automatically classify craquelure (cracks) in paintings for the purpose of aiding damage assessment using non-destructive monitoring and testing. Craquelure in paintings can be an important element in judging authenticity, use of material as well as environmental and physical impact, which these can contribute to different craquelure patterns. Mass screening of craquelure patterns will help to establish a better platform for conservators to identify cause of damage and a content-based approach is seen as an appropriate path. This thesis covers the issues of crack enhancement and detection, using a mathematical morphology technique, namely the top-hat operator and also a grid-based automatic thresholding. Craquelure representation aids the processes of craquelure pattern analysis in which the Freeman chain-code is used as a basis for converting the image-based representation into a hierarchically structured numerical form. This hierarchical representation offers several advantages in the sense that detected craquelure patterns can be pruned, according to a certain rule for eliminating suspected noise and insignificant structures. Information can be retrieved in a flexible way, given multi-level access into structural detail. A grouping technique determines ‘objects-of-interest’ and structured craquelure patterns, named crack-networks are grouped using proximity and characteristic rules. Craquelure patterns are generalised by utilising conservative approximations based on the minimum bounding rectangle (MBR) and rotated minimum bounding rectangle (RMBR). Meaningful features based on orientation histograms and structural statistics are extracted to distinguish between craquelure patterns. The resultant features are used as inputs for a three-stage average distance k-nearest neighbour (k-NN) classifier with fuzzy outputs where the goal is to produce class memberships. A prototype architecture of a craquelure retrieval system is also discussed.

This thesis presents a fully automatic facial expression analysis system based on the Facial Action Coding System (FACS). FACS is the best known and the most commonly used system to describe facial activity in terms of facial muscle actions (i.e., action units, AUs). We will present our research on the analysis of the morphological, spatio-temporal and behavioural aspects of facial expressions. In contrast with most other researchers in the field who use appearance based techniques, we use a geometric feature based approach. We will argue that that approach is more suitable for analysing facial expression temporal dynamics. Our system is capable of explicitly exploring the temporal aspects of facial expressions from an input colour video in terms of their onset (start), apex (peak) and offset (end). The fully automatic system presented here detects 20 facial points in the first frame and tracks them throughout the video. From the tracked points we compute geometry-based features which serve as the input to the remainder of our systems. The AU activation detection system uses GentleBoost feature selection and a Support Vector Machine (SVM) classifier to find which AUs were present in an expression. Temporal dynamics of active AUs are recognised by a hybrid GentleBoost-SVM-Hidden Markov model classifier. The system is capable of analysing 23 out of 27 existing AUs with high accuracy. The main contributions of the work presented in this thesis are the following: we have created a method for fully automatic AU analysis with state-of-the-art recognition results. We have proposed for the first time a method for recognition of the four temporal phases of an AU. We have build the largest comprehensive database of facial expressions to date. We also present for the first time in the literature two studies for automatic distinction between posed and spontaneous expressions.

Analysing objects interacting in a 3D environment and captured by a video camera requires knowledge of their motions. Motion estimation provides such information, and consists of re-covering 2D image velocity, or optical flow, of the corresponding moving 3D objects. A gradient-based optical flow estimator is implemented in this thesis to produce a dense field of velocity vectors across an image. An iterative and parameterised approach is adopted which fits planar motion models locally on the image plane. Motion is then estimated using a least-squares minimisation approach. The possible approximations of the optical flow derivative are shown to differ greatly when the magnitude of the motion increases. However, the widely used derivative term remains the optimal approximation to use in the range of accuracies of the gradient-based estimators i.e. small motion magnitudes. Gradient-based estimators do not estimate motion robustly when noise, large motions and multiple motions are present across object boundaries. A robust statistical and multi-resolution estimator is developed in this study to address these limitations. Despite significant improvement in performance, the multiple motion problem remains a major limitation. A confidence measurement is designed around optical flow covariance to represent motion accuracy, and is shown to visually represent the lack of robustness across motion boundaries. The recent hyperplane technique is also studied as a global motion estimator but proved unreliable compared to the gradient-based approach. A computationally expensive optical flow estimator is then designed for the purpose of detecting at frame-rate moving objects occluding background scenes which are composed of static objects captured by moving pan and tilt cameras. This was achieved by adapting the estimator to perform global motion estimation i.e. estimating the motion of the background scenes. Moving objects are segmented from a thresholding operation on the grey-level differences between motion compensated background frames and captured frames. Filtering operations on small object dimensions and using moving edge information produced reliable results with small levels of noise. The issue of tracking moving objects is studied with the specific problem of data correspondence in occlusion scenarios.

De nos jours, la vidéo-surveillance est une thématique pour laquelle se pose le problème du traitement de données de masse pour la reconnaissance et le suivi de personnes. L’objectif est de créer un système de reconnaissance automatique de personnes basée sur l’apparence en environnement réel. Deux différentes tâches sont visées : ré-identification et découverte de nouvelles personnes. Le système proposé se divise en quatre modules : acquisition des données, extraction du fond et de la silhouette, extraction et sélection des attributs basées sur l’apparence et reconnaissance. Pour l’évaluation du système, en sus d’une base de données publique (CASIA), une nouvelle base de données a été créée avec de très faibles contraintes sur le scénario. Des attributs couleurs normalisés et les attributs de textures d’Haralick sont extraits, puis des algorithmes de sélection d’attributs sont comparés. Ces sous-ensembles d’attributs optimaux sont utilisés tout d’abord pour la ré-identification de personnes à l’aide de SVM incrémental et décrémental (MID-SVM), ayant l’avantage de ne nécessiter que peu de données pour la création du modèle. Une seconde utilisation de ces données se fait pour ajouter la découverte de nouvelles personnes inconnues jusqu’alors, en utilisant un algorithme de regroupement (Self-Adaptive Kernel Machine – SAKM) capable de différentier des personnes existantes qui peuvent être classifiées de nouvelles personnes pour lesquelles un modèle est créé. Le système proposé est capable de ré-identifier des personnes avec un taux de succès supérieur à 95% et atteint des performances satisfaisantes pour la découverte de nouvelles personnes avec un taux de plus de 90%. / Video surveillance is nowadays an important topic to address, as it is broadly used for security and it brings problems related to big data processing. A part of it is identification and re-identification of persons in multicamera environments. The objective of this thesis work is to design a complete automatic appearance-based human recognition system working in real-life environment, with the goal to achieve two main tasks: person re-identification and new person discovery. The proposed system consists of four modules: video data acquisition; background extraction and silhouette extraction; feature extraction and selection; and person recognition. For evaluation purposes, in addition to the public available CASIA Database, a more challenging new database has been created under low constraints. Grey-world normalized color features and Haralick texture features are extracted as initial feature subset, then features selection approaches are tested and compared. These optimized subsets of features are then used firstly for person re-identification using Multi-category Incremental and Decremental SVM (MID-SVM) algorithm with the advantage of training only with few initial images and secondly for person discovery and classification using Self-Adaptive Kernel Machine (SAKM) algorithm able to differentiate existing persons who can be classified from new persons who have to be learned and added. The proposed system succeed in person re-identification with classification rate of over 95\% and achieved satisfying performances on person discovery with accuracy rate of over 90%.

Récemment, le développement de systèmes intelligents dédiés pour la compréhension du comportement humain est devenu un axe de recherche très important. En effet, il est très important de comprendre le comportement humain pour rendre les machines capables d'aider et interagir avec les humains. Pour cela, plusieurs approches de l'état de l'art commencent par détecter automatiquement un ensemble de points 2D ou 3D, appelés marqueurs, sur le corps et/ou le visage humain à partir de données visuelles. L’analyse des séquences temporelles de ces marqueurs pose plusieurs défis dus aux erreurs de suivi et aux variabilités temporelles et de pose. Dans cette thèse, nous proposons deux nouvelles représentations spatio-temporelles avec des outils de calcul appropriés pour la compréhension du comportement humain. La première consiste à représenter une séquence temporelle de marqueurs par une trajectoire de matrices de Gram. Les matrices de Gram sont des matrices semi-définies positives de rang fixe et vivent dans un espace non-linéaire dans lequel les outils d’apprentissage automatique conventionnels ne peuvent pas être appliqués directement. Nous évaluons l’efficacité de notre approche dans plusieurs applications, impliquant des marqueurs 2D et 3D de visages et de corps humain, tels que la reconnaissance des émotions à partir des expressions faciales la reconnaissance d’actions et des émotions à partir des données de profondeur 3D. La deuxième représentation proposée dans cette thèse est basée sur les coordonnées barycentriques des marqueurs de visages 2D. Cette représentation permet d’utiliser les outils de calcul et d’apprentissage automatique tels que les techniques d’apprentissage de métrique. Les résultats obtenus en reconnaissance des expressions faciales et en mesure automatique de la sévérité de la dépression à partir du visage montrent tout l’intérêt de la représentation barycentrique combinée à des techniques d’apprentissage automatique. Les résultats obtenus avec les deux méthodes proposées sur des bases de données réelles montrent la compétitivité de nos approches avec les méthodes récentes de l’état de l’art. / Developing intelligent systems dedicated to human behavior understanding has been a very hot research topic in the few recent decades. Indeed, it is crucial to understand the human behavior in order to make machines able to interact with, assist, and help humans in their daily life.. Recent breakthroughs in computer vision and machine learning have made this possible. For instance, human-related computer vision problems can be approached by first detecting and tracking 2D or 3D landmark points from visual data. Two relevant examples of this are given by the facial landmarks detected on the human face and the skeletons tracked along videos of human bodies. These techniques generate temporal sequences of landmark configurations, which exhibit several distortions in their analysis, especially in uncontrolled environments, due to view variations, inaccurate detection and tracking, missing data, etc. In this thesis, we propose two novel space-time representations of human landmark sequences along with suitable computational tools for human behavior understanding. Firstly, we propose a representation based on trajectories of Gram matrices of human landmarks. Gram matrices are positive semi-definite matrices of fixed rank and lie on a nonlinear manifold where standard computational and machine learning techniques could not be applied in a straightforward way. To overcome this issue, we make use of some notions of the Riemannian geometry and derive suitable computational tools for analyzing Gram trajectories. We evaluate the proposed approach in several human related applications involving 2D and 3D landmarks of human faces and bodies such us emotion recognition from facial expression and body movements and also action recognition from skeletons. Secondly, we propose another representation based on the barycentric coordinates of 2D facial landmarks. While being related to the Gram trajectory representation and robust to view variations, the barycentric representation allows to directly work with standard computational tools. The evaluation of this second approach is conducted on two face analysis tasks namely, facial expression recognition and depression severity level assessment. The obtained results with the two proposed approaches on real benchmarks are competitive with respect to recent state-of-the-art methods.

Les caméras multispectrales échantillonnent le spectre du visible et/ou de l'infrarouge selon des bandes spectrales étroites. Parmi les technologies disponibles, Les caméras snapshot équipées d'une mosaïque de filtres acquièrent des images brutes à cadence vidéo. Ces images brutes nécessitent un processus de dématriçage permettant d'estimer l'image multispectrale en pleine définition. Dans ce manuscrit nous examinons les méthodes de dématriçage multispectrale et proposons une nouvelle méthode basée sur l'image panchromatique. De plus, nous mettons en évidence l'influence de l'illumination sur les performances de dématriçage, puis nous proposons des étapes de normalisation rendant ce dernier robuste aux propriétés d'acquisition. Les résultats expérimentaux montrent que notre méthode fournit de meilleurs résultats que les méthodes classiques.Afin d'effectuer une analyse de texture, nous étendons les opérateurs basés sur les motifs binaires locaux aux images de texture multispectrale au détriment d'exigences de mémoire et de calcul accrues. Nous proposons alors de calculer les descripteurs de texture directement à partir d'images brutes, ce qui évite l'étape de dématriçage tout en réduisant la taille du descripteur. Afin d'évaluer la classification sur des images multispectrales, nous avons proposé la première base de données multispectrale de textures proches dans les domaines spectraux du visible et du proche infrarouge. Des expériences approfondies sur cette base montrent que le descripteur proposé a à la fois un coût de calcul réduit et un pouvoir de discrimination élevé en comparaison avec les descripteurs classiques appliqués aux images dématriçées. / Multispectral cameras sample the visible and/or the infrared spectrum according to narrow spectral bands. Available technologies include snapshot multispectral cameras equipped with filter arrays that acquire raw images at video rate. Raw images require a demosaicing procedure to estimate a multispectral image with full spatio-spectral definition. In this manuscript we review multispectral demosaicing methods and propose a new one based on the pseudo-panchromatic image. We highlight the influence of illumination on demosaicing performances, then we propose pre- and post-processing normalization steps that make demosaicing robust to acquisition properties. Experimental results show that our method provides estimated images of better objective quality than classical ones.Multispectral images can be used for texture classification. To perform texture analysis, we extend local binary pattern operators to multispectral texture images at the expense of increased memory and computation requirements. We propose to compute texture descriptors directly from raw images, which both avoids the demosaicing step and reduces the descriptor size. In order to assess classification on multispectral images we have proposed the first significant multispectral database of close-range textures in the visible and near infrared spectral domains. Extensive experiments on this database show that the proposed descriptor has both reduced computational cost and high discriminating power with regard to classical local binary pattern descriptors applied to demosaiced images.

In this thesis visual search experiments are devised to explore the feasibility of an eye gaze driven search mechanism. The thesis first explores gaze behaviour on images possessing different levels of saliency. Eye behaviour was predominantly attracted by salient locations, but appears to also require frequent reference to non-salient background regions which indicated that information from scan paths might prove useful for image search. The thesis then specifically investigates the benefits of eye tracking as an image retrieval interface in terms of speed relative to selection by mouse, and in terms of the efficiency of eye tracking mechanisms in the task of retrieving target images. Results are analysed using ANOVA and significant findings are discussed. Results show that eye selection was faster than a computer mouse and experience gained during visual tasks carried out using a mouse would benefit users if they were subsequently transferred to an eye tracking system. Results on the image retrieval experiments show that users are able to navigate to a target image within a database confirming the feasibility of an eye gaze driven search mechanism. Additional histogram analysis of the fixations, saccades and pupil diameters in the human eye movement data revealed a new method of extracting intentions from gaze behaviour for image search, of which the user was not aware and promises even quicker search performances. The research has two implications for Content Based Image Retrieval: (i) improvements in query formulation for visual search and (ii) new methods for visual search using attentional weighting. Futhermore it was demonstrated that users are able to find target images at sufficient speeds indicating that pre-attentive activity is playing a role in visual search. A current review of eye tracking technology, current applications, visual perception research, and models of visual attention is discussed. A review of the potential of the technology for commercial exploitation is also presented.

Malgré les progrès technologiques modernes, l'industrie du vêtement est toujours ancrée dans une approche traditionnelle en 2D et en 3D lors d’essayage virtuel. La question fondamentale de l’accessibilité aux données morphologiques du consommateur en ligne n'a toujours pas été résolue de manière appropriée. De plus, l’interactivité et le relationnel entre le corps humain et le vêtement ne sont pas suffisamment explorés pour atteindre une performance satisfaisante lors de l'essayage du vêtement en ligne. Mes travaux de recherche ont donc pour objectif de combler ces lacunes en proposant une plate-forme numérique intégrant à la fois la connaissance des experts du secteur de l’habillement (ajustement et confort), et les retours sensoriels des clients (au porté du produit) en misant en place une nouvelle stratégie de conception de vêtement en 3D afin de calculer et d’ajuster les valeurs de l’aisance 3D de celui-ci, comme les points clefs lors de la perception et la satisfaction du produit par le client. C’est à partir d’une méthode de classification supervisée associée à un descripteur de forme 2D que nous avons retrouvé le morphotype du client en 3D avec son avatar. Cette relation complexe entre la reconnaissance du corps porteurs et la conception de vêtements 3D approprié dans essayage virtuel a été testée et analysée dans le cadre de ce projet pour bâtir une solution de conception adaptée à un environnement à distance. À cette fin, nous avons présenté les principes de la modélisation du vêtement directement adaptés à la morphologie du porteur afin de couvrir toute la gamme de formes et de mesures corporelles. / Despite modern technological progresses, the apparel industry is still anchored in the traditional 2D-to-3D design approach. Additionally, the aspects of the relation between human body and garment are not sufficiently explored in order to provide satisfactory performance of virtual try-on in the aspects of providing not only right fit and comfort to the customer but also avoiding returns to the retailer. However the main aspect is a lack of appropriately resolved issue of consumers’ body recognition in an online environment and proper 3D design methodology for individual client. In my PhD research, we challenge those gaps by proposing a foundation of a digital and knowledge-based platform for garment design and fit and comfort evaluation by integrating customers' and experts’ knowledge with the design parameters. By building a new 3D design strategy, we proposed an original method to calculate and adjust the 3D ease allowance values, which constitutes the key issues of satisfaction perception. Our 3D design method is linked to the consumer’s virtual representation, which come from a new pattern recognition method permitting to identify individual morphology from a single web-camera. It was experimentally shown that using the supervised method to create 2D shape descriptors enables to detect wearers’ morphotypes for a target population. The complex relationship between wearers’ body recognition, 3D garment design and garment fitting in virtual try-on has been tested and analyzed in the scope of this research project to build a suitable design solution applied to the remote environment.

Les outils d’aide au diagnostic sont de nos jours au cœur de plusieurs protocoles cliniques car ils améliorent la qualité du diagnostic posé et des soins médicaux. Ce travail de recherche met en avant une architecture hiérarchique pour la conception d'un outil d'aide à la détection du cancer du sein robuste et performant. Il s’intéresse à la réduction des fausses alarmes en identifiant les régions potentiellement cancérogènes. La gamme dynamique des niveaux de gris des zones sombres est étirée pour améliorer le contraste entre la région du sein et l'arrière plan et permettre une meilleure extraction de celle-ci. Toutefois, le muscle pectoral demeure incrusté dans la région du sein et interfère avec l'analyse des tissus. Son extraction est à la fois difficile et complexe à mettre en œuvre à cause de son chevauchement avec les tissus denses du sein. Dans ces conditions, même en exploitant l'information spatiale pendant la clusterisation par un algorithme de fuzzy C-means ne produit pas toujours des résultats de segmentation pertinents. Pour s'affranchir de cette difficulté, une étape de validation suivie d'un ajustement de contour est mise sur pied pour détecter et corriger les imperfections de segmentation. La seconde étape est consacrée à la caractérisation de la densité des tissus. Pour faire face au problème de variabilité des distributions de niveaux de gris dans les classes de densités, nous introduisons une modification de contraste basée sur un transport optimisé de niveaux de gris. Grâce à cette technique, la surface relative de tissus denses estimée par simple segmentation est très fortement corrélée aux classes de densités issues d’un jeu de données étiquetées. / Computer-aided diagnosis systems are currently at the heart of many clinical protocols since they significantly improve diagnosis making and therefore medical care. This research work therefore puts forward a hierarchical architecture for the design of a robust and efficient CAD tool for breast cancer detection. More precisely, it focuses on the reduction of false alarms rate through the identification of image regions of foremost interest i.e potential cancerous areas. The dynamic range of gray level intensities in dark regions is, first of all stretched to enhance the contrast between tissues and background and thus favors accurate breast region extraction. A second segmentation follows since pectoral muscle which regularly tampers breast tissue analysis remains inlaid in the foreground region. Extracting pectoral muscle tissues is both hard and challenging due to its overlap with dense tissues. In such conditions, even exploiting spatial information during the clustering process of the fuzzy C-means algorithm does not always produce a relevant segmentation. To overcome this difficulty, a new validation process followed by a refinement strategy is proposed to detect and correct the segmentation imperfections. The second macro-step is devoted to breast tissue density analysis. To address the variability in gray levels distributions with of mammographic density classes, we introduce an optimized gray level transport map for mammographic image contrast standardization. Thanks to this technique, dense region areas computed using simple thresholding are highly correlated to density classes from an annotated dataset.