Les progrès récents des réseaux de neurones artificiels (plus connus sous le nom d'apprentissage profond) ont permis d'améliorer l’état de l’art dans plusieurs domaines de la vision par ordinateur. Dans cette thèse, nous étudions des techniques d'apprentissage profond dans le cadre de l’analyse du genre et de l’âge à partir du visage humain. En particulier, deux problèmes complémentaires sont considérés : (1) la prédiction du genre et de l’âge, et (2) la synthèse et l’édition du genre et de l’âge.D’abord, nous effectuons une étude détaillée qui permet d’établir une liste de principes pour la conception et l’apprentissage des réseaux de neurones convolutifs (CNNs) pour la classification du genre et l’estimation de l’âge. Ainsi, nous obtenons les CNNs les plus performants de l’état de l’art. De plus, ces modèles nous ont permis de remporter une compétition internationale sur l’estimation de l’âge apparent. Nos meilleurs CNNs obtiennent une précision moyenne de 98.7% pour la classification du genre et une erreur moyenne de 4.26 ans pour l’estimation de l’âge sur un corpus interne particulièrement difficile.Ensuite, afin d’adresser le problème de la synthèse et de l’édition d’images de visages, nous concevons un modèle nommé GA-cGAN : le premier réseau de neurones génératif adversaire (GAN) qui produit des visages synthétiques réalistes avec le genre et l’âge souhaités. Enfin, nous proposons une nouvelle méthode permettant d’employer GA-cGAN pour le changement du genre et de l’âge tout en préservant l’identité dans les images synthétiques. Cette méthode permet d'améliorer la précision d’un logiciel sur étagère de vérification faciale en présence d’écarts d’âges importants. / The recent progress in artificial neural networks (rebranded as deep learning) has significantly boosted the state-of-the-art in numerous domains of computer vision. In this PhD study, we explore how deep learning techniques can help in the analysis of gender and age from a human face. In particular, two complementary problem settings are considered: (1) gender/age prediction from given face images, and (2) synthesis and editing of human faces with the required gender/age attributes.Firstly, we conduct a comprehensive study which results in an empirical formulation of a set of principles for optimal design and training of gender recognition and age estimation Convolutional Neural Networks (CNNs). As a result, we obtain the state-of-the-art CNNs for gender/age prediction according to the three most popular benchmarks, and win an international competition on apparent age estimation. On a very challenging internal dataset, our best models reach 98.7% of gender classification accuracy and an average age estimation error of 4.26 years.In order to address the problem of synthesis and editing of human faces, we design and train GA-cGAN, the first Generative Adversarial Network (GAN) which can generate synthetic faces of high visual fidelity within required gender and age categories. Moreover, we propose a novel method which allows employing GA-cGAN for gender swapping and aging/rejuvenation without losing the original identity in synthetic faces. Finally, in order to show the practical interest of the designed face editing method, we apply it to improve the accuracy of an off-the-shelf face verification software in a cross-age evaluation scenario.

Les modèles d’apprentissage profond à large capacité ont souvent tendance à présenter de hauts écarts de généralisation lorsqu’ils sont entrainés avec une quantité limitée de données étiquetées. Dans ce cas, des réseaux de neurones très profonds et larges auront tendance à mémoriser les échantillons de données et donc ils risquent d’être vulnérables lors d’un léger décalage dans la distribution des données au moment de tester. Ce problème produit une généralisation pauvre lors de changements dans la répartition des données au moment du test. Pour surmonter ce problème, certaines méthodes basées sur la dépendance et l’indépendance de données ont été proposées. Une récente classe de méthodes efficaces pour aborder ce problème utilise plusieurs manières de contruire un nouvel échantillon d’entrainement, en mixant une paire (ou plusieurs) échantillons d’entrainement. Dans cette thèse, nous introduisons PatchUp, une régularisation de l’espace des caractéristiques au niveau des blocs dépendant des données qui opère dans l’espace caché en masquant des blocs contigus parmi les caractéristiques mappées, sélectionnés parmi une paire aléatoire d’échantillons, puis en mixant (Soft PatchUp) ou en échangeant (Hard PatchUp) les blocs contigus sélectionnés. Notre méthode de régularisation n’ajoute pas de surcharge de calcul significative au CNN pendant l’entrainement du modèle. Notre approche améliore la robustesse des modèles CNN face au problème d’intrusion du collecteur qui pourrait apparaitre dans d’autres approches de mixage telles que Mixup et CutMix. De plus, vu que nous mixons des blocs contigus de caractéristiques dans l’espace caché, qui a plus de dimensions que l’espace d’entrée, nous obtenons des échantillons plus diversifiés pour entrainer vers différentes dimensions. Nos expériences sur les ensembles de données CIFAR-10, CIFAR-100, SVHN et Tiny-ImageNet avec des architectures ResNet telles que PreActResnet18, PreActResnet34, WideResnet-28-10, ResNet101 et ResNet152 montrent que PatchUp dépasse ou égalise les performances de méthodes de régularisation pour CNN considérée comme état de l’art actuel. Nous montrons aussi que PatchUp peut fournir une meilleure généralisation pour des transformations affines d’échantillons et est plus robuste face à des attaques d’exemples contradictoires. PatchUp aide aussi les modèles CNN à produire une plus grande variété de caractéristiques dans les blocs résiduels en comparaison avec les méthodes de pointe de régularisation pour CNN telles que Mixup, Cutout, CutMix, ManifoldMixup et Puzzle Mix. Mots clés: Apprentissage en profondeur, Réseau Neuronal Convolutif, Généralisation,Régularisation, Techniques de régularisation dépendantes et indépendantes des données, Robustesse aux attaques adverses. / Large capacity deep learning models are often prone to a high generalization gap when trained with a limited amount of labeled training data. And, in this case, very deep and wide networks have a tendency to memorize the samples, and therefore they might be vulnerable under a slight distribution shift at testing time. This problem yields poor generalization for data outside of the training data distribution. To overcome this issue some data-dependent and data-independent methods have been proposed. A recent class of successful methods to address this problem uses various ways to construct a new training sample by mixing a pair (or more) of training samples. In this thesis, we introduce PatchUp, a feature-space block-level data-dependent regularization that operates in the hidden space by masking out contiguous blocks of the feature map of a random pair of samples, and then either mixes (Soft PatchUp) or swaps (Hard PatchUp) these selected contiguous blocks. Our regularization method does not incur significant computational overhead for CNNs during training. Our approach improves the robustness of CNN models against the manifold intrusion problem that may occur in other state-of-the-art mixing approaches like Mixup and CutMix. Moreover, since we are mixing the contiguous block of features in the hidden space, which has more dimensions than the input space, we obtain more diverse samples for training towards different dimensions. Our experiments on CIFAR-10, CIFAR-100, SVHN, and Tiny-ImageNet datasets using ResNet architectures including PreActResnet18, PreActResnet34, WideResnet-28-10, ResNet101, and ResNet152 models show that PatchUp improves upon, or equals, the performance of current state-of-the-art regularizers for CNNs. We also show that PatchUp can provide a better generalization to affine transformations of samples and is more robust against adversarial attacks. PatchUp also helps a CNN model to produce a wider variety of features in the residual blocks compared to other state-of-the-art regularization methods for CNNs such as Mixup, Cutout, CutMix, ManifoldMixup, and Puzzle Mix. Key words: Deep Learning, Convolutional Neural Network, Generalization, Regular-ization, Data-dependent and Data-independent Regularization Techniques, Robustness to Adversarial Attacks.

<p>The most important networks from different domains—such as Computing, Organization, Economic, Social, Academic, and Biology—are networks that change over time. For example, in an organization there are email and collaboration networks (e.g., different people or teams working on a document). Apart from the connectivity of the networks changing over time, they can contain attributes such as the topic of an email or message, contents of a document, or the interests of a person in an academic citation or a social network. Analyzing these dynamic networks can be critical in decision-making processes. For instance, in an organization, getting insight into how people from different teams collaborate, provides important information that can be used to optimize workflows.</p> <p><br></p> <p>Network generative models provide a way to study and analyze networks. For example, benchmarking model performance and generalization in tasks like node classification, can be done by evaluating models on synthetic networks generated with varying structure and attribute correlation. In this work, we begin by presenting our systemic study of the impact that graph structure and attribute auto-correlation on the task of node classification using collective inference. This is the first time such an extensive study has been done. We take advantage of a recently developed method that samples attributed networks—although static—with varying network structure jointly with correlated attributes. We find that the graph connectivity that contributes to the network auto-correlation (i.e., the local relationships of nodes) and density have the highest impact on the performance of collective inference methods.</p> <p><br></p> <p>Most of the literature to date has focused on static representations of networks, partially due to the difficulty of finding readily-available datasets of dynamic networks. Dynamic network generative models can bridge this gap by generating synthetic graphs similar to observed real-world networks. Given that motifs have been established as building blocks for the structure of real-world networks, modeling them can help to generate the graph structure seen and capture correlations in node connections and activity. Therefore, we continue with a study of motif evolution in <em>dynamic</em> temporal graphs. Our key insight is that motifs rarely change configurations in fast-changing dynamic networks (e.g. wedges intotriangles, and vice-versa), but rather keep reappearing at different times while keeping the same configuration. This finding motivates the generative process of our proposed models, using temporal motifs as building blocks, that generates dynamic graphs with links that appear and disappear over time.</p> <p><br></p> <p>Our first proposed model generates dynamic networks based on motif-activity and the roles that nodes play in a motif. For example, a wedge is sampled based on the likelihood of one node having the role of hub with the two other nodes being the spokes. Our model learns all parameters from observed data, with the goal of producing synthetic graphs with similar graph structure and node behavior. We find that using motifs and node roles helps our model generate the more complex structures and the temporal node behavior seen in real-world dynamic networks.</p> <p><br></p> <p>After observing that using motif node-roles helps to capture the changing local structure and behavior of nodes, we extend our work to also consider the attributes generated by nodes’ activities. We propose a second generative model for attributed dynamic networks that (i) captures network structure dynamics through temporal motifs, and (ii) extends the structural roles of nodes in motifs to roles that generate content embeddings. Our new proposed model is the first to generate synthetic dynamic networks and sample content embeddings based on motif node roles. To the best of our knowledge, it is the only attributed dynamic network model that can generate <em>new</em> content embeddings—not observed in the input graph, but still similar to that of the input graph. Our results show that modeling the network attributes with higher-order structures (e.g., motifs) improves the quality of the networks generated.</p> <p><br></p> <p>The generative models proposed address the difficulty of finding readily-available datasets of dynamic networks—attributed or not. This work will also allow others to: (i) generate networks that they can share without divulging individual’s private data, (ii) benchmark model performance, and (iii) explore model generalization on a broader range of conditions, among other uses. Finally, the evaluation measures proposed will elucidate models, allowing fellow researchers to push forward in these domains.</p>

[pt] Esta dissertação estuda métodos e algoritmos para critérios de seleções dos retângulos envolventes focando em arquiteturas de detecção de objetos baseada redes neurais convolucionais para tempo real, que processam mais de 30fps, que também possibilitam a expansão para outras arquiteturas. O objetivo desta dissertação é melhorar as métricas Recall e Precision, proporcionando mais assertividade nos resultados destas arquiteturas sem a necessidade de recriá-las ou retreiná-las, diminuindo, assim, os recursos para manutenções. As arquiteturas que trabalham em tempo real normalmente não apresentam melhores resultados, pois são desenvolvidas visando a redução do tempo de execução. Para resolver estes problemas, serão testados outros métodos de critérios de seleção de retângulos envolventes em estado da arte, são eles: Nonmaximum Suppression (NMS), Soft-NMS, Non-Maximum Weighted (NMW) e Weighted Boxes Fusion (WBF). Os resultados obtidos foram comparados aos originais das arquiteturas, utilizando as métricas mAP, Recall e Precision. Através desta comparação foi possível comprovar que os novos critérios apresentaram bons resultados. O tempo de execução dos novos critérios também foi analisado com execuções de imagens em lotes, contornando alguns overheads dos critérios mais pesados. As arquiteturas utilizadas como base nos experimentos foram baseadas nos sistemas YOLOv3-Tiny e YOLOv4-Tiny, utilizando o dataset QMUL-OpenLogo público e especializado em logotipos e baseado em fotos reais. / [en] This dissertation studies methods and algorithms for bounding box selection criteria focusing on object detection architectures based on convolutional neural networks for real-time, processing over 30fps, which also allow expansion to other architectures. The goal of this study is to improve the Recall and Precision metrics, providing more assertiveness in the results of these architectures without the need to recreate or retrain them, thus reducing the resources for maintenance. Architectures that work in real-time usually do not present good results, because they are developed aiming to reduce execution time. To solve these problems, other state-of-the-art bounding box selection criteria methods will be tested: Non-maximum Suppression (NMS), Soft-NMS, Non-Maximum Weighted (NMW) and Weighted Boxes Fusion (WBF). The results obtained were compared to the original architectures, using the mAP, Recall and Precision metrics. Through this comparison it was possible to prove that the new criteria presented satisfactory results. The execution time of the new criteria was also analyzed with batch image executions, bypassing some overheads of the heavier criteria. The architectures used as basis for the experiments were based on the YOLOv3-Tiny and YOLOv4-Tiny systems, using the public dataset QMUL-OpenLogo specialized in logos and based on real photos.

Recently, Deep Learning has revolutionized many fields, where one such area is Voice Activity Detection (VAD). This is of great interest to sectors of society concerned with detecting speech in sound signals. One such sector is the police, where criminal investigations regularly involve analysis of audio material. Convolutional Neural Networks (CNN) have recently become the state-of-the-art method of detecting speech in audio. But so far, understanding the impact of noise and sampling rates on such methods remains incomplete. Additionally, there are evaluation metrics from neighboring fields that remain unintegrated into VAD. We trained on four different sampling rates and found that changing the sampling rate could have dramatic effects on the results. As such, we recommend explicitly evaluating CNN-based VAD systems on pertinent sampling rates. Further, with increasing amounts of white Gaussian noise, we observed better performance by increasing the capacity of our Gated Recurrent Unit (GRU). Finally, we discuss how careful consideration is necessary when choosing a main evaluation metric, leading us to recommend Polyphonic Sound Detection Score (PSDS).

The aim of this study was to elaborate the possibilities of sentiment analyzing Swedish news article titles using machine learning approaches and find how the text is best represented in such conditions. Sentiment analysis has traditionally been conducted by part-of-speech tagging and counting word polarities, which performs well for large domains and in absence of large sets of training data. For narrower domains and previously labeled data, supervised learning can be used. The work of this thesis tested the performance of a convolutional neural network and a Support Vector Machine on different sets of data. The data sets were constructed to represent various language features. This included for example a simple unigram bag-of-words model storing word counts, a bigram bag-of-words model to include the ordering of words and an integer vector summary of the title. The study concluded that each of the tested feature sets gave information about the sentiment to various extents. The neural network approach with all feature sets combined performed better than the two annotators of the study. Despite the limited data set, overfitting did not seem to be a problem when using the features together. / Målet med detta arbete var att undersöka möjligheten till sentimentanalys av svenska nyhetsrubriker med hjälp av maskininlärning och förstå hur dessa rubriker bäst representeras. Sentimentanalys har traditionellt använt ordklassmärkning och räknande av ordpolariteter, som fungerar bra för stora domäner där avsaknaden av större uppmärkt träningsdata är stor. För mindre domäner och tidigare uppmärkt data kan övervakat lärande användas. Inom ramen för detta arbete undersöktes ett artificiellt neuronnät med faltning och en stödvektormaskin på olika datamängder. Datamängderna formades för att representera olika språkegenskaper. Detta inkluderade bland annat en enkel ordräkningsmodell, en bigramräkningsmodell och en heltalssummering av generella egenskaper för rubriken. I studien dras slutsatsen att varje datamängd innebar att ny information kunde tillföras i olika stor utsträckning. Det artificiella neuronnätet med alla datamängder tillsammans presterade bättre än de två personer som märkte upp data till denna studie. Trots en begränsad datamängd inträffade verkade inte modellerna övertränas.

Machine learning is rapidly being integrated into all areas of society, however, that puts a lot of pressure on resource costraint hardware such as embedded systems. The company Ericsson is gradually integrating machine learning based on neural networks, so-called deep learning, into their radio products. One promising product is their vectorized Digital Signal Processor (DSP) that are based upon the machine learning suitable Single Instruction, Multiple Data (SIMD) paradigm and Very Long Instruction Word (VLIW) architecture. However, despite the suitability of the SIMD paradigm, the embedded system needs to efficiently execute a computation-intensive deep learning algorithm with proper use of its limited resources. Therefore commonly used methods of implementing each layer of the computation-intensive Convolutional Neural Network (CNN), a type of Deep Neural Network (DNN), have been used and evaluated its implementation on the hardware and to assess the vectorized DSP’s deep learning suitability and capabilities. Despite the suitability of the hardware, the implementation utilized less than half of the available resources at all times during the execution. The main limitations were identified to be the limited 16-bit element instructions. To enhance the performance and improve the utilization of the available resources, easy-to-implement hardware instructions have been suggested. This work has made the first steps of implementing an efficiently performing CNN implementation on the examined vectorized DSP. / Integreringen av maskininlärning in i alla samhällsområden sker idag i rusande fart, men det sätter stor press på begränsad hårdvara som inbyggda system. Företaget Ericsson integrerar successivt maskininlärning baserad på neurala nätverk, så kallad djupinlärning, i sina radioprodukter. En lovande produkt är deras vektoriserade DSP som är baserade på maskininlärningspasset SIMD-paradigm och VLIW-arkitektur. Men trots lämpligheten av SIMD-paradigmet, är den största utmaningen att utnyttja de begränsade resurserna i inbyggda systemet för att effektivt exekvera en beräkningsintensiv djupinlärningsalgoritm. Därför har vanligt använda metoder för att implementera varje lager av den beräkningsintensiva CNN, en typ av DNN, använts och utvärderats på hårdvaran för att bedöma den vektoriserade DSP:s djupinlärningslämplighet samt förmågor. Trots hårdvarans lämplighet använde alla implementeringar mindre än hälften av de tillgängliga resurserna vid alla tidpunkter under exekveringen. De huvudsakliga begränsningarna identifierades vara den begränsade tillgången på 16-bitars element instruktioner. För att förbättra prestandan för ett närmare fullt utnyttjande av tillgängliga resurser har hårdvaruinstruktioner som är enkla att implementera föreslagits. Detta arbete har tagit de första stegen för att implementera ett effektivt förformande CNN på den undersökta vekotriserade DSP.

Passive Intermodulation is a type of distortion affecting the sensitive receiving signals in a cellular network, which is a growing problem in the telecommunication field. One way to mitigate this problem is through Passive Intermodulation Cancellation, where the predicted noise in a signal is modeled with polynomials. Recent experiments using neural networks instead of polynomials to model this noise have shown promising results. However, one drawback with neural networks is their lack of explainability. In this work, we identify a suitable method that provides explanations for this use case. We apply this technique to explain the neural networks used for Passive Intermodulation Cancellation and discuss the result with domain expertise. We show that the input space as well as the architecture could be altered, and propose an alternative architecture for the neural network used for Passive Intermodulation Cancellation. This alternative architecture leads to a significant reduction in trainable parameters, a finding which is valuable in a cellular network where resources are heavily constrained. When performing an explainability analysis of the alternative model, the explanations are also more in line with domain expertise. / Passiv Intermodulation är en typ av störning som påverkar de känsliga mottagarsignalerna i ett mobilnät. Detta är ett växande problem inom telekommunikation. Ett tillvägagångssätt för att motverka detta problem är genom passiv intermodulations-annullering, där störningarna modelleras med hjälp av polynomiska funktioner. Nyligen har experiment där neurala nätverk används istället för polynomiska funktioner för att modellera dessa störningar påvisat intressanta resultat. Användandet av neurala nätverk är dock förenat med vissa nackdelar, varav en är svårigheten att tyda och tolka utfall av neurala nätverk. I detta projekt identifieras en passande metod för att erbjuda förklaringar av neurala nätverk tränade för passiv intermodulations-annullering. Vi applicerar denna metod på nämnda neurala nätverk och utvärderar resultatet tillsammans med domänexpertis. Vi visar att formatet på indatan till neurala nätverket kan manipuleras, samt föreslår en alternativ arkitektur för neurala nätverk tränade för passiv intermodulations-annullering. Denna alternativa arkitektur innebär en avsevärd reduktion av antalet träningsbara parametrar, vilket är ett värdefullt resultat i samband med mobilnät där det finns kraftiga begränsningar på hårdvaruresurser. När vi applicerar metoder för att förklara utfall av denna alternativa arkitektur finner vi även att förklaringarna bättre motsvarar förväntningarna från domänexpertis.

Epilepsy affects more than 50 million people and is one of the most prevalent neurological disorders and has a high impact on the quality of life of those suffering from it. However, 70% of epilepsy patients can live seizure free with proper diagnosis and treatment. Patients are evaluated using scalp EEG recordings which is cheap and non-invasive. Diagnostic yield is however low and qualified personnel need to process large amounts of data in order to accurately assess patients. MindReader is an unsupervised classifier which detects spectral anomalies and generates a hypothesis of the underlying patient state over time. The aim is to highlight abnormal, potentially epileptiform states, which could expedite analysis of patients and let qualified personnel attest the results. It was used to evaluate 95 scalp EEG recordings from healthy adults and adult patients with epilepsy. Interictal Epileptiform discharges (IED) occurring in the samples had been retroactively annotated, along with the patient state and maneuvers performed by personnel, to enable characterization of the classifier’s detection performance. The performance was slightly worse than previous benchmarks on pediatric scalp EEG recordings, with a 7% and 33% drop in specificity and sensitivity, respectively. Electrode positioning and partial spatial extent of events saw notable impact on performance. However, no correlation between annotated disturbances and reduction in performance could be found. Additional explorative analysis was performed on serialized intermediate data to evaluate the analysis design. Hyperparameters and electrode montage options were exposed to optimize for the average Mathew’s correlation coefficient (MCC) per electrode per patient, on a subset of the patients with epilepsy. An increased window length and lowered amount of training along with an common average montage proved most successful. The Euclidean distance of cumulative spectra (ECS), a metric suitable for spectral analysis, and homologous L2 and L1 loss function were implemented, of which the ECS further improved the average performance for all samples. Four additional analyses, featuring new time-frequency transforms and multichannel convolutional autoencoders were evaluated and an analysis using the continuous wavelet transform (CWT) and a convolutional autoencoder (CNN) performed the best, with an average MCC score of 0.19 and 56.9% sensitivity with approximately 13.9 false positives per minute.

<p dir="ltr">Detecting anomalies in spacecraft time-series data is a high priority, especially considering the harshness of the spacecraft operating environment. These anomalies often function as precursors for system failure. Traditionally, the time-series data channels are monitored manually by domain experts, which is time-consuming. Additionally, there are thousands of channels to monitor. Machine learning methods have proven to be useful for automatic anomaly detection, but a unique model must be trained from scratch for each time-series. This thesis proposes three approaches for reducing training costs. First, a transfer learning approach that finetunes a general pre-trained model to reduce training time and the number of unique models required for a given spacecraft. The second and third approaches both use online learning to reduce the amount of training data and time needed to identify anomalies. The second approach leverages an ensemble of extreme learning machines while the third approach uses deep learning models. All three approaches are shown to achieve reasonable anomaly detection performance with reduced training costs.</p><p dir="ltr">Measuring the phenotypes, or observable traits, of a plant enables plant scientists to understand the interaction between the growing environment and the genetic characteristics of a plant. Plant phenotyping is typically done manually, and often involves destructive sampling, making the entire process labor-intensive and difficult to replicate. In this thesis, we use image processing for characterizing two different disease progressions. Tar spot disease can be identified visually as it induces small black circular spots on the leaf surface. We propose using a Mask R-CNN to detect tar spots from RGB images of leaves, thus enabling rapid non-destructive phenotyping of afflicted plants. The second disease, bacteria-induced wilting, is measured using a visual assessment that is often subjective. We design several metrics that can be extracted from RGB images that can be used to generate consistent wilting measurements with a random forest. Both approaches ensure faster, replicable results, enabling accurate, high-throughput analysis to draw conclusions about effective disease treatments and plant breeds.</p>