Philosophiae Doctor - PhD / This research investigated the application of recurrent neural networks (RNNs) for recognition of facial expressions based on facial action coding system (FACS). Support vector machines (SVMs) were used to validate the results obtained by RNNs. In this approach, instead of recognizing whole facial expressions, the focus was on the recognition of action units (AUs) that are defined in FACS. Recurrent neural networks are capable of gaining knowledge from temporal data while SVMs, which are time invariant, are known to be very good classifiers. Thus, the research consists of four important components: comparison of the use of image sequences against single static images, benchmarking feature selection and network optimization approaches, study of inter-AU correlations by implementing multiple output RNNs, and study of difference images as an approach for performance improvement. In the comparative studies, image sequences were classified using a combination of Gabor filters and RNNs, while single static images were classified using Gabor filters and SVMs. Sets of 11 FACS AUs were classified by both approaches, where a single RNN/SVM classifier was used for classifying each AU. Results indicated that classifying FACS AUs using image sequences yielded better results than using static images. The average recognition rate (RR) and false alarm rate (FAR) using image sequences was 82.75% and 7.61%, respectively, while the classification using single static images yielded a RR and FAR of 79.47% and 9.22%, respectively. The better performance by the use of image sequences can be at- tributed to RNNs ability, as stated above, to extract knowledge from time-series data. Subsequent research then investigated benchmarking dimensionality reduction, feature selection and network optimization techniques, in order to improve the performance provided by the use of image sequences. Results showed that an optimized network, using weight decay, gave best RR and FAR of 85.38% and 6.24%, respectively. The next study was of the inter-AU correlations existing in the Cohn-Kanade database and their effect on classification models. To accomplish this, a model was developed for the classification of a set of AUs by a single multiple output RNN. Results indicated that high inter-AU correlations do in fact aid classification models to gain more knowledge and, thus, perform better. However, this was limited to AUs that start and reach apex at almost the same time. This suggests the need for availability of a larger database of AUs, which could provide both individual and AU combinations for further investigation. The final part of this research investigated use of difference images to track the motion of image pixels. Difference images provide both noise and feature reduction, an aspect that was studied. Results showed that the use of difference image sequences provided the best results, with RR and FAR of 87.95% and 3.45%, respectively, which is shown to be significant when compared to use of normal image sequences classified using RNNs. In conclusion, the research demonstrates that use of RNNs for classification of image sequences is a new and improved paradigm for facial expression recognition.

Masters of Science / This research investigates the creation of an audio-shape recognition system that is able to interpret a user’s drawn audio shapes—fundamental shapes, digits and/or letters— on a given surface such as a table-top using a generic stylus such as the back of a pen. The system aims to make use of one, two or three Piezo microphones, as required, to capture the sound of the audio gestures, and a combination of the Mel-Frequency Cepstral Coefficients (MFCC) feature descriptor and Support Vector Machines (SVMs) to recognise audio shapes. The novelty of the system is in the use of piezo microphones which are low cost, light-weight and portable, and the main investigation is around determining whether these microphones are able to provide sufficiently rich information to recognise the audio shapes mentioned in such a framework.

La thèse porte sur l'intégration d éléments explicatifs au sein d'un modèle de classification. Plus précisément la solution proposée se compose de la combinaison entre un algorithme de chemin de régularisation appelé DRSVM et une approche noyau appelée KERNEL BASIS. La première partie de la thèse consiste en l'amélioration d'un algorithme appelé DRSVM à partir d'une reformulation du chemin via la théorie de la sous-différentielle. La seconde partie décrit l'extension de l'algorithme DRSVM au cadre KERNEL BASIS via une approche dictionnaire. Enfin une série d'expérimentation sont réalisées afin de valider l'aspect interprétable du modèle. / This thesis aims at finding classification rnodeIs which include explanatory elements. More specifically the proposed solution consists in merging a regularization path algorithm called DRSVM with a kernel approach called KERNEL BASIS. The first part of the thesis focuses on improving an algorithm called DRSVM from a reformulation of the thanks to the suh-differential theory. The second part of the thesis describes the extension of DRSVM afgorithm under a KERNEL BASIS framework via a dictionary approach. Finally, a series of experiments are conducted in order to validate the interpretable aspect of the rnodel.

In this thesis, the performance of the classification methods Linear Discriminant Analysis (LDA), Random Forests (RF), and Support Vector Machines (SVM) are compared using procurement data to predict what size company will win a procurement. This is useful information for companies, since bidding on a procurement takes time and resources, which they can save if they know their chances of winning are low. The data used in the models are collected from OpenTender and allabolag.se and represent procurements that were awarded to companies in 2020. A total of 8 models are created, two versions of the LDA model, two versions of the RF model, and four versions of the SVM model, where some models are more complex than others. All models are evaluated on overall performance using hit rate, Huberty’s I Index, mean average error, and Area Under the Curve. The most complex SVM model performed the best across all evaluation measurements, whereas the less complex LDA model performed overall worst. Hit rates and mean average errors are also calculated within each class, and the complex SVM models performed best on all company sizes, except the small companies which were best predicted by the less complex Random Forest model.

This master thesis describes a journaling system for compression ultrasonography and a clinical assessment system for deep vein thrombosis (DVT). We evaluate Support Vector Machines (SVM) models with linear- and radial basis function-kernels for predicting deep vein thrombosis, and for facilitating creation of new clinical DVT assessment. Data from 159 patients where analysed, with our dataset, Wells Score with a high clinical probability have an accuracy of 58%, sensitivity 60% and specificity of 57% these figured should be compared to those of our base models accuracy of 81%, sensitivity 66% and specificity 84%. A 23 percentage point increase in accuracy.The diagnostic odds ratio went from 2.12 to 11.26. However a larger dataset is required to report anything conclusive. As our system is both a journaling and prediction system, every patient examined helps the accuracy of the assessment. / I denna rapport beskrivs ett journalsystem samt ett system för klinisk bedömning av djupvenstromboser.Vår modell baserar sig på en stödvektormaskin (eng. Support Vector Machine) med linjär och radial basfunktion för att fastställa förekomsten av djupa ventromboser samt att hjälpa till i skapandet av nya modeller för bedömning. 159 patientjournaler användes för att fastställa att Wells Score har en klinisk precision på 58%, 60% sensitivitet och specificitet på 57% somkan jämföras med våran modell som har en precision på 81%, 66% sensitivitet och specificitet på 84%. En 23 procentenheters ökning i precision.Den diagnostiska oddskvoten gick från 2.12 till 11.26. Det behövs dock en större datamängd för att rapportera något avgörande. Då vårt system både är för journalskapande och klinisk bedömning så kommer varje undersökt patient att bidra till högre precision i modellen.

Background. Hyperplane folding was introduced by Lars Lundberg et al. in Hyperplane folding increased the margin while suffering from a flaw, referred to asover-rotation in this thesis. The aim of this thesis is to introduce a new different technique thatwould not over-rotate data points. This novel technique is referred to as RubberBand folding in the thesis. The following research questions are addressed: 1) DoesRubber Band folding increases classification accuracy? 2) Does Rubber Band fold-ing increase the Margin? 3) How does Rubber Band folding effect execution time? Rubber Band folding was implemented and its result was compared toHyperplane folding and the Support-vector machine. This comparison was done byapplying Stratified ten-fold cross-validation on four data sets for research question1 & 2. Four folds were applied for both Hyperplane folding and Rubber Band fold-ing, as more folds can lead to over-fitting. While research question 3 used 15 folds,in order to see trends and is not affected by over-fitting. One BMI data set, wasartificially made for the initial Hyperplane folding paper. Another data set labeled patients with, or without a liver disorder. Another data set predicted if patients havebenign- or malign cancer cells. Finally, a data set predicted if a hepatitis patient isalive within five years.Results.Rubber Band folding achieved a higher classification accuracy when com-pared to Hyperplane folding in all data sets. Rubber Band folding increased theclassification in the BMI data set and cancer data set while the accuracy for Rub-ber Band folding decreased in liver and hepatitis data sets. Hyperplane folding’saccuracy decreased in all data sets.Both Rubber Band folding and Hyperplane folding increases the margin for alldata sets tested. Rubber Band folding achieved a margin higher than Hyperplanefolding’s in the BMI and Liver data sets. Execution time for both the classification ofdata points and the training time for the classifier increases linearly per fold. RubberBand folding has slower growth in classification time when compared to Hyperplanefolding. Rubber Band folding can increase the classification accuracy, in whichexact cases are unknown. It is howevered believed to be when the data is none-linearly seperable.Rubber Band folding increases the margin. When compared to Hyperplane fold-ing, Rubber Band folding can in some cases, achieve a higher increase in marginwhile in some cases Hyperplane folding achieves a higher margin.Both Hyperplane folding and Rubber Band folding increases training time andclassification time linearly. The difference between Hyperplane folding and RubberBand folding in training time was negligible while Rubber bands increase in classifi-cation time was lower. This was attributed to Rubber Band folding rotating fewerpoints after 15 folds.

In the light of the ongoing loss of biodiversity at the global scale, monitoring grasslands is nowadays of utmost importance considering their functional relevance in terms of the ecosystem services that they provide. Here, guidelines of the European Union like the Fauna-Flora-Habitat Directive and the European Agricultural fund for Rural Development with its HNV indicators are crucial. Indeed, they form the legal framework for nature conservation and define grasslands as one of their conservation targets, whose status needs to be assessed and reported by all member states on a regular basis. In the light of these reporting requirements, the need for a harmonised and thorough grassland monitoring is highly demanding since most member states are still currently adopting intensive field surveys or photointerpretation with differing levels of detail for mapping habitat distribution. To this purpose, a cost-effective solution is offered by Earth Observation data for which specific grassland monitoring methodologies shall be then implemented which are capable of processing multitemporal acquisitions collected throughout the entire growing season. Although optical data are most suited for characterising vegetation in terms of spectral information content, they are actually subject to weather conditions (especially cloud coverage), which hinder the possibility of collecting enough information over the full phenological cycle. Furthermore, so far only few studies started employing high and very high resolution optical time series for grassland habitat monitoring since they have become available e.g., from the RapidEye satellites, only in the recent past. To overcome this limitation, SAR systems can be employed which provide imagery independent from weather or daytime conditions, hence enabling vegetation analysis by means of complete time series. Compared to optical data, radar imagery is less affected by the physical-chemical characteristics of the surface, but rather it is sensitive to structural features like geometry and roughness. However, in this context presently only very few techniques have been implemented, which are anyhow not suitable to be employed in an operational framework. Furthermore, to address the classification task, supervised approaches (which require in situ information for all the land-cover classes present in the study area) represent the most accurate methodological solution; nevertheless, collecting an exhaustive ground truth is generally expensive both in terms of time and economic costs and is not even feasible when the test site is remote. However, in many applications the end-users are generally only interested in very few specific targeted land-cover classes which, for instance, have high ecological value or are associated with support actions, subsidies or benefits from national or international institutions. The categorisation of specific grasslands and habitat types as those addressed in this thesis falls within such category of problems, which is defined in the literature as targeted land-cover classification. In this framework, a robust and effective targeted classification system for the automatic identification of grassland types by means of multi-temporal and multi-polarised SAR data has been developed within this thesis. In particular, the proposed system is composed of three main blocks: the preprocessing of the SAR image time series including the Kennaugh decomposition, the feature extraction including multi-temporal filtering and texture analysis, and the hierarchical targeted classification, which consist of two phases where first a one-class classifier is employed to outline the merger of all the grassland types of interest considered as a single information class and then a multi-class classifier is applied for discriminating the specific targeted classes within the areas identified as positive by the one-class classifier. To evaluate the capabilities of the proposed methodology, several experimental trials have been carried out over two test sites located in Southern Bavaria (Germany) and Mecklenburg Western-Pomerania (Germany) for which six diverse datasets have been derived from multitemporal series of dualpol TerraSAR-X as well as dual-/quadpol Radarsat-2 images. Four among the Natura 2000 habitat types of the Fauna-Flora-Habitat Directive as well all High Nature Value grassland types have been considered as targeted classes for this study. Overall, the proposed system proved to be robust and confirmed the effectiveness of employing multitemporal and multi-polarisation VHR SAR data for discriminating habitat types and High Nature Value grassland types, exhibiting high potential for future employment even at larger scales. In particular, it could be demonstrated that the proposed hierarchical targeted classification approach outperforms the available state-of-the-art methods and has a clear advantage with respect to the standard approaches in terms of robustness, reliability and transferability.

In this work, we investigate different techniques to predict the monthly trend direction of the S&P 500 market index. The techniques use a machine learning classifier with technical and macroeconomic indicators as input features. The Support Vector Machine (SVM) classifier was explored in-depth in order to optimize the performance using four different kernels; Linear, Radial Basis Function (RBF), Polynomial, and Quadratic. A result found was the performance of the classifier can be optimized by reducing the number of macroeconomic features needed by 30% using Sequential Feature Selection. Further performance enhancement was achieved by optimizing the RBF kernel and SVM parameters through gridsearch. This resulted in final classification accuracy rates of 62% using technical features alone with gridsearch and 60.4% using macroeconomic features alone using Rankfeatures

Tranquilizer misuse is a behavior associated with substance use disorder. As of now there is only one published article that includes a predictive model on misuse of subscription tranquilizers. The aim of this study is to predict ongoing tranquilizer misuse whilst comparing three different methods of classification; (1) regularized logistic regression, (2) adaptive boosting and (3) support vector machines. Data from the National Survey of Drug Use and Health (NSDUH) from 2019 is used to predict misuse among the individuals in the sample from 2020. The regularized logistic regression and the support vector machines models both yield an AUC of 0.88, which is slightly higher than the adaptive boosting model. However, the support vector machine model yields a higher level of sensitivity, meaning that it is better at detecting individuals who misuse. Although the difference in performance between the methods is relatively small and is most likely caused by the fact that different methods perform differently depending on the characteristics of the data.

This dissertation material provided in this work details the techniques that are developed to aid in the Classification of tumors, non-tumors, and dense masses in a Mammogram, certain characteristics such as texture in a mammographic image are used to identify the regions of interest as a part of classification. Pattern recognizing techniques such as nearest mean classifier and Support vector machine classifier are also used to classify the features. The initial stages include the processing of mammographic image to extract the relevant features that would be necessary for classification and during the final stage the features are classified using the pattern recognizing techniques mentioned above. The goal of this research work is to provide the Medical Experts and Researchers an effective method which would aid them in identifying the tumors, non-tumors, and dense masses in a mammogram. At first the breast region extraction is carried using the entire mammogram. The extraction is carried out by creating the masks and using those masks to extract the region of interest pertaining to the tumor. A chain code is employed to extract the various regions, the extracted regions could potentially be classified as tumors, non-tumors, and dense regions. Adaptive histogram equalization technique is employed to enhance the contrast of an image. After applying the adaptive histogram equalization for several times which will provide a saturated image which would contain only bright spots of the mammographic image which appear like dense regions of the mammogram. These dense masses could be potential tumors which would need treatment. Relevant Characteristics such as texture in the mammographic image are used for feature extraction by using the nearest mean and support vector machine classifier. A total of thirteen Haralick features are used to classify the three classes. Support vector machine classifier is used to classify two class problems and radial basis function (RBF) kernel is used to find the best possible (c and gamma) values. Results obtained in this research suggest the best classification accuracy was achieved by using the support vector machines for both Tumor vs Non-Tumor and Tumor vs Dense masses. The maximum accuracies achieved for the tumor and non-tumor is above 90 % and for the dense masses is 70.8% using 11 features for support vector machines. Support vector machines performed better than the nearest mean majority classifier in the classification of the classes. Various case studies were performed using two distinct datasets in which each dataset consisting of 24 patients’ data in two individual views. Each patient data will consist of both the cranio caudal view and medio lateral oblique views. From these views the region of interest which could possibly be a tumor, non-tumor, or a dense regions(mass).