Daugelio klasių atpažinimo uždaviniams spręsti yra sukurta aibė sprendimų ir ne visada vieningų rekomendacijų. Dauguma jų paremta empiriniais bandymais, retai atsižvelgiama į statistines duomenų savybes. Dėl to sprendžiant daugelio klasių klasifikavimo uždavinį kyla klausimų, kurį metodą ir kada geriausia naudoti, koks vieno ar kito metodo patikimumas. Disertacijoje nagrinėjami dviejų pakopų sprendimo priėmimo metodai, kai pirmame etape sudaromi klasifikatoriai poroms (angl. pair-wise), sugebantys geriau išnaudoti klasių tarpusavio statistines savybes, o kitame etape yra atliekamas klasifikatorių poroms rezultatų apjungimas. Tyrime ypatingas dėmesys yra skiriamas klasifikatorių poroms sudėtingumui, mokymo duomenų kiekiui bei algoritmų kokybės įvertinimo tikslumui. Tikslumas labai priklauso nuo duomenų bei atliktų eksperimentų kiekio (duomenų permaišymo klasėse, juos skirstant į mokymo ir testavimo). Parodyta, jog dėl žemo įvertinimo tikslumo kai kurių publikuotų algoritmų deklaruojamas pranašumas prieš žinomus algoritmus nėra patikimas. Darbe atliktas detalus žinomų metodų palyginimas bei pristatytas naujai sukurtas klasifikatorių poroms apjungimo algoritmas, kuris yra paremtas analogišku algoritmu daugelio klasių klasifikatorių rezultatų apjungimui. Pateiktos bendros rekomendacijos, kaip projektuotojui elgtis daugelio klasių atveju. Pasiūlyti metodai, leidžiantys sumažinti klasifikavimo klaidą atliekant klasifikatorių poroms apjungimo koregavimą, kad algoritmas nebūtų... [toliau žr. visą tekstą] / There are plenty of solutions for the task of multi-class recognition. Unfortunately, these solutions are not always unanimous. Most of them are based on empirical experiments while statistical data features consideration is often omitted. That’s why questions like when and which method should be used, what the reliability of any chosen method is for solving a multi-class recognition task arise. In this dissertation two-stage multi-class decision methods are analyzed. Pair-wise classifiers able to better exploit statistical data features are used in the first stage of such methods. In the second stage a particular fusion rule of the first stage results is used to fuse the first stage results in order to produce the final classification decision. Complexity issues of pair-wise classifiers, training data size and precision of method quality estimation are pointed out in the research. The precision of algorithm highly depends on the data and the number of experiments performed (data permutation, division into training and testing data). It is shown that the declared superiority of some known algorithms is not reliable due to low precision of estimation. A detailed comparison of well known multi-class classification methods is performed and a new pair-wise classifier fusion method based on similar method used in multi-class classifier fusion is presented. The recommendations for multi-class classification task designer are provided. Methods which allow reducing classification... [to full text]

This study is on the service models for dynamic scheduling of multi-class make-to-stock systems. An exponential single-server facility processes different types of items one by one and demand arrivals for different item types occur according to independent Poisson processes. Inventories of the items are managed by base-stock policies and backordering is allowed. The objective is to minimize base-stock investments or average inventory holding costs subject to a constraint on the aggregate fill rate, which is a weighted average of the fill rates of the item types. The base-stock controlled policy that maximizes aggregate fill rate is numerically investigated, for both symmetric and asymmetric systems, and is shown to be optimal for minimizing base-stock investments under an aggregate fill rate constraint. Alternative policies are generated by heuristics in order to approximate the policy that maximizes aggregate fill rate and performances of these policies are compared to those of two well-known Longest Queue and First Come First Served policies. Also, optimal policy for the service model to minimize average inventory holding cost subject to an aggregate fill rate constraint is investigated without restricting the attention to only base-stock controlled dynamic scheduling policies. Based on the equivalence relations between this service model and the corresponding cost model, it is observed that the base-stock controlled policy that maximizes aggregate fill rate is almost the same as the solution to the service model and cost model under consideration, especially when backorder penalties are large in the cost model as compared to cost parameters for inventory holding or equivalently when the target fill rate is large in the service model.

Labeling of trademark images with Vienna codes from the Vienna classification is a manual process carried out by domain experts, which enables searching trademark image databases using specific keywords that describe the semantic meaning of the figurative elements. In this research, we are investigating how application of supervised learning algorithms can improve and automate the manual process of labeling of new un-labeled trademark images. The successful implementation of deep learning algorithms in the task of computer vision for image classification has motivated us to investigate which of the supervised learning algorithms performs better trademark image classification. More specifically, to solve the problem of identification of figurative elements in new un-labeled images, we have used multi-class image classification approach based on deep learning and machine learning. To address this problem, we have generated a unique benchmarking dataset composed of 14,500 unique logos extracted from the European Union Intellectual Property Office Open Data Portal. The results after executing a set of controlled experiments on the given dataset indicate that deep learning models have overall better performance than machine learning models. In particular, CNN models reach better accuracy and precision, and significantly higher recall and F1 score for shorter training times, compared to recurrent neural networks such as LSTMs and GRUs. From the machine learning models, results indicate that Support Vector Machines have higher accuracy and overall better performance time compared to Decision Trees, Random Forests and Naïve Bayes models. This study shows that deep learning models can solve the problem of the labeling of trademark images with Vienna codes, and that can be applied by Intellectual Property Offices in real-world application for automation of the classification task which is carried out manually by the domain experts.

No description available.

Online supervised learning from fast-evolving imbalanced data streams has applications in many areas. That is, the development of techniques that are able to handle highly skewed class distributions (or 'class imbalance') is an important area of research in domains such as manufacturing, the environment, and health. Solutions should be able to analyze large repositories in near real-time and provide accurate models to describe rare classes that may appear infrequently or in bursts while continuously accommodating new instances. Although numerous online learning methods have been proposed to handle binary class imbalance, solutions suitable for multi-class streams with varying degrees of imbalance in evolving streams have received limited attention. To address this knowledge gap, the first contribution of this thesis introduces the Online Learning from Imbalanced Multi-Class Streams through Dynamic Sampling (DynaQ) algorithm for learning in such multi-class imbalanced settings. Our approach utilizes a queue-based learning method that dynamically creates an instance queue for each class. The number of instances is balanced by maintaining a queue threshold and removing older samples during training. In addition, new and rare classes are dynamically added to the training process as they appear. Our experimental results confirm a noticeable improvement in minority-class detection and classification performance. A comparative evaluation shows that the DynaQ algorithm outperforms the state-of-the-art approaches. Our second contribution in this thesis focuses on fairness-aware learning from imbalanced streams. Our work is motivated by the observation that the decisions made by online learning algorithms may negatively impact individuals or communities. Indeed, the development of approaches to handle these concerns is an active area of research in the machine learning community. However, most existing methods process the data in offline settings and are not directly suitable for online learning from evolving data streams. Further, these techniques fail to take the effects of class imbalance, on fairness-aware supervised learning into account. In addition, recent fairness-aware online learning supervised learning approaches focus on one sensitive attribute only, which may lead to subgroup discrimination. In a fair classification, the equality of fairness metrics across multiple overlapping groups must be considered simultaneously. In our second contribution, we thus address the combined problem of fairness-aware online learning from imbalanced evolving streams, while considering multiple sensitive attributes. To this end, we introduce the Multi-Sensitive Queue-based Online Fair Learning (MQ-OFL) algorithm, an online fairness-aware approach, which maintains valid and fair models over evolving streams. MQ-OFL changes the training distribution in an online fashion based on both stream imbalance and discriminatory behavior of the model evaluated over the historical stream. We compare our MQ-OFL method with state-of-art studies on real-world datasets and present comparative insights on the performance. Our final contribution focuses on explainability and interpretability in fairness-aware online learning. This research is guided by the concerns raised due to the black-box nature of models, concealing internal logic from users. This lack of transparency poses practical and ethical challenges, particularly when these algorithms make decisions in finance, healthcare, and marketing domains. These systems may introduce biases and prejudices during the learning phase by utilizing complex machine learning algorithms and sensitive data. Consequently, decision models trained on such data may make unfair decisions and it is important to realize such issues before deploying the models. To address this issue, we introduce techniques for interpreting the outcomes of fairness-aware online learning. Through a case study predicting income based on features such as ethnicity, biological sex, age, and education level, we demonstrate how our fairness-aware learning process (MQ-OFL) maintains a balance between accuracy and discrimination trade-off using global and local surrogate models.

No description available.

Feature extraction is one of the major challenges in object recognition. Features that are extracted from one type of objects cannot always be used directly for a different type of objects, therefore limiting the performance of feature extraction. Having an automatic feature learning algorithm could be a big advantage for an object recognition algorithm. This research first introduces several improvements on a fully automatic feature construction method called Evolution COnstructed Feature (ECO-Feature). These improvements are developed to construct more robust features and make the training process more efficient than the original version. The main weakness of the original ECO-Feature algorithm is that it is designed only for binary classification and cannot be directly applied to multi-class cases. We also observe that the recognition performance depends heavily on the size of the feature pool from which features can be selected and the ability of selecting the best features. For these reasons, we have developed an enhanced evolutionary learning method for multi-class object classification to address these challenges. Our method is called Evolutionary Learning of Boosted Features (ECO-Boost). ECO-Boost method is an efficient evolutionary learning algorithm developed to automatically construct highly discriminative image features from the training image for multi-class image classification. This unique method constructs image features that are often overlooked by humans, and is robust to minor image distortion and geometric transformations. We evaluate this algorithm with a few visual inspection datasets including specialty crops, fruits and road surface conditions. Results from extensive experiments confirm that ECO-Boost performs closely comparable to other methods and achieves a good balance between accuracy and simplicity for real-time multi-class object classification applications. It is a hardware-friendly algorithm that can be optimized for hardware implementation in an FPGA for real-time embedded visual inspection applications.

Gaussian graphical model has been a popular tool to investigate conditional dependency between random variables by estimating sparse precision matrices. The estimated precision matrices could be mapped into networks for visualization. For related but different classes, jointly estimating networks by taking advantage of common structure across classes can help us better estimate conditional dependencies among variables. Furthermore, there may exist multilevel structure among variables; some variables are considered as higher level variables and others are nested in these higher level variables, which are called lower level variables. In this dissertation, we made several contributions to the area of joint estimation of Gaussian graphical models across heterogeneous classes: the first is to propose a joint estimation method for estimating Gaussian graphical models across unbalanced multi-classes, whereas the second considers multilevel variable information during the joint estimation procedure and simultaneously estimates higher level network and lower level network. For the first project, we consider the problem of jointly estimating Gaussian graphical models across unbalanced multi-class. Most existing methods require equal or similar sample size among classes. However, many real applications do not have similar sample sizes. Hence, in this dissertation, we propose the joint adaptive graphical lasso, a weighted L1 penalized approach, for unbalanced multi-class problems. Our joint adaptive graphical lasso approach combines information across classes so that their common characteristics can be shared during the estimation process. We also introduce regularization into the adaptive term so that the unbalancedness of data is taken into account. Simulation studies show that our approach performs better than existing methods in terms of false positive rate, accuracy, Mathews correlation coefficient, and false discovery rate. We demonstrate the advantage of our approach using liver cancer data set. For the second one, we propose a method to jointly estimate the multilevel Gaussian graphical models across multiple classes. Currently, methods are still limited to investigate a single level conditional dependency structure when there exists the multilevel structure among variables. Due to the fact that higher level variables may work together to accomplish certain tasks, simultaneously exploring conditional dependency structures among higher level variables and among lower level variables are of our main interest. Given multilevel data from heterogeneous classes, our method assures that common structures in terms of the multilevel conditional dependency are shared during the estimation procedure, yet unique structures for each class are retained as well. Our proposed approach is achieved by first introducing a higher level variable factor within a class, and then common factors across classes. The performance of our approach is evaluated on several simulated networks. We also demonstrate the advantage of our approach using breast cancer patient data. / Ph. D.

[pt] Este trabalho apresenta a proposta de um novo método para a extração de regras fuzzy de máquinas de vetor suporte (SVMs) treinadas para problemas de classificação. SVMs são sistemas de aprendizado baseados na teoria estatística do aprendizado e apresentam boa habilidade de generalização em conjuntos de dados reais. Estes sistemas obtiveram sucesso em vários tipos de problemas. Entretanto, as SVMs, da mesma forma que redes neurais (RN), geram um modelo caixa preta, isto é, um modelo que não explica o processo pelo qual sua saída é obtida. Alguns métodos propostos para reduzir ou eliminar essa limitação já foram desenvolvidos para o caso de classificação binária, embora sejam restritos à extração de regras simbólicas, isto é, contêm funções ou intervalos nos antecedentes das regras. No entanto, a interpretabilidade de regras simbólicas ainda é reduzida. Deste modo, propõe-se, neste trabalho, uma técnica para a extração de regras fuzzy de SVMs treinadas, com o objetivo de aumentar a interpretabilidade do conhecimento gerado. Além disso, o modelo proposto foi desenvolvido para classificação em múltiplas classes, o que ainda não havia sido abordado até agora. As regras fuzzy obtidas são do tipo se x1 pertence ao conjunto fuzzy C1, x2 pertence ao conjunto fuzzy C2,..., xn pertence ao conjunto fuzzy Cn, então o ponto x = (x1,...,xn) é da classe A. Para testar o modelo foram realizados estudos de caso detalhados com quatro bancos de dados: Íris, Wine, Bupa Liver Disorders e Winconsin Breast Cancer. A cobertura das regras resultantes da aplicação desse modelo nos testes realizados mostrou-se muito boa, atingindo 100% no caso da Íris. Após a geração das regras, foi feita uma avaliação das mesmas, usando dois critérios, a abrangência e a acurácia fuzzy. Além dos testes acima mencionados foi comparado o desempenho dos métodos de classificação em múltiplas classes usados no trabalho. / [en] This text proposes a new method for fuzzy rule extraction from support vector machines (SVMs) trained to solve classification problems. SVMs are learning systems based on statistical learning theory and present good ability of generalization in real data base sets. These systems have been successfully applied to a wide variety of application. However SVMs, as well as neural networks, generates a black box model, i.e., a model which does not explain the process used in order to obtain its result. Some considered methods to reduce this limitation already has been proposed for the binary classification case, although they are restricted to symbolic rules extraction, and they have, in their antecedents, functions or intervals. However, the interpretability of the symbolic generated rules is small. Hence, to increase the linguistic interpretability of the generating rules, we propose a new technique for extracting fuzzy rules of a trained SVM. Moreover, the proposed model was developed for classification in multiple classes, which was not introduced till now. Fuzzy rules obtained are presented in the format if x1 belongs to the fuzzy set C1, x2 belongs to the fuzzy set C2 , … , xn belongs to the fuzzy set Cn , then the point x=(x1, x2, …xn) belongs to class A. For testing this new model, we performed detailed researches on four data bases: Iris, Wine, Bupa Liver Disorders and Wisconsin Breast Cancer. The rules´ coverage resultant of the application of this method was quite good, reaching 100% in Iris case. After the rules generation, its evaluation was performed using two criteria: coverage and accuracy. Besides the testing above, the performance of the methods for multi-class SVM described in this work was evaluated.

Une nouvelle méthode, simple et rapide, a été développée pour isoler dans le miel différents antimicrobiens usuellement recherchés en contrôle sanitaire et appartenant à quatre classes différentes: les sulfamides, les tétracyclines, les macrolides et lincosamides associés et les aminoglycosides. Ces molécules antimicrobiennes sont analysées par chromatographie liquide couplée à la spectrométrie de masse en tandem (CL-SM/SM) après avoir été extraites de l’échantillon de miel par une méthode d’extraction unique. Afin de définir les conditions optimales de séparation et de détection, l’influence de la nature et de la concentration d’un agent d’appariement d’ions tel que le HFBA ou le PFPA, introduit dans la phase mobile, a pu être évaluée sur une colonne analytique en chromatographie de phase inversée de type C18. Plusieurs paramètres ont été pris en compte et étudiés lors de l’élaboration de la méthode d’extraction tels que la nature du solvant d’extraction, le pH, l’étape d’hydrolyse acide, l’efficacité de l’extraction par ultrasons et enfin la purification de l’extrait avant injection. La méthode développée a ensuite été validée suivant les recommandations de la Décision de la Commission Européenne (CE) No 2002/657 puis a subi une étape de validation supplémentaire en participant à une comparaison inter-laboratoire organisée sur des matériaux de miel contaminés et gérée par un organisme extérieur accrédité suivant la norme ISO17043. Par la suite, une démarche de transfert de la méthode analytique validée en CL-SM/SM a été mise en place pour son utilisation en chromatographie liquide couplée à la spectrométrie de masse à haute résolution (CL-SMHR). Une validation de cette démarche a été menée par l’application d’une étude statistique descriptive basée sur la notion de profil d’exactitude. Finalement, un programme expérimental de surveillance a été entrepris sur une série d’échantillons de divers miels collectés sur des marchés locaux pour tester la qualité des produits commercialisés au Liban. Contrôlés au regard de leur contamination en résidus d’antimicrobiens en CL-SM/SM parmi la trentaine de molécules prédéfinies dans l’étude, la positivité et/ou la non-conformité de certains échantillons ont pu être confirmées par l’utilisation de la CL-SMHR. / A new, simple and rapid method has been developed for the determination of multiclass antimicrobial residues in honey (sulfonamides, tetracyclines, macrolides, lincosamides and aminoglycosides). All the compounds were extracted from honey within single extraction method and analyzed by liquid chromatography - tandem mass spectrometry (LC-MS/MS) operating in positive electrospray ionization mode. In our study, we examined the behavior of volatile perfluorinated carboxylic acids (HFBA and PFPA) used as ion-pairing reagents for the separation of multiclass of antibiotic residues by reversed phase Zorbax SB C18 column. Furthermore, the extraction and clean-up steps were investigated and optimized by using ultrasonic-assisted extraction and dispersive solid phase extraction (d-SPE). Different parameters affecting the extraction efficiency including type of solvent, pH, breaking efficiencies of N-glycosidic linkage by hydrochloric acid, ultrasonic extraction and its duration compared to shaking technique, along with dispersive SPE clean-up were examined prior sample injection. The method was then validated according to European Commission Decision (EC) No 2002/657. Furthermore, the method was tested for its validity through participation in proficiency testing scheme organized by FAPAS for the analysis of tetracycline in honey. Afterwards, a transfer of the validated LC-MS/MS analytical method has been applied for the determination of antimicrobial residues in honey from low resolution to High Resolution Mass Spectrometry (HRMS). For that purpose, descriptive statistical approach was performed to assess the performance of the method based on simultaneous evaluation of the trueness and the intermediate precision. Finally, the method was applied for the determination of antimicrobial residues in honey collected from local markets at different regions in Lebanon. Positive samples were then analyzed by the LC-HRMS to confirm the presence of analytes detected by LC-MSMS.