Multiple instance (MI) learning is a relatively new topic in machine learning. It is concerned with supervised learning but differs from normal supervised learning in two points: (1) it has multiple instances in an example (and there is only one instance in an example in standard supervised learning), and (2) only one class label is observable for all the instances in an example (whereas each instance has its own class label in normal supervised learning). In MI learning there is a common assumption regarding the relationship between the class label of an example and the ``unobservable'' class labels of the instances inside it. This assumption, which is called the ``MI assumption'' in this thesis, states that ``An example is positive if at least one of its instances is positive and negative otherwise''. In this thesis, we first categorize current MI methods into a new framework. According to our analysis, there are two main categories of MI methods, instance-based and metadata-based approaches. Then we propose a new assumption for MI learning, called the ``collective assumption''. Although this assumption has been used in some previous MI methods, it has never been explicitly stated,\footnote{As a matter of fact, for some of these methods, it is actually claimed that they use the standard MI assumption stated above.} and this is the first time that it is formally specified. Using this new assumption we develop new algorithms --- more specifically two instance-based and one metadata-based methods. All of these methods build probabilistic models and thus implement statistical learning algorithms. The exact generative models underlying these methods are explicitly stated and illustrated so that one may clearly understand the situations to which they can best be applied. The empirical results presented in this thesis show that they are competitive on standard benchmark datasets. Finally, we explore some practical applications of MI learning, both existing and new ones. This thesis makes three contributions: a new framework for MI learning, new MI methods based on this framework and experimental results for new applications of MI learning.

Motivated by various challenging real-world applications, such as drug activity prediction and image retrieval, multi-instance (MI) learning has attracted considerable interest in recent years. Compared with standard supervised learning, the MI learning task is more difficult as the label information of each training example is incomplete. Many MI algorithms have been proposed. Some of them are specifically designed for MI problems whereas others have been upgraded or adapted from standard single-instance learning algorithms. Most algorithms have been evaluated on only one or two benchmark datasets, and there is a lack of systematic comparisons of MI learning algorithms. This thesis presents a comprehensive study of MI learning algorithms that aims to compare their performance and find a suitable way to properly address different MI problems. First, it briefly reviews the history of research on MI learning. Then it discusses five general classes of MI approaches that cover a total of 16 MI algorithms. After that, it presents empirical results for these algorithms that were obtained from 15 datasets which involve five different real-world application domains. Finally, some conclusions are drawn from these results: (1) applying suitable standard single-instance learners to MI problems can often generate the best result on the datasets that were tested, (2) algorithms exploiting the standard asymmetric MI assumption do not show significant advantages over approaches using the so-called collective assumption, and (3) different MI approaches are suitable for different application domains, and no MI algorithm works best on all MI problems.

No description available.

Multi-instance (MI) learning is a variant of supervised machine learning, where each learning example contains a bag of instances instead of just a single feature vector. MI learning has applications in areas such as drug activity prediction, fruit disease management and image classification. This thesis investigates the case where each instance has a weight value determining the level of influence that it has on its bag's class label. This is a more general assumption than most existing approaches use, and thus is more widely applicable. The challenge is to accurately estimate these weights in order to make predictions at the bag level. An existing approach known as MILES is retroactively identified as an algorithm that uses instance weights for MI learning, and is evaluated using a variety of base learners on benchmark problems. New algorithms for learning instance weights for MI learning are also proposed and rigorously evaluated on both artificial and real-world datasets. The new algorithms are shown to achieve better root mean squared error rates than existing approaches on artificial data generated according to the algorithms' underlying assumptions. Experimental results also demonstrate that the new algorithms are competitive with existing approaches on real-world problems.

The importance of attribute vector ambiguity has been largely overlooked by the machine learning community. A pattern recognition problem can be solved in many ways within the scope of machine learning. Neural Networks, Decision Tree Algorithms such as C4.5, Bayesian Classifiers, and Instance Based Learning are the main algorithms. All listed solutions fail to address ambiguity in the attribute vector. The research reported shows, ignoring this ambiguity leads to problems of classifier scalability and issues with instance collection and aggregation. The Algorithm presented accounts for both ambiguity of the attribute vector and class label thus solving both issues of scalability and instance collection. The research also shows that when applied to sanitized data sets, suitable for traditional instance based learning, the presented algorithm performs equally as well.

Seres humanos são capazes de desenvolver complexas estruturas de conhecimento que podem ser utilizadas de modo flexível para lidar com o ambiente de maneira apropriada. Estas estruturas de conhecimento constituem um núcleo que suporta processos cognitivos, tais como a percepção, a categorização, o planejamento, etc. A Inteligência Artificial, enquanto área de investigação, ocupa-se de desenvolver meios que viabilizem a reprodução destas capacidades cognitivas em agentes artificiais. Por este motivo, a investigação de abordagens que permitam a representação de conhecimento de um modo flexível se revela altamente relevante. Com o objetivo de superar algumas das limitações típicas da teoria clássica, que é adotada por várias abordagens propostas na Inteligência Artificial, este trabalho propõe um arcabouço cognitivamente inspirado para representação de conhecimento e raciocínio que integra aspectos de três diferentes teorias cognitivas a respeito de como conceitos são representados na cognição humana: teoria clássica, teoria do protótipo e teoria do exemplar. O arcabouço resultante é capaz de suportar a composicionalidade, a tipicalidade, a representação de instâncias atípicas dos conceitos, e a representação da variabilidade de indivíduos classificados por cada conceito. Consequentemente, o arcabouço proposto também suporta raciocínio lógico e baseado em similaridade. As principais contribuições deste trabalho são a concepção teórica e a formalização de um arcabouço cognitivamente inspirado para representação de conhecimento e raciocínio. Uma outra contribuição deste trabalho é uma abordagem de raciocínio para classificação que utiliza a abordagem de representação de conhecimento proposta. Além disso, este trabalho também apresenta duas abordagens para seleção de exemplares representativos de cada conceito e uma abordagem para extração de protótipos de conceitos. Nesta tese também é apresentado um sistema para interpretação automática de processos deposicionais que adota o arcabouço proposto. Experimentos realizados em uma tarefa de classificação sugerem que o arcabouço proposto é capaz de oferecer classificações mais informativas que as oferecidas por uma abordagem puramente clássica. / Human beings can develop complex knowledge structures that can be used for dealing with the environment in suitable ways. These knowledge structures constitute a core that supports several cognitive processes, such as perception, categorization, planning, etc. The Artificial Intelligence, as a research field, aims at developing approaches for mimicking these cognitive capabilities in machines. Due to this, it is important to investigate approaches that allow representing the knowledge in flexible ways. In order to overcome some limitations of the classical theory of knowledge representation, which is adopted by several approaches proposed in the Artificial Intelligence field, this work proposes a cognitively-inspired framework for knowledge representation and reasoning which integrates aspects from three different cognitive theories about concept representation in the human cognition: classical theory, prototype theory and exemplar theory. The resulting framework can support compositionality, typicality, representation of atypical instances of concepts, and representation of the variability of the individuals classified by each concept. Consequently, the proposed framework also supports logical reasoning and similarity-based reasoning. The main contributions of this work are the formalization of a cognitively-inspired framework for knowledge representation and reasoning, two approaches for selecting representative exemplars of each concept and an approach of reasoning for classification that integrates logical reasoning and similarity-based reasoning and that is supported by definitions, prototypes and exemplars of concepts. This thesis also presents a system for automatic interpretation of depositional processes application that adopts the proposed framework. The experiments, which were performed on a classification task, suggest that the proposed framework provides classifications that are more informative than the ones provided by a classical approach.

Finding an efficient data reduction method for large-scale problems is an imperative task. In this paper, we propose a similarity-based self-constructing fuzzy clustering algorithm to do the sampling of instances for the classification task. Instances that are similar to each other are grouped into the same cluster. When all the instances have been fed in, a number of clusters are formed automatically. Then the statistical mean for each cluster will be regarded as representing all the instances covered in the cluster. This approach has two advantages. One is that it can be faster and uses less storage memory. The other is that the number of new representative instances need not be specified in advance by the user. Experiments on real-world datasets show that our method can run faster and obtain better reduction rate than other methods.

Seres humanos são capazes de desenvolver complexas estruturas de conhecimento que podem ser utilizadas de modo flexível para lidar com o ambiente de maneira apropriada. Estas estruturas de conhecimento constituem um núcleo que suporta processos cognitivos, tais como a percepção, a categorização, o planejamento, etc. A Inteligência Artificial, enquanto área de investigação, ocupa-se de desenvolver meios que viabilizem a reprodução destas capacidades cognitivas em agentes artificiais. Por este motivo, a investigação de abordagens que permitam a representação de conhecimento de um modo flexível se revela altamente relevante. Com o objetivo de superar algumas das limitações típicas da teoria clássica, que é adotada por várias abordagens propostas na Inteligência Artificial, este trabalho propõe um arcabouço cognitivamente inspirado para representação de conhecimento e raciocínio que integra aspectos de três diferentes teorias cognitivas a respeito de como conceitos são representados na cognição humana: teoria clássica, teoria do protótipo e teoria do exemplar. O arcabouço resultante é capaz de suportar a composicionalidade, a tipicalidade, a representação de instâncias atípicas dos conceitos, e a representação da variabilidade de indivíduos classificados por cada conceito. Consequentemente, o arcabouço proposto também suporta raciocínio lógico e baseado em similaridade. As principais contribuições deste trabalho são a concepção teórica e a formalização de um arcabouço cognitivamente inspirado para representação de conhecimento e raciocínio. Uma outra contribuição deste trabalho é uma abordagem de raciocínio para classificação que utiliza a abordagem de representação de conhecimento proposta. Além disso, este trabalho também apresenta duas abordagens para seleção de exemplares representativos de cada conceito e uma abordagem para extração de protótipos de conceitos. Nesta tese também é apresentado um sistema para interpretação automática de processos deposicionais que adota o arcabouço proposto. Experimentos realizados em uma tarefa de classificação sugerem que o arcabouço proposto é capaz de oferecer classificações mais informativas que as oferecidas por uma abordagem puramente clássica. / Human beings can develop complex knowledge structures that can be used for dealing with the environment in suitable ways. These knowledge structures constitute a core that supports several cognitive processes, such as perception, categorization, planning, etc. The Artificial Intelligence, as a research field, aims at developing approaches for mimicking these cognitive capabilities in machines. Due to this, it is important to investigate approaches that allow representing the knowledge in flexible ways. In order to overcome some limitations of the classical theory of knowledge representation, which is adopted by several approaches proposed in the Artificial Intelligence field, this work proposes a cognitively-inspired framework for knowledge representation and reasoning which integrates aspects from three different cognitive theories about concept representation in the human cognition: classical theory, prototype theory and exemplar theory. The resulting framework can support compositionality, typicality, representation of atypical instances of concepts, and representation of the variability of the individuals classified by each concept. Consequently, the proposed framework also supports logical reasoning and similarity-based reasoning. The main contributions of this work are the formalization of a cognitively-inspired framework for knowledge representation and reasoning, two approaches for selecting representative exemplars of each concept and an approach of reasoning for classification that integrates logical reasoning and similarity-based reasoning and that is supported by definitions, prototypes and exemplars of concepts. This thesis also presents a system for automatic interpretation of depositional processes application that adopts the proposed framework. The experiments, which were performed on a classification task, suggest that the proposed framework provides classifications that are more informative than the ones provided by a classical approach.

Seres humanos são capazes de desenvolver complexas estruturas de conhecimento que podem ser utilizadas de modo flexível para lidar com o ambiente de maneira apropriada. Estas estruturas de conhecimento constituem um núcleo que suporta processos cognitivos, tais como a percepção, a categorização, o planejamento, etc. A Inteligência Artificial, enquanto área de investigação, ocupa-se de desenvolver meios que viabilizem a reprodução destas capacidades cognitivas em agentes artificiais. Por este motivo, a investigação de abordagens que permitam a representação de conhecimento de um modo flexível se revela altamente relevante. Com o objetivo de superar algumas das limitações típicas da teoria clássica, que é adotada por várias abordagens propostas na Inteligência Artificial, este trabalho propõe um arcabouço cognitivamente inspirado para representação de conhecimento e raciocínio que integra aspectos de três diferentes teorias cognitivas a respeito de como conceitos são representados na cognição humana: teoria clássica, teoria do protótipo e teoria do exemplar. O arcabouço resultante é capaz de suportar a composicionalidade, a tipicalidade, a representação de instâncias atípicas dos conceitos, e a representação da variabilidade de indivíduos classificados por cada conceito. Consequentemente, o arcabouço proposto também suporta raciocínio lógico e baseado em similaridade. As principais contribuições deste trabalho são a concepção teórica e a formalização de um arcabouço cognitivamente inspirado para representação de conhecimento e raciocínio. Uma outra contribuição deste trabalho é uma abordagem de raciocínio para classificação que utiliza a abordagem de representação de conhecimento proposta. Além disso, este trabalho também apresenta duas abordagens para seleção de exemplares representativos de cada conceito e uma abordagem para extração de protótipos de conceitos. Nesta tese também é apresentado um sistema para interpretação automática de processos deposicionais que adota o arcabouço proposto. Experimentos realizados em uma tarefa de classificação sugerem que o arcabouço proposto é capaz de oferecer classificações mais informativas que as oferecidas por uma abordagem puramente clássica. / Human beings can develop complex knowledge structures that can be used for dealing with the environment in suitable ways. These knowledge structures constitute a core that supports several cognitive processes, such as perception, categorization, planning, etc. The Artificial Intelligence, as a research field, aims at developing approaches for mimicking these cognitive capabilities in machines. Due to this, it is important to investigate approaches that allow representing the knowledge in flexible ways. In order to overcome some limitations of the classical theory of knowledge representation, which is adopted by several approaches proposed in the Artificial Intelligence field, this work proposes a cognitively-inspired framework for knowledge representation and reasoning which integrates aspects from three different cognitive theories about concept representation in the human cognition: classical theory, prototype theory and exemplar theory. The resulting framework can support compositionality, typicality, representation of atypical instances of concepts, and representation of the variability of the individuals classified by each concept. Consequently, the proposed framework also supports logical reasoning and similarity-based reasoning. The main contributions of this work are the formalization of a cognitively-inspired framework for knowledge representation and reasoning, two approaches for selecting representative exemplars of each concept and an approach of reasoning for classification that integrates logical reasoning and similarity-based reasoning and that is supported by definitions, prototypes and exemplars of concepts. This thesis also presents a system for automatic interpretation of depositional processes application that adopts the proposed framework. The experiments, which were performed on a classification task, suggest that the proposed framework provides classifications that are more informative than the ones provided by a classical approach.

We define instance compressibility ([13, 17]) for parametric problems in the classes PH and PSPACE.We observe that the problem ƩiCIRCUITSAT of deciding satisfiability of a quantified Boolean circuit with i-1 alternations of quantifiers starting with an existential quantifier is complete for parametric problems in the class Ʃp/i with respect to w-reductions, and that analogously the problem QBCSAT (Quantified Boolean Circuit Satisfiability) is complete for parametric problems in PSPACE with respect to w-reductions. We show the following results about these problems: 1. If CIRCUITSAT is non-uniformly compressible within NP, then ƩiCIRCUITSAT is non-uniformly compressible within NP, for any i≥1. 2. If QBCSAT is non-uniformly compressible (or even if satisfiability of quantified Boolean CNF formulae is non-uniformly compressible), then PSPACE ⊆ NP/poly and PH collapses to the third level. Next, we define Succinct Interactive Proof (Succinct IP) and by adapting the proof of IP = PSPACE ([11, 6]) , we show that QBCNFSAT (Quantified Boolean Formula (in CNF) Satisfiability) is in Succinct IP. On the contrary if QBCNFSAT has Succinct PCPs ([32]) , Polynomial Hierarchy (PH) collapses. After extending the notion of instance compression to higher classes, we study the hierarchical structure of the parametric problems with respect to compressibility. For that purpose, we extend the existing definition of VC-hierarchy ([13]) to parametric problems. After that, we have considered a long list of natural NP problems and tried to classify them into some level of VC-hierarchy. We have shown some of the new w-reductions in this context and pointed out a few interesting results including the ones as follows. 1. CLIQUE is VC1-complete (using the results in [14]). 2. SET SPLITTING and NAE-SAT are VC2-complete. We have also introduced a new complexity class VCE in this context and showed some hardness and completeness results for this class. We have done a comparison of VC-hierarchy with other related hierarchies in parameterized complexity domain as well. Next, we define the compression of counting problems and the analogous classification of them with respect to the notion of instance compression. We define #VC-hierarchy for this purpose and similarly classify a large number of natural counting problems with respect to this hierarchy, by showing some interesting hardness and completeness results. We have considered some of the interesting practical problems as well other than popular NP problems (e.g., #MULTICOLOURED CLIQUE, #SELECTED DOMINATING SET etc.) and studied their complexity for both decision and counting version. We have also considered a large variety of circuit satisfiability problems (e.g., #MONOTONE WEIGHTED-CNFSAT, #EXACT DNF-SAT etc.) and proved some interesting results about them with respect to the theory of instance compressibility.