O Aprendizado de Máquina trata da questão de como desenvolver programas de computador capazes de aprender um conceito ou hipótese a partir de um conjunto de exemplos ou casos observados. Baseado no conjunto de treinamento o algoritmo de aprendizado induz a classificação de uma hipótese capaz de determinar corretamente a classe de novos exemplos ainda não rotulados. Linguagens de descrição são necessárias para escrever exemplos, conhecimento do domínio bem como as hipóteses aprendidas a partir dos exemplos. Em geral, essas linguagens podem ser divididas em dois tipos: linguagem baseada em atributo-valor ou proposicional e linguagem relacional. Algoritmos de aprendizado são classificados como proposicional ou relacional dependendo da liguagem de descrição que eles utilizam. Além disso, no aprendizado simbólico o objetivo é gerar a classificação de hipóteses que possam ser facilmente interpretadas pelos humanos. Algoritmos de aprendizado proposicional utilizam a representação atributo-valor, a qual é inadequada para representar objetos estruturados e relações entre esses objetos. Por outro lado, a Programação lógica Indutiva (PLI) é realizada com o desenvolvimento de técnicas e ferramentas para o aprendizado relacional. Sistemas de PLI são capazes de aprender levando em consideração conhecimento do domínio na forma de um programa lógico e também usar a linguagem de programas lógicos para descrever o conhecimento induzido. Neste trabalho foi implementado um módulo chamado Kaeru para converter dados no formato atributo-valor para o formato relacional utilizado pelo sistema de PLI Aleph. Uma série de experimentos foram realizados com quatro conjuntos de dados naturais e um conjunto de dados real no formato atributo valor. Utilizando o módulo conversor Kaeru esses dados foram convertidos para o formato relacional utilizado pelo Aleph e hipóteses de classificação foram induzidas utilizando aprendizado proposicional bem como aprendizado relacional. É mostrado também, que o aprendizado proposicional pode ser utilizado para incrementar o conhecimento do domínio utilizado pelos sistemas de aprendizado relacional para melhorar a qualidade das hipóteses induzidas. / Machine Learning addresses the question of how to build computer programs that learn a concept or hypotheses from a set of examples, objects or cases. Descriptive languages are necessary in machine learning to describe the set of examples, domain knowledge as well as the hypothesis learned from these examples. In general, these languages can be divided into two types: languages based on attribute values, or em propositional languages, and relational languages. Learning algorithms are often classified as propositional or relational taking into consideration the descriptive language they use. Typical propositional learning algorithms employ the attribute value representation, which is inadequate for problem-domains that require reasoning about the structure of objects in the domain and relations among such objects. On the other hand, Inductive Logig Programming (ILP) is concerned with the development of techniques and tools for relational learning. ILP systems are able to take into account domain knowledge in the form of a logic program and also use the language of logic programs for describing the induced knowledge or hypothesis. In this work we propose and implement a module, named kaeru, to convert data in the attribute-value format to the relational format used by the ILP system Aleph. We describe a series of experiments performed on four natural data sets and one real data set in the attribute value format. Using the kaeru module these data sets were converted to the relational format used by Aleph and classifying hipoteses were induced using propositional as well as relational learning. We also show that propositional knowledge can be used to increment the background knowledge used by relational learners in order to improve the induded hypotheses quality.

Este trabalho investiga Aprendizado Indutivo de Máquina como função das linguagens de descrição, utilizadas para expressar instancias, conceitos e teoria do domínio. A ampliação do poder de representação do aprendizado proporcional e abordada no contexto de indução construtiva, no domínio de funções booleanas, com a proposta de uma estratégia de composição de atributos denominada root-fringe. Avaliações experimentais dessa e de outras estratégias de construção de novos atributos foram conduzidas e os resultados analisados. Dois métodos de poda, para tratamento de ruídos, em aprendizado de arvores de decisão, foram avaliados num ambiente de indução construtiva e os resultados discutidos. Devido a limitação do aprendizado proposicional, foram investigadas formas de ampliação dos limites do aprendizado, através da ampliação do poder representacional das linguagens de descrição. Foi escolhida Programação Lógica Indutiva - PLI - que e um paradigma de aprendizado indutivo que usa restrições de Lógica de Primeira Ordem como linguagens de descrição. O aprendizado em PLI só é factível quando as linguagens utilizadas estão restritas e é fortemente controlado, caso contrário, o aprendizado em PLI se torna indecidível. A pesquisa em PLI se direcionou a formas de restrição das linguagens de descrição da teoria do domínio e de hipóteses. Três algoritmos que \"traduzem\" a teoria do domínio de sua forma intencional, para extensional, são apresentados. As implementações de dois deles são discutidas. As implementações realizadas deram origem a dois ambientes experimentais de aprendizado: o ambiente proposicional experimental, do qual fazem parte o ambiente experimental construtivo, e o ambiente experimental relacional. / This work investigates Inductive Machine Learning as a function of the description languages employed to express instances, concepts and domain theory. The enlargement of the representational power of propositional learning methods is approached via constructive induction, in the domain of boolean functions, through the proposal of a bias for composing attributes, namely, the bias root-fringe. Experimental evaluation of root-fringe, as well as other biases for constructing new attributes was conducted and the results analyzed. Two pruning methods for decision trees were evaluated in an environment of constructive induction and the results discussed. Due to the limitations of propositional learning, ways of enlarging the limits of the learning process were investigated through enlarging the representational power of the description languages. It was chosen Inductive Logic Programming - ILP - that is an inductive learning paradigm that uses restrictions of First Order Logic as description languages. Learning using ILP is only feasible when the languages are restricted and are strongly controlled; otherwise, learning in ILP becomes undecidible. Research work in ILP was directed towards restricting domain theory and hypotheses description languages. Three algorithms that \"translate\" the intentional expression of a domain theory into its extensional expression are presented. The implementations of two of them are discussed. The implementations gave rise to two experimental learning environments: the propositional environment, which includes the constructive environment, and the relational environment.

Concept discovery is the problem of finding definitions of target relation in terms or other relation given as a background knowledge. Inductive Logic Programming (ILP)-based and graph-based approaches are two competitors in concept discovery problem. Although ILP-based systems have long dominated the area, graph-based systems have recently gained popularity as they overcome certain shortcomings of ILP-based systems. While having applications in numerous domains, ILP-based concept discovery systems still sustain scalability and efficiency issues. These issues generally arose due to the large search spaces such systems build. In this work we propose memoization-based and parallelization-based methods that modify the search space construction step and the evaluation step of ILP-based concept discovery systems to overcome these problem. In this work we propose three memoization-based methods, called Tabular CRIS, Tabular CRIS-wEF, and Selective Tabular CRIS. In these methods, basically, evaluation queries are stored in look-up tables for later uses. While preserving some core functions in common, each proposed method improves e_ciency and scalability of its predecessor by introducing constraints on what kind of evaluation queries to store in look-up tables and for how long. The proposed parallelization method, called pCRIS, parallelizes the search space construction and evaluation steps of ILP-based concept discovery systems in a data-parallel manner. The proposed method introduces policies to minimize the redundant work and waiting time among the workers at synchronization points. Graph-based approaches were first introduced to the concept discovery domain to handle the so called local plateau problem. Graph-based approaches have recently gained more popularity in concept discovery system as they provide convenient environment to represent relational data and are able to overcome certain shortcomings of ILP-based concept discovery systems. Graph-based approaches can be classified as structure-based approaches and path-finding approaches. The first class of approaches need to employ expensive algorithms such as graph isomorphism to find frequently appearing substructures. The methods that fall into the second class need to employ sophisticated indexing mechanisms to find out the frequently appearing paths that connect some nodes in interest. In this work, we also propose a hybrid method for graph-based concept discovery which does not require costly substructure matching algorithms and path indexing mechanism. The proposed method builds the graph in such a way that similar facts are grouped together and paths that eventually turn to be concept descriptors are build while the graph is constructed.

Real-world learning tasks present a range of issues for learning systems. Learning tasks can be complex and the training data noisy. When operating as part of a larger system, there may be limitations on available memory and computational resources. Learners may also be required to provide results from a stream. This thesis investigates the problem of incremental, relational learning from imperfect data with constrained time and memory resources. The learning process involves incremental update of a theory when an example is presented that contradicts the theory. Contradictions occur if there is an incorrect theory or noisy data. The learner cannot discriminate between the two possibilities, so both are considered and the better possibility used. Additionally, all changes to the theory must have support from multiple examples. These two principles allow learning from imperfect data. The Minimum Description Length principle is used for selection between possible worlds and determining appropriate levels of additional justification. A new encoding scheme allows the use of MDL within the framework of Inductive Logic Programming. Examples must be stored to provide additional justification for revisions without violating resource requirements. A new algorithm determines when to discard examples, minimising total usage while ensuring sufficient storage for justifications. Searching for revisions is the most computationally expensive part of the process, yet not all searches are successful. Another new algorithm uses a notion of theory stability as a guide to occasionally disallow entire searches to reduce overall time. The approach has been implemented as a learner called NILE. Empirical tests include two challenging domains where this type of learner acts as one component of a larger task. The first of these involves recognition of behavior activation conditions in another agent as part of an opponent modeling task. The second, more challenging task is learning to identify objects in visual images by recognising relationships between image features. These experiments highlight NILE'S strengths and limitations as well as providing new n domains for future work in ILP.

Este trabalho investiga Aprendizado Indutivo de Máquina como função das linguagens de descrição, utilizadas para expressar instancias, conceitos e teoria do domínio. A ampliação do poder de representação do aprendizado proporcional e abordada no contexto de indução construtiva, no domínio de funções booleanas, com a proposta de uma estratégia de composição de atributos denominada root-fringe. Avaliações experimentais dessa e de outras estratégias de construção de novos atributos foram conduzidas e os resultados analisados. Dois métodos de poda, para tratamento de ruídos, em aprendizado de arvores de decisão, foram avaliados num ambiente de indução construtiva e os resultados discutidos. Devido a limitação do aprendizado proposicional, foram investigadas formas de ampliação dos limites do aprendizado, através da ampliação do poder representacional das linguagens de descrição. Foi escolhida Programação Lógica Indutiva - PLI - que e um paradigma de aprendizado indutivo que usa restrições de Lógica de Primeira Ordem como linguagens de descrição. O aprendizado em PLI só é factível quando as linguagens utilizadas estão restritas e é fortemente controlado, caso contrário, o aprendizado em PLI se torna indecidível. A pesquisa em PLI se direcionou a formas de restrição das linguagens de descrição da teoria do domínio e de hipóteses. Três algoritmos que \"traduzem\" a teoria do domínio de sua forma intencional, para extensional, são apresentados. As implementações de dois deles são discutidas. As implementações realizadas deram origem a dois ambientes experimentais de aprendizado: o ambiente proposicional experimental, do qual fazem parte o ambiente experimental construtivo, e o ambiente experimental relacional. / This work investigates Inductive Machine Learning as a function of the description languages employed to express instances, concepts and domain theory. The enlargement of the representational power of propositional learning methods is approached via constructive induction, in the domain of boolean functions, through the proposal of a bias for composing attributes, namely, the bias root-fringe. Experimental evaluation of root-fringe, as well as other biases for constructing new attributes was conducted and the results analyzed. Two pruning methods for decision trees were evaluated in an environment of constructive induction and the results discussed. Due to the limitations of propositional learning, ways of enlarging the limits of the learning process were investigated through enlarging the representational power of the description languages. It was chosen Inductive Logic Programming - ILP - that is an inductive learning paradigm that uses restrictions of First Order Logic as description languages. Learning using ILP is only feasible when the languages are restricted and are strongly controlled; otherwise, learning in ILP becomes undecidible. Research work in ILP was directed towards restricting domain theory and hypotheses description languages. Three algorithms that \"translate\" the intentional expression of a domain theory into its extensional expression are presented. The implementations of two of them are discussed. The implementations gave rise to two experimental learning environments: the propositional environment, which includes the constructive environment, and the relational environment.

O Aprendizado de Máquina trata da questão de como desenvolver programas de computador capazes de aprender um conceito ou hipótese a partir de um conjunto de exemplos ou casos observados. Baseado no conjunto de treinamento o algoritmo de aprendizado induz a classificação de uma hipótese capaz de determinar corretamente a classe de novos exemplos ainda não rotulados. Linguagens de descrição são necessárias para escrever exemplos, conhecimento do domínio bem como as hipóteses aprendidas a partir dos exemplos. Em geral, essas linguagens podem ser divididas em dois tipos: linguagem baseada em atributo-valor ou proposicional e linguagem relacional. Algoritmos de aprendizado são classificados como proposicional ou relacional dependendo da liguagem de descrição que eles utilizam. Além disso, no aprendizado simbólico o objetivo é gerar a classificação de hipóteses que possam ser facilmente interpretadas pelos humanos. Algoritmos de aprendizado proposicional utilizam a representação atributo-valor, a qual é inadequada para representar objetos estruturados e relações entre esses objetos. Por outro lado, a Programação lógica Indutiva (PLI) é realizada com o desenvolvimento de técnicas e ferramentas para o aprendizado relacional. Sistemas de PLI são capazes de aprender levando em consideração conhecimento do domínio na forma de um programa lógico e também usar a linguagem de programas lógicos para descrever o conhecimento induzido. Neste trabalho foi implementado um módulo chamado Kaeru para converter dados no formato atributo-valor para o formato relacional utilizado pelo sistema de PLI Aleph. Uma série de experimentos foram realizados com quatro conjuntos de dados naturais e um conjunto de dados real no formato atributo valor. Utilizando o módulo conversor Kaeru esses dados foram convertidos para o formato relacional utilizado pelo Aleph e hipóteses de classificação foram induzidas utilizando aprendizado proposicional bem como aprendizado relacional. É mostrado também, que o aprendizado proposicional pode ser utilizado para incrementar o conhecimento do domínio utilizado pelos sistemas de aprendizado relacional para melhorar a qualidade das hipóteses induzidas. / Machine Learning addresses the question of how to build computer programs that learn a concept or hypotheses from a set of examples, objects or cases. Descriptive languages are necessary in machine learning to describe the set of examples, domain knowledge as well as the hypothesis learned from these examples. In general, these languages can be divided into two types: languages based on attribute values, or em propositional languages, and relational languages. Learning algorithms are often classified as propositional or relational taking into consideration the descriptive language they use. Typical propositional learning algorithms employ the attribute value representation, which is inadequate for problem-domains that require reasoning about the structure of objects in the domain and relations among such objects. On the other hand, Inductive Logig Programming (ILP) is concerned with the development of techniques and tools for relational learning. ILP systems are able to take into account domain knowledge in the form of a logic program and also use the language of logic programs for describing the induced knowledge or hypothesis. In this work we propose and implement a module, named kaeru, to convert data in the attribute-value format to the relational format used by the ILP system Aleph. We describe a series of experiments performed on four natural data sets and one real data set in the attribute value format. Using the kaeru module these data sets were converted to the relational format used by Aleph and classifying hipoteses were induced using propositional as well as relational learning. We also show that propositional knowledge can be used to increment the background knowledge used by relational learners in order to improve the induded hypotheses quality.

abstract: While in recent years deep learning (DL) based approaches have been the popular approach in developing end-to-end question answering (QA) systems, such systems lack several desired properties, such as the ability to do sophisticated reasoning with knowledge, the ability to learn using less resources and interpretability. In this thesis, I explore solutions that aim to address these drawbacks. Towards this goal, I work with a specific family of reading comprehension tasks, normally referred to as the Non-Extractive Reading Comprehension (NRC), where the given passage does not contain enough information and to correctly answer sophisticated reasoning and ``additional knowledge" is required. I have organized the NRC tasks into three categories. Here I present my solutions to the first two categories and some preliminary results on the third category. Category 1 NRC tasks refer to the scenarios where the required ``additional knowledge" is missing but there exists a decent natural language parser. For these tasks, I learn the missing ``additional knowledge" with the help of the parser and a novel inductive logic programming. The learned knowledge is then used to answer new questions. Experiments on three NRC tasks show that this approach along with providing an interpretable solution achieves better or comparable accuracy to that of the state-of-the-art DL based approaches. The category 2 NRC tasks refer to the alternate scenario where the ``additional knowledge" is available but no natural language parser works well for the sentences of the target domain. To deal with these tasks, I present a novel hybrid reasoning approach which combines symbolic and natural language inference (neural reasoning) and ultimately allows symbolic modules to reason over raw text without requiring any translation. Experiments on two NRC tasks shows its effectiveness. The category 3 neither provide the ``missing knowledge" and nor a good parser. This thesis does not provide an interpretable solution for this category but some preliminary results and analysis of a pure DL based approach. Nonetheless, the thesis shows beyond the world of pure DL based approaches, there are tools that can offer interpretable solutions for challenging tasks without using much resource and possibly with better accuracy. / Dissertation/Thesis / Doctoral Dissertation Computer Science 2019

Traditional Inductive Logic Programming (ILP) focuses on the setting where the target theory is a generalisation of the observations. This is known as Observational Predicate Learning (OPL). In the Theory Completion setting the target theory is not in the same predicate as the observations (non-OPL). This thesis investigates two alternative simple extensions to traditional ILP to perform non-OPL or Theory Completion. Both techniques perform extraction-case abduction from an existing background theory and one seed observation. The first technique -- Logical Back-propagation -- modifies the existing background theory so that abductions can be achieved by a form of constructive negation using a standard SLD-resolution theorem prover. The second technique -- SOLD-resolution -- modifies the theorem prover, and leaves the existing background theory unchanged. It is shown that all abductions produced by Logical Back-propagation can also be generated by SOLD-resolution; but the reverse does not hold. The implementation using the SOLD-resolution technique -- the ALECTO system -- was applied to the problems of completing context free and context dependant grammars; and learning Event Calculus programs. It was successfully able to learn an Event Calculus program to control the navigation of a real-life robot. The Event Calculus is a formalism to represent common-sense knowledge. It follows that the discovery of some common-sense knowledge was produced with the assistance of a machine.

While most statistical learning methods are designed to work with data stored in a single table, many large datasets are stored in relational database systems. Probabilistic Relational Models (PRM) extend Bayesian networks by introducing relations and individuals, thus making it possible to represent information in a relational database. However, learning a PRM from relational data is a more complex task than learning a Bayesian Network from \"flat\" data. The main difficulties that arise while learning a PRM are establishing what are the legal dependency structures, searching for possible structures, and scoring them. This thesis focuses on the development of a novel approach to learn the structure of a PRM, describes a package in the R language to support the learning framework, and applies it to a real, large scale scenario of a city named Atibaia, in the state of São Paulo, Brazil. The research is based on a database combining three different tables, each representing one class in the domain of study. The first table contains 27 attributes from 110,816 citizens of Atibaia. The second table contains 9 attributes from 20,162 companies located in the city. And finally, the third table has 8 attributes from 327 census sectors (small territorial units that comprise the city of Atibaia). The proposed framework is applied to learn a PRM structure and parameters from the database. The model is used to verify if the Social Class of a person can be explained by the location where they live, their neighbors, and the companies nearby. Preliminary experiments have been conducted and a paper published in the 2017 Symposium on Knowledge Discovery, Mining and Learning (KDMiLe). The algorithm performance was further evaluated by extensive experimentation, and a broader study using Serasa Experian data was conducted. Finally, the package in the R language that supports our method was refined along with proper documentation and a tutorial. / Embora a maioria dos métodos de aprendizado estatístico tenha sido desenvolvida para se trabalhar com dados armazenados em uma única tabela, muitas bases de dados estão armazenadas em bancos de dados relacionais. Modelos Probabilísticos Relacionai (PRM) estendem Redes Bayesianas introduzindo relações e indivíduos, tornando possível a representação de informação em uma base de dados relacional. Entretanto, aprender um PRM através de dados relacionais é uma tarefa mais complexa que aprender uma Rede Bayesiana de uma única tabela. As maiores dificuldades que se impõe enquanto se aprende um PRM são estabelecer quais são as estruturas de dependência legais, procurar por possíveis estruturas, e avalia-las. Esta tese foca em desenvolver um novo método de aprendizado de estruturas de PRM, descrever um pacote na linguagem R que suporte este método e aplica-lo a um cenário real e de grande escala, a cidade de Atibaia, no estado de São Paulo, Brasil. Esta pesquisa está baseada em uma base de dados combinando três tabelas distintas, cada uma representando uma classe no domínio de estudo. A primeira tabela contém 27 atributos de 110.816 habitantes de Atibaia, e a segunda tabela contém 9 atributos de 20.162 empresas da cidade. Por fim, a terceira tabela possui 8 atributos para 327 setores censitários (pequenas unidades territoriais que formam a cidade de Atibaia). A proposta é aplicada para aprender-se a estrutura de um PRM e seus parâmetros através desta base de dados. O modelo foi utilizado para verificar se a classe social de uma pessoa pode ser explicada pelo local onde ela vive, seus vizinhos e as companhias próximas. Experimentos preliminares foram conduzidos e um artigo foi publicado no Symposium on Knowledge Discovery, Mining and Learning (KDMiLe). O desempenho do algoritmo foi reavaliada através de extensiva experimentação, e um estudo mais amplo foi conduzido com os dados da Serasa Experian. Por fim, o pacote em R que suporta o método proposto foi refinado, e documentação e tutorial apropriado foram descritos.

Harmony is the aspect of music concerned with the structure, progression, and relation of chords. In Western tonal music each period had different rules and practices of harmony. Similarly some composers and musicians are recognised for their characteristic harmonic patterns which differ from the chord sequences used by other musicians of the same period or genre. This thesis is concerned with the automatic induction of the harmony rules and patterns underlying a genre, a composer, or more generally a 'style'. Many of the existing approaches for music classification or pattern extraction make use of statistical methods which present several limitations. Typically they are black boxes, can not be fed with background knowledge, do not take into account the intricate temporal dimension of the musical data, and ignore rare but informative events. To overcome these limitations we adopt first-order logic representations of chord sequences and Inductive Logic Programming techniques to infer models of style. We introduce a fixed length representation of chord sequences similar to n-grams but based on first-order logic, and use it to characterise symbolic corpora of pop and jazz music. We extend our knowledge representation scheme using context-free definite-clause grammars, which support chord sequences of any length and allow to skip ornamental chords, and test it on genre classification problems, on both symbolic and audio data. Through these experiments we also compare various chord and harmony characteristics such as degree, root note, intervals between root notes, chord labels and assess their characterisation and classification accuracy, expressiveness, and computational cost. Moreover we extend a state- of-the-art genre classifier based on low-level audio features with such harmony-based models and prove that it can lead to statistically significant classification improvements. We show our logic-based modelling approach can not only compete with and improve on statistical approaches but also provides expressive, transparent and musicologically meaningful models of harmony which makes it suitable for knowledge discovery purposes.