O aprendizado por reforço é uma técnica bem sucedida, porém lenta, para treinar agentes autônomos. Algumas soluções baseadas em políticas parciais podem ser usadas para acelerar o aprendizado e para transferir comportamentos aprendidos entre tarefas encapsulando uma política parcial. No entanto, geralmente essas políticas parciais são específicas para uma única tarefa, não levam em consideração recursos semelhantes entre tarefas e podem não corresponder exatamente a um comportamento ideal quando transferidas para outra tarefa diferente. A transferência descuidada pode fornecer más soluções para o agente, dificultando o processo de aprendizagem. Sendo assim, este trabalho propõe uma maneira de descobrir e reutilizar de modo probabilístico políticas parciais orientadas a objetos aprendidas, a fim de permitir melhores escolhas de atuação para o agente em múltiplas tarefas diferentes. A avaliação experimental mostra que a proposta é capaz de aprender e reutilizar com sucesso políticas parciais em diferentes tarefas. / Reinforcement Learning is a successful yet slow technique to train autonomous agents. Option-based solutions can be used to accelerate learning and to transfer learned behaviors across tasks by encapsulating a partial policy. However, commonly these options are specific for a single task, do not take in account similar features between tasks and may not correspond exactly to an optimal behavior when transferred to another task. Therefore, careless transfer might provide bad options to the agent, hampering the learning process. This work proposes a way to discover and reuse learned objectoriented options in a probabilistic way in order to enable better actuation choices to the agent in multiple different tasks. The experimental evaluation show that the proposal is able to learn and successfully reuse options across different tasks.

A Mineração de Opiniões é um processo que tem por objetivo extrair as opiniões e suas polaridades de sentimentos expressas em textos em língua natural. Essa área de pesquisa tem ganhado destaque devido ao volume de opiniões que os usuários compartilham na Internet, como revisões em sites de e-commerce, rede sociais e tweets. A Mineração de Opiniões baseada em Aspectos é uma alternativa promissora para analisar a polaridade do sentimento em um maior nível de detalhes. Os métodos tradicionais para extração de aspectos e classificação de sentimentos exigem a participação de especialistas de domínio para criar léxicos ou definir regras de extração para diferentes idiomas e domínios. Além disso, tais métodos usualmente exploram algoritmos de aprendizado supervisionado, porém exigem um grande conjunto de dados rotulados para induzir um modelo de classificação. Os desafios desta tese de doutorado estão relacionados a como diminuir a necessidade de grande esforço humano tanto para rotular dados, quanto para tratar a dependência de domínio para as tarefas de extração de aspectos e classificação de sentimentos dos aspectos para Mineração de Opiniões. Para reduzir a necessidade de grande quantidade de exemplos rotulados foi proposta uma abordagem semissupervisionada, denominada por Aspect-based Sentiment Propagation on Heterogeneous Networks (ASPHN) em que são propostas representações de textos nas quais os atributos linguísticos, os aspectos candidatos e os rótulos de sentimentos são modelados por meio de redes heterogêneas. Para redução dos esforços para construir recursos específicos de domínio foi proposta uma abordagem baseada em aprendizado por transferência entre domínios denominada Cross-Domain Aspect Label Propagation through Heterogeneous Networks (CD-ALPHN) que utiliza dados rotulados de outros domínios para suportar tarefas de aprendizado em domínios sem dados rotulados. Nessa abordagem são propostos uma representação em uma rede heterogênea e um método de propagação de rótulos. Os vértices da rede são os aspectos rotulados do domínio de origem, os atributos linguísticos e os candidatos a aspectos do domínio alvo. Além disso, foram analisados métodos de extração de aspectos e propostas algumas variações para considerar cenários nãosupervisionados e independentes de domínio. As soluções propostas nesta tese de doutorado foram avaliadas e comparadas as do estado-da-arte utilizando coleções de revisões de diferentes produtos e serviços. Os resultados obtidos nas avaliações experimentais são competitivos e demonstram que as soluções propostas são promissoras. / Opinion Mining is a process that aims to extract opinions and their sentiment polarities expressed in natural language texts. This area of research has been in the highlight because of the volume of opinions that users share on the available visualization means on the Internet (reviews on e-commerce sites, social networks, tweets, others). Aspect-based Opinion Mining is a promising alternative for analyzing the sentiment polarity on a high level of detail. The traditional methods for aspect extraction and sentiment classification require the participation of domain experts to create lexicons or define extraction rules for different languages and domains. In addition, such methods usually exploit supervised machine learning algorithms, but require a large set of labeled data to induce a classification model. The challenges of this doctoral thesis are related on to how to reduce the need for great human effort both: (i) to label data; and (ii) to treat domain dependency for the tasks of aspect extraction and aspect sentiment classification for Opinion Mining. In order to reduce the need for a large number of labeled examples, a semi-supervised approach was proposed, called Aspect-based Sentiment Propagation on Heterogeneous Networks (ASPHN). In this approach, text representations are proposed in which linguistic attributes, candidate aspects and sentiment labels are modeled by heterogeneous networks. Also, a cross-domain learning approach called Cross-Domain Aspect Label Propagation through Heterogeneous Networks (CD-ALPHN) is proposed in order to reduce efforts to build domain-specific resources, This approach uses labeled data from other domains to support learning tasks in domains without labeled data. A representation in a heterogeneous network and a label propagation method are proposed in this cross-domain learning approach. The vertices of the network are the labeled aspects of the source domain, the linguistic attributes, and the candidate aspects of the target domain. In addition, aspect extraction methods were analyzed and some variations were proposed to consider unsupervised and domain independent scenarios. The solutions proposed in this doctoral thesis were evaluated and compared to the state-of-the-art solutions using collections of different product and service reviews. The results obtained in the experimental evaluations are competitive and demonstrate that the proposed solutions are promising.

Para desenvolver um novo medicamento, pesquisadores devem analisar os alvos biológicos de uma dada doença, descobrir e desenvolver candidatos a fármacos para este alvo biológico, realizando em paralelo, testes em laboratório para validar a eficiência e os efeitos colaterais da substância química. O estudo quantitativo da relação estrutura-atividade (QSAR) envolve a construção de modelos de regressão que relacionam um conjunto de descritores de um composto químico e a sua atividade biológica com relação a um ou mais alvos no organismo. Os conjuntos de dados manipulados pelos pesquisadores para análise QSAR são caracterizados geralmente por um número pequeno de instâncias e isso torna mais complexa a construção de modelos preditivos. Nesse contexto, a transferência de conhecimento utilizando informações de outros modelos QSAR\'s com mais dados disponíveis para o mesmo alvo biológico seria desejável, diminuindo o esforço e o custo do processo para gerar novos modelos de descritores de compostos químicos. Este trabalho apresenta uma abordagem de transferência de aprendizagem indutiva (por parâmetros), tal proposta baseia-se em uma variação do método de Regressão por Vetores Suporte adaptado para transferência de aprendizagem, a qual é alcançada ao aproximar os modelos gerados separadamente para cada tarefa em questão. Considera-se também um método de transferência de aprendizagem por instâncias, denominado de TrAdaBoost. Resultados experimentais mostram que as abordagens de transferência de aprendizagem apresentam bom desempenho quando aplicadas a conjuntos de dados de benchmark e a conjuntos de dados químicos / To develop a new medicament, researches must analyze the biological targets of a given disease, discover and develop drug candidates for this biological target, performing in parallel, biological tests in laboratory to validate the effectiveness and side effects of the chemical substance. The quantitative study of structure-activity relationship (QSAR) involves building regression models that relate a set of descriptors of a chemical compound and its biological activity with respect to one or more targets in the organism. Datasets manipulated by researchers to QSAR analysis are generally characterized by a small number of instances and this makes it more complex to build predictive models. In this context, the transfer of knowledge using information other\'s QSAR models with more data available to the same biological target would be desirable, nince its reduces the effort and cost to generate models of chemical descriptors. This work presents an inductive learning transfer approach (by parameters), such proposal is based on a variation of the Vector Regression method Adapted support for learning transfer, which is achieved by approaching the separately generated models for each task. It is also considered a method of learning transfer by instances, called TrAdaBoost. Experimental results show that learning transfer approaches perform well when applied to some datasets of benchmark and dataset chemical

Background. Nowadays, information retrieval system become more and more popular, it helps people retrieve information more efficiently and accelerates daily task. Within this context, Image processing technology play an important role that help transcribing content in printed or handwritten documents into digital data in information retrieval system. This transcribing procedure is called document digitization. In this transcribing procedure, image processing technique such as layout analysis and word recognition are employed to segment the document content and transcribe the image content into words. At this point, a Swedish company (ArkivDigital® AB) has a demand to transcribe their document data into digital data. Objectives. In this study, the aim is to find out effective solution to extract document layout regard to the Swedish handwritten historical documents, which are featured by their tabular forms containing the handwritten content. In this case, outcome of application of OCRopus, OCRfeeder, traditional image processing techniques, machine learning techniques on Swedish historical hand-written document is compared and studied. Methods. Implementation and experiment are used to develop three comparative solutions in this study. One is Hessian filtering with mask operation; another one is Gabor filtering with morphological open operation; the last one is Gabor filtering with machine learning classification. In the last solution, different alternatives were explored to build up document layout extraction pipeline. Hessian filter and Gabor filter are evaluated; Secondly, filter images with the better filter evaluated at previous stage, then refine the filtered image with Hough line transform method. Third, extract transfer learning feature and custom feature. Fourth, feed classifier with previous extracted features and analyze the result. After implementing all the solutions, sample set of the Swedish historical handwritten document is applied with these solutions and compare their performance with survey. Results. Both open source OCR system OCRopus and OCRfeeder fail to deliver the outcome due to these systems are designed to handle general document layout instead of table layout. Traditional image processing solutions work in more than a half of the cases, but it does not work well. Combining traditional image process technique and machine leaning technique give the best result, but with great time cost. Conclusions. Results shows that existing OCR system cannot carry layout analysis task in our Swedish historical handwritten document. Traditional image processing techniques are capable to extract the general table layout in these documents. By introducing machine learning technique, better and more accurate table layout can be extracted, but comes with a bigger time cost. / Scalable resource-efficient systems for big data analytics

<p>In recent years, deep learning models have become popular in the real-time embedded application, but there are many complexities for hardware deployment because of limited resources such as memory, computational power, and energy. Recent research in the field of deep learning focuses on reducing the model size of the Convolution Neural Network (CNN) by various compression techniques like Architectural compression, Pruning, Quantization, and Encoding (e.g., Huffman encoding). Network pruning is one of the promising technique to solve these problems.</p> <p>This thesis proposes methods to prune the convolution neural network (SqueezeNet) without introducing network sparsity in the pruned model. </p> <p>This thesis proposes three methods to prune the CNN to decrease the model size of CNN without a significant drop in the accuracy of the model.</p> <p>1: Pruning based on Taylor expansion of change in cost function Delta C.</p> <p>2: Pruning based on L₂ normalization of activation maps.</p> <p>3: Pruning based on a combination of method 1 and method 2.</p><p>The proposed methods use various ranking methods to rank the convolution kernels and prune the lower ranked filters afterwards SqueezeNet model is fine-tuned by backpropagation. Transfer learning technique is used to train the SqueezeNet on the CIFAR-10 dataset. Results show that the proposed approach reduces the SqueezeNet model by 72% without a significant drop in the accuracy of the model (optimal pruning efficiency result). Results also show that Pruning based on a combination of Taylor expansion of the cost function and L₂ normalization of activation maps achieves better pruning efficiency compared to other individual pruning criteria and most of the pruned kernels are from mid and high-level layers. The Pruned model is deployed on BlueBox 2.0 using RTMaps software and model performance was evaluated.</p><p></p>

Para desenvolver um novo medicamento, pesquisadores devem analisar os alvos biológicos de uma dada doença, descobrir e desenvolver candidatos a fármacos para este alvo biológico, realizando em paralelo, testes em laboratório para validar a eficiência e os efeitos colaterais da substância química. O estudo quantitativo da relação estrutura-atividade (QSAR) envolve a construção de modelos de regressão que relacionam um conjunto de descritores de um composto químico e a sua atividade biológica com relação a um ou mais alvos no organismo. Os conjuntos de dados manipulados pelos pesquisadores para análise QSAR são caracterizados geralmente por um número pequeno de instâncias e isso torna mais complexa a construção de modelos preditivos. Nesse contexto, a transferência de conhecimento utilizando informações de outros modelos QSAR\'s com mais dados disponíveis para o mesmo alvo biológico seria desejável, diminuindo o esforço e o custo do processo para gerar novos modelos de descritores de compostos químicos. Este trabalho apresenta uma abordagem de transferência de aprendizagem indutiva (por parâmetros), tal proposta baseia-se em uma variação do método de Regressão por Vetores Suporte adaptado para transferência de aprendizagem, a qual é alcançada ao aproximar os modelos gerados separadamente para cada tarefa em questão. Considera-se também um método de transferência de aprendizagem por instâncias, denominado de TrAdaBoost. Resultados experimentais mostram que as abordagens de transferência de aprendizagem apresentam bom desempenho quando aplicadas a conjuntos de dados de benchmark e a conjuntos de dados químicos / To develop a new medicament, researches must analyze the biological targets of a given disease, discover and develop drug candidates for this biological target, performing in parallel, biological tests in laboratory to validate the effectiveness and side effects of the chemical substance. The quantitative study of structure-activity relationship (QSAR) involves building regression models that relate a set of descriptors of a chemical compound and its biological activity with respect to one or more targets in the organism. Datasets manipulated by researchers to QSAR analysis are generally characterized by a small number of instances and this makes it more complex to build predictive models. In this context, the transfer of knowledge using information other\'s QSAR models with more data available to the same biological target would be desirable, nince its reduces the effort and cost to generate models of chemical descriptors. This work presents an inductive learning transfer approach (by parameters), such proposal is based on a variation of the Vector Regression method Adapted support for learning transfer, which is achieved by approaching the separately generated models for each task. It is also considered a method of learning transfer by instances, called TrAdaBoost. Experimental results show that learning transfer approaches perform well when applied to some datasets of benchmark and dataset chemical

<p>This dissertation proposes an approach to reduce the cost of manual inspections for as large a number of false positive warnings that are being reported by Static Code Analysis (SCA) tools as much as possible using Machine Learning (ML) techniques. The proposed approach neither assume to use the particular SCA tools nor depends on the specific programming language used to write the target source code or the application. To reduce the number of false positive warnings we first evaluated a number of SCA tools in terms of software engineering metrics using a highlighted synthetic source code named the Juliet test suite. From this evaluation, we concluded that the SCA tools report plenty of false positive warnings that need a manual inspection. Then we generated a number of datasets from the source code that forced the SCA tool to generate either true positive, false positive, or false negative warnings. The datasets, then, were used to train four of ML classifiers in order to classify the collected warnings from the synthetic source code. From the experimental results of the ML classifiers, we observed that the classifier that built using the Random Forests</p> <p>(RF) technique outperformed the rest of the classifiers. Lastly, using this classifier and an instance-based transfer learning technique, we ranked a number of warnings that were aggregated from various open-source software projects. The experimental results show that the proposed approach to reduce the cost of the manual inspection of the false positive warnings outperformed the random ranking algorithm and was highly correlated with the ranked list that the optimal ranking algorithm generated.</p>

With the rise of Deep Learning the field of Artificial Intelligence (AI) Research has entered a new era. Together with an increasing amount of data and vastly improved computing capabilities, Machine Learning builds the backbone of AI, providing many of the tools and algorithms that drive development and applications. While we have already achieved many successes in the fields of image recognition, language processing, recommendation engines, robotics, or autonomous systems, most progress was achieved when the algorithms were focused on learning only a single task with little regard to effort and reusability. Since learning a new task from scratch often involves an expensive learning process, in this work, we are considering the use of previously acquired knowledge to speed up the learning of a new task. For that, we investigated the application of Transfer Learning methods for Deep Reinforcement Learning (DRL) agents and propose a novel framework for knowledge preservation and reuse. We show, that the knowledge transfer can make a big difference if the source knowledge is chosen carefully in a systematic approach. To get to this point, we provide an overview of existing literature of methods that realize knowledge transfer for DRL, a field which has been starting to appear frequently in the relevant literature only in the last two years. We then formulate the Case-based Reasoning methodology, which describes a framework for knowledge reuse in general terms, in Reinforcement Learning terminology to facilitate the adaption and communication between the respective communities. Building on this framework, we propose Deep Case-based Policy Inference (DECAF) and demonstrate in an experimental evaluation the usefulness of our approach for sequential task learning with knowledge preservation and reuse. Our results highlight the benefits of knowledge transfer while also making aware of the challenges that come with it. We consider the work in this area as an important step towards more stable general learning agents that are capable of dealing with the most complex tasks, which would be a key achievement towards Artificial General Intelligence. / Com a evolução da Aprendizagem Profunda (Deep Learning), o campo da Inteligência Artificial (IA) entrou em uma nova era. Juntamente com uma quantidade crescente de dados e recursos computacionais cada vez mais aprimorados, o Aprendizado de Máquina estabelece a base para a IA moderna, fornecendo muitas das ferramentas e algoritmos que impulsionam seu desenvolvimento e aplicações. Apesar dos muitos sucessos nas áreas de reconhecimento de imagem, processamento de linguagem natural, sistemas de recomendação, robótica e sistemas autônomos, a maioria dos avanços foram feitos focando no aprendizado de apenas uma única tarefa, sem muita atenção aos esforços dispendidos e reusabilidade da solução. Como o aprendizado de uma nova tarefa geralmente envolve um processo de aprendizado despendioso, neste trabalho, estamos considerando o reúso de conhecimento para acelerar o aprendizado de uma nova tarefa. Para tanto, investigamos a aplicação dos métodos de Transferência de Aprendizado (Transfer Learning) para agentes de Aprendizado por Reforço profundo (Deep Reinforcement Learning - DRL) e propomos um novo arcabouço para preservação e reutilização de conhecimento. Mostramos que a transferência de conhecimento pode fazer uma grande diferença no aprendizado se a origem do conhecimento for escolhida cuidadosa e sistematicamente. Para chegar a este ponto, nós fornecemos uma visão geral da literatura existente de métodos que realizam a transferência de conhecimento para DRL, um campo que tem despontado com frequência na literatura relevante apenas nos últimos dois anos. Em seguida, formulamos a metodologia Raciocínio baseado em Casos (Case-based Reasoning), que descreve uma estrutura para reutilização do conhecimento em termos gerais, na terminologia de Aprendizado por Reforço, para facilitar a adaptação e a comunicação entre as respectivas comunidades. Com base nessa metodologia, propomos Deep Casebased Policy Inference (DECAF) e demonstramos, em uma avaliação experimental, a utilidade de nossa proposta para a aprendizagem sequencial de tarefas, com preservação e reutilização do conhecimento. Nossos resultados destacam os benefícios da transferência de conhecimento e, ao mesmo tempo, conscientizam os desafios que a acompanham. Consideramos o trabalho nesta área como um passo importante para agentes de aprendizagem mais estáveis, capazes de lidar com as tarefas mais complexas, o que seria um passo fundamental para a Inteligência Geral Artificial.

Recent work has shown humanoid robots with spinal columns, instead of rigid torsos, benefit from both better balance and an increased ability to absorb external impact. Similarly, snake robots have shown promise as a viable option for exploration in confined spaces with limited human access, such as during power plant maintenance. Both spines and snakes, as well as hyper-redundant manipulators, can simplify to a model of a system with multiple links. The multi-link inverted pendulum is a well known benchmark problem in control systems due to its ability to accommodate varying model complexity. Such a system is useful for testing new learning algorithms or laying the foundation for autonomous control of more complex devices such as robotic spines and multi-segmented arms which currently use traditional control methods or are operated by humans. It is often easy to view these systems as single-agent learners due to the high level of interaction among the segments. However, as the number of links in the system increases, the system becomes harder to control. This work replaces the centralized learner with a team of coevolved agents. The use of a multiagent approach allows for control of larger systems. The addition of transfer learning not only increases the learning rate, but also enables the training of larger teams which were previously infeasible due to extended training times. The results presented support these claims by examining neuro-evolutionary control of 3-, 6-, and 12-link systems with nominal conditions as well as with sensor noise, actuator noise, and the addition of more complex physics. / Graduation date: 2012

Traditionally, machine learning algorithms assume that training data is provided as a set of independent instances, each of which can be described as a feature vector. In contrast, many domains of interest are inherently multi-relational, consisting of entities connected by a rich set of relations. For example, the participants in a social network are linked by friendships, collaborations, and shared interests. Likewise, the users of a search engine are related by searches for similar items and clicks to shared sites. The ability to model and reason about such relations is essential not only because better predictive accuracy is achieved by exploiting this additional information, but also because frequently the goal is to predict whether a set of entities are related in a particular way. This thesis falls within the area of Statistical Relational Learning (SRL), which combines ideas from two traditions within artificial intelligence, first-order logic and probabilistic graphical models to address the challenge of learning from multi-relational data. We build on one particular SRL model, Markov logic networks (MLNs), which consist of a set of weighted first-order-logic formulae and provide a principled way of defining a probability distribution over possible worlds. We develop algorithms for learning of MLN structure both from scratch and by transferring a previously learned model, as well as an application of MLNs to the problem of Web query disambiguation. The ideas we present are unified by two main themes: the need to deal with limited training data and the use of bottom-up learning techniques. Structure learning, the task of automatically acquiring a set of dependencies among the relations in the domain, is a central problem in SRL. We introduce BUSL, an algorithm for learning MLN structure from scratch that proceeds in a more bottom-up fashion, breaking away from the tradition of top-down learning typical in SRL. Our approach first constructs a novel data structure called a Markov network template that is used to restrict the search space for clauses. Our experiments in three relational domains demonstrate that BUSL dramatically reduces the search space for clauses and attains a significantly higher accuracy than a structure learner that follows a top-down approach. Accurate and efficient structure learning can also be achieved by transferring a model obtained in a source domain related to the current target domain of interest. We view transfer as a revision task and present an algorithm that diagnoses a source MLN to determine which of its parts transfer directly to the target domain and which need to be updated. This analysis focuses the search for revisions on the incorrect portions of the source structure, thus speeding up learning. Transfer learning is particularly important when target-domain data is limited, such as when data on only a few individuals is available from domains with hundreds of entities connected by a variety of relations. We also address this challenging case and develop a general transfer learning approach that makes effective use of such limited target data in several social network domains. Finally, we develop an application of MLNs to the problem of Web query disambiguation in a more privacy-aware setting where the only information available about a user is that captured in a short search session of 5-6 previous queries on average. This setting contrasts with previous work that typically assumes the availability of long user-specific search histories. To compensate for the scarcity of user-specific information, our approach exploits the relations between users, search terms, and URLs. We demonstrate the effectiveness of our approach in the presence of noise and show that it outperforms several natural baselines on a large data set collected from the MSN search engine. / text