31 |
A Reasoning Module for Long-lived Cognitive AgentsVassos, Stavros 03 March 2010 (has links)
In this thesis we study a reasoning module for agents that have cognitive abilities, such as memory, perception, action, and are expected to function autonomously for long periods of time. The module provides the ability to reason about action and change using the language of the situation calculus and variants of the basic action theories. The main focus of this thesis is on the logical problem of progressing an action theory.
First, we investigate the conjecture by Lin and Reiter that a practical first-order definition of progression is not appropriate for the general case. We show that Lin and Reiter were indeed correct in their intuitions by providing a proof for the conjecture, thus resolving the open question about the first-order definability of progression and justifying the need for a second-order definition.
Then we proceed to identify three cases where it is possible to obtain a first-order progression with the desired properties: i) we extend earlier work by Lin and Reiter and present a case where we restrict our attention to a practical class of queries that may only quantify over situations in a limited way; ii) we revisit the local-effect assumption of Liu and Levesque that requires that the effects of an action are fixed by the arguments of the action and show that in this case a first-order progression is suitable; iii) we investigate a way that the local-effect assumption can be relaxed and show that when the initial knowledge base is a database of possible closures and the effects of the actions are range-restricted then a first-order progression is also suitable under a just-in-time assumption.
Finally, we examine a special case of the action theories with range-restricted effects and present an algorithm for computing a finite progression. We prove the correctness and the complexity of the algorithm, and show its application in a simple example that is inspired by video games.
|
32 |
Resource-Bounded Reasoning about KnowledgeHo, Ngoc Duc 28 November 2004 (has links) (PDF)
Der Begriff ``Agent'' hat sich als eine sehr nützliche Abstraktion erwiesen, um verschiedene Problembereiche auf eine intuitive und natürliche Art und Weise zu konzeptualisieren. Intelligente Agenten haben daher Anwendung gefunden in verschiedenen Teilbereichen der Informatik. Zur Modellierung werden intelligente Agenten meist als intentionale Systeme aufgefaßt und mit Hilfe von mentalistischen Begriffen wie Wissen, Glauben (oder Überzeugung), Wunsch, Pflicht, Intention usw. beschrieben. Unter diesen mentalen Begriffen gehören die epistemischen Begriffe (d.h., Wissen und Glauben) zu den wichtigsten und wurden auch am intensivsten untersucht. Zur Modellierung von Wissen und Glauben werden in der Regel modale epistemische Logiken verwendet. Solche Systeme sind aber nicht geeignet, um ressourcenbeschränkte Agenten zu beschreiben, weil sie zu starke Annahmen bezüglich der Rationalität von Agenten machen. Zum Beispiel wird angenommen, daß Agenten alle logischen Wahrheiten sowie alle Konsequenzen seines Wissens kennen. Dieses Problem ist bekannt als das Problem der logischen Allwissenheit (``logical omniscience problem''). Da alle Agenten grundsätzlich nur über begrenzte Ressourcen (wie z.B. Zeit, Information, Speicherplatz) verfügen, können sie nur eine begrenzte Menge von Informationen verarbeiten. Daher müssen alternative Modelle entwickelt werden, um Agenten realistisch modellieren zu können (siehe Kapitel 2). Daß modale epistemische Logik für die Formalisierung des ressourcenbeschränkten Schließens (``resource-bounded reasoning'') nicht geeignet ist, wird als ein offenes Problem der Agententheorien anerkannt. Es gibt bisher aber keine brauchbaren Alternativen zur Modallogik. Die meisten Ansätze zur Lösung des logischen Allwissenheitsproblems versuchen, Wissen und Glauben mit Hilfe schwacher Modallogiken zu beschreiben. Solche Versuche sind nicht befriedigend, da sie eine willkürliche Einschränkung der Rationalität der Agenten zur Folge haben (siehe Kapitel 3). Mein Ziel ist es, einen Rahmen für das ressourcenbeschränktes Schließen über Wissen und Glauben zu entwickeln. Damit soll eine solide Grundlage für Theorien intelligenter Agenten geschaffen werden. Als Nebenergebnis wird das logische Allwissenheitsproblem auf eine sehr intuitive Art und Weise gelöst: obwohl Agenten rational sind und alle logischen Schlußregeln anwenden können, sind sie nicht logisch allwissend, weil ihnen nicht genügend Ressourcen zu Verfügung stehen, um alle logischen Konsequenzen ihres Wissens zu ziehen. Im Kapitel 4 wird eine Reihe von Logiken vorgestellt, die den Begriff des expliziten Wissens formalisieren. Es wird eine Lösung des Problems der logischen Allwissenheit der epistemischen Logik vorgeschlagen, die die Rationalität der Agenten nicht willkürlich einschränkt. Der Grundgedanke dabei ist der folgende. Ein Agent kennt die logischen Konsequenzen seines Wissens nur dann, wenn er sie tatsächlich hergeleitet hat. Wenn ein Agent alle Prämissen einer gültigen Schlußregel kennt, kennt er nicht notwendigerweise die Konklusion: er kennt sie nur nach der Anwendung der Regel. Wenn er den Schluß nicht ziehen kann, z.B. weil er nicht die notwendigen Ressourcen dazu hat, wird sein Wissen nicht um diese herleitbare Information erweitert. Die Herleitung neuer Informationen wird als die Ausführung mentaler Handlungen aufgefaßt. Mit Hilfe einer Variante der dynamischen Logik können diese Handlungen beschrieben werden. Im Kapitel 5 werden Systeme für das ressourcenbeschränkte Schließen über Wissen und Glauben entwickelt, die auch quantitative Bedingungen über die Verfügbarkeit von Ressourcen modellieren können. Mit Hilfe dieser Logiken können Situationen beschrieben werden, wo Agenten innerhalb einer bestimmten Zeitspanne entscheiden müssen, welche Handlungen sie ausführen sollen. Der Ansatz besteht darin, epistemische Logik mit Komplexitätstheorie zu verbinden. Mit Hilfe einer Komplexitätsanalyse kann ein Agent feststellen, ob ein bestimmtes Problem innerhalb vorgegebener Zeit lösbar ist. Auf der Grundlage dieses Wissens kann er dann die für die Situation geeignete Entscheidung treffen. Damit ist es gelungen, eine direkte Verbindung zwischen dem Wissen eines Agenten und der Verfügbarkeit seiner Ressourcen herzustellen. / One of the principal goals of agent theories is to describe realistic, implementable agents, that is, those which have actually been constructed or are at least in principle implementable. That goal cannot be reached if the inherent resource-boundedness of agents is not treated correctly. Since the modal approach to epistemic logic is not suited to formalize resource-bounded reasoning, the issue of resource-boundedness remains one of the main foundational problems of any agent theory that is developed on the basis of modal epistemic logic. My work is an attempt to provide theories of agency with a more adequate epistemic foundation. It aims at developing theories of mental concepts that make much more realistic assumptions about agents than other theories. The guiding principle of my theory is that the capacities attributed to agents must be empirically verifiable, that is, it must be possible to construct artificial agents which satisfy the specifications determined by the theory. As a consequence, the unrealistic assumption that agents have unlimited reasoning capacities must be rejected. To achieve the goal of describing resource-bounded agents accurately, the cost of reasoning must be taken seriously. In the thesis I have developed a framework for modeling the relationship between knowledge, reasoning, and the availability of resources. I have argued that the correct form of an axiom for epistemic logic should be: if an agent knows all premises of a valid inference rule and if he performs the right reasoning, then he will know the conclusion as well. Because reasoning requires resources, it cannot be safely assumed that the agent can compute his knowledge if he does not have enough resources to perform the required reasoning. I have demonstrated that on the basis of that idea, the problems of traditional approaches can be avoided and rich epistemic logics can be developed which can account adequately for our intuitions about knowledge.
|
33 |
Intelligent agent control of an unmanned aerial vehicle /Carryer, J. Andrew January 1900 (has links)
Thesis (M.App.Sc.) - Carleton University, 2005. / Includes bibliographical references (p. 172-178). Also available in electronic format on the Internet.
|
34 |
A Logical Theory of Joint Ability in the Situation CalculusGhaderi, Hojjat 17 February 2011 (has links)
Logic-based formalizations of dynamical systems are central to the field of knowledge representation and reasoning. These formalizations can be used to model agents that act, reason,and perceive in a changing and incompletely known environment. A key aspect of reasoning about agents and their behaviors is the notion of joint ability. A team of agents is jointly able
to achieve a goal if despite any incomplete knowledge or even false beliefs about the world or each other, they still know enough to be able to get to a goal state, should they choose to do so. A particularly challenging issue associated with joint ability is how team members can coordinate their actions. Existing approaches often require the agents to communicate to agree
on a joint plan. In this thesis, we propose an account of joint ability that supports coordination among agents without requiring communication, and that allows for agents to have incomplete (or even false) beliefs about the world or the beliefs of other agents. We use ideas from game theory to address coordination among agents. We introduce the notion of a strategy for each
agent which is basically a plan that the agent knows how to follow. Each agent compares her strategies and iteratively discards those that she believes are not good considering the strategies that the other agents have kept. Our account is developed in the situation calculus, a logical language suitable for representing and reasoning about action and change that is extended to support reasoning about multiple agents. Through several examples involving public, private, and sensing actions, we demonstrate how symbolic proof techniques allow us to reason about team ability despite incomplete specifications about the beliefs of agents.
|
35 |
[en] HIERARCHICAL NEURAL FUZZY MODELS BASED ON REINFORCEMENT LEARNING OF INTELLIGENT AGENTS / [pt] MODELOS NEURO-FUZZY HIERÁRQUICOS COM APRENDIZADO POR REFORÇO PARA MULTI-AGENTES INTELIGENTESMARCELO FRANCA CORREA 20 February 2013 (has links)
[pt] Os benefícios trazidos pela aplicação de Sistemas Multi-Agentes (SMA) são
diversos. Através da computação paralela, agentes podem trabalhar em conjunto para
explorar melhor a estrutura descentralizada de uma determinada tarefa e acelerar sua
conclusão. Além disso, agentes também podem trocar experiências se comunicando,
fornecer alto grau de escalabilidade, através da inclusão de novos agentes quando
necessário, e ainda fazer com que agentes assumam as atividades de outros agentes em
casos de falha. Vários modelos de agentes desenvolvidos até o momento usam o
aprendizado por reforço como algoritmo base no processo de aprendizado. Quando o
agente está inserido em ambientes pequenos ou discretos, os resultados obtidos com o uso
de métodos como Q-learning são satisfatórios. No entanto, quando o ambiente é grande
ou contínuo, o uso de métodos de aprendizado por reforço torna-se inviável, devido à
grande dimensão do espaço de estados. Nos SMA, este problema é consideravelmente
maior, já que a memória necessária passa a crescer exponencialmente com a quantidade
de agentes envolvidos na aplicação. Esta tese teve como finalidade o desenvolvimento de
um novo modelo de aprendizado autônomo para Sistemas Multi-Agentes (SMA) visando
superar estas limitações. O trabalho foi realizado em três etapas principais: levantamento
bibliográfico, seleção e implementação do modelo proposto, e desenvolvimento de estudo
de casos. O levantamento bibliográfico contemplou o estudo de agentes inteligentes e
Sistemas Multi-Agentes, buscando identificar as propriedades e limitações dos algoritmos
já desenvolvidos, as aplicações existentes, e as características desejadas em um SMA. A
seleção e utilização de um modelo neuro-fuzzy hierárquico da família RL-NFH foi
motivada especialmente pela importância de se estender a autonomia e aprendizado de
agentes através do quesito inteligência, e pela sua capacidade de superar limitações
presentes em algoritmos de aprendizado por reforço tradicionais. Inicialmente, ao modelo
anterior foram adicionados os conceitos de satisfatoriedade e não-dominação, com a
finalidade de acelerar o processo de aprendizado do algoritmo. Em seguida, o novo
modelo multi-agente foi criado, viabilizando o desenvolvimento de aplicações de
natureza tanto cooperativa como competitiva, com múltiplos agentes. Os estudos de caso
contemplaram situações distintas de cooperação e competição entre agentes autônomos.
Foram implementadas três aplicações distintas: uma aplicação benckmark do jogo da
presa-predador (Pursuit-Game); um leilão energia elétrica, em que os fornecedores de
energia fazem ofertas para atender à previsão de demanda em um período de tempo
determinado; e uma aplicação na área de gerenciamento de projetos, onde agentes
inteligentes são criados com o objetivo de fornecer estimativas de duração de atividades e
automatizar alguns processos realizados pelo Gerente de Projetos. Em todos os Estudos
de Caso, os resultados foram comparados com técnicas convencionais e/ou com o
desempenho de outros Sistemas Multi-Agente. Os resultados alcançados pelo novo
modelo se mostraram promissores. Os testes evidenciaram que o modelo teve a
capacidade de coordenar as ações entre agentes totalmente autônomos em diferentes
situações e ambientes. Além disso, o novo modelo mostrou-se genérico e flexível,
podendo ser usado no futuro em outras aplicações envolvendo múltiplos agentes. / [en] There are several benefits provided by Multi-Agent Systems (MAS). Through
parallel computing, agents can work together to better explore the decentralized structure
of a given task and speed up its completion. In addition, agents can also exchange
knowledge through communication, provide scalability by adding new agents when
appropriate, and replace troubled agents in cases of failures. A great number of existing
agent models is based on reinforcement learning algorithms for learning. When the agent
works in small or discrete environments, the results obtained with methods such as Qlearning
are satisfactory. However, when the environment is large or continuous
reinforcement learning methods become unfeasible due to the large state space. In MAS,
this problem is considerably greater, since the required memory begins to grow
exponentially with the number of agents involved in the application. The main objective
of this thesis is to develop a new model of autonomous learning for multi-agents in order
to overcome these limitations. The study consisted of three main stages: literature review,
new model development and implementation, and case studies. Literature review
included the study of intelligent agents and Multi-Agent Systems, seeking to identify the
properties and limitations of the algorithms already developed, existing applications, and
desired features in the new MAS. The choice of a neuro-fuzzy hierarchical model of the
family RL-NFH as a basis was especially motivated by the importance of extending the
autonomy and learning of the agents through intelligence. And also, because of its
capacity to overcome some of the limitations present in traditional reinforcement learning
algorithms. Initially, the concepts of satisficing and non-domination were incorporated
into the previous model to accelerate the learning algorithm. Then, the new multi-agent
model was elaborated and implemented, enabling the development of cooperative and
competitive applications, with multiple agents. Case studies have covered different
situations of cooperation and competition between autonomous agents. Three applications
were considered: the Pursuit-Game benckmark game, an electricity auction, where energy
suppliers make offers to meet forecast demand in a given period of time, and an
application in project management area, where intelligent agents are created to provide
activity duration estimates and to automate some processes done usually by the Project
Manager. In all case studies, results were compared with conventional techniques and/or
the performance of other MAS. The results achieved by the new model are encouraging.
The tests showed that the new system has the capacity to coordinate actions between fully
autonomous agents in different situations and environments. Moreover, the new model is
strongly generic and flexible. Due to these properties, it can be used in future in several
other applications involving multiple agents.
|
36 |
MECANISMO DE AUXÍLIO AO PROFESSOR PARA O AMBIENTE NETCLASS DE ENSINO-APRENDIZAGEM. / Aid mechanism to the teacher for the environment teaching-learning netclass.BRANDÃO NETO, PEDRO 18 June 2007 (has links)
Submitted by Maria Aparecida (cidazen@gmail.com) on 2017-08-22T12:35:16Z
No. of bitstreams: 1
Pedro Brandão Neto.pdf: 938638 bytes, checksum: 79ae100045510d32510d4a705eee798f (MD5) / Made available in DSpace on 2017-08-22T12:35:16Z (GMT). No. of bitstreams: 1
Pedro Brandão Neto.pdf: 938638 bytes, checksum: 79ae100045510d32510d4a705eee798f (MD5)
Previous issue date: 2007-06-18 / FAPEMA / Distance Education environments, nowadays, provide means for learner
evaluation. This task, still, requires effort of the teacher. So, the learner
accompaniment in distance teaching-learning environments, obtained through
the observation of the learner activities, it presents referring problems to the
lack of mechanisms that offer an effective support to the teacher in the
orientation for the learner. These existent mechanisms only present results to
the teacher without to do any inference, like this, self teacher needs to analyze
those results manually in search of subsidies to do a precise orientation,
showing the possible causes and addressing the learner for the correct road.
However, to dower a mechanism with inference capacity is necessary to
capture the knowledge of as the teacher fulfills the leaner accompaniment for,
later, to transfer this knowledge for a knowledge base with inference capacity.
The necessary knowledge for the teacher to realize the learner accompaniment
in a teaching-learning environment is the evaluation criterions that are defined
by course pedagogic coordination or teacher of the teaching institution.
However, those evaluation criterions are generic and not standardized
hindering, therefore, the consent of the same; was realized a theoretical rising
for the definition of the evaluation criterions in the NetClass environment. This
research generated the knowledge on the development of a model
denominated Learner Accompaniment Support Mechanism based on
techniques of Artificial Intelligence, which is applied to the NetClass
environment of teaching-learning. / Os ambientes de Educação a Distância, atualmente, proporcionam meios de
avaliação de o aprendiz. Essa tarefa, de fato, ainda, exige muito esforço e
empenho da parte do professor. Portanto, o acompanhamento do aprendiz em
ambientes de ensino-aprendizagem a distância, obtida através da observação
das atividades dos aprendizes, apresenta problemas referentes à carência de
mecanismos que ofereçam um suporte eficaz ao professor na orientação para
o aprendiz. Esses mecanismos existentes somente apresentam resultados ao
professor sem fazer qualquer inferência, desta maneira, o próprio professor
precisa analisar manualmente esses resultados em busca de subsídios para
fazer uma orientação precisa, mostrando as possíveis causas e direcionando o
aprendiz para o caminho correto. Entretanto, para dotar um mecanismo com
capacidade de inferência faz-se necessário capturar o conhecimento de como
o professor realiza o acompanhamento do aprendiz para, posteriormente,
transferir este conhecimento para uma base de conhecimento com capacidade
de inferência. O conhecimento necessário para o professor realizar o
acompanhamento do aprendiz em um ambiente de ensino-aprendizagem são
os critérios de avaliação que são definidos pela coordenação pedagógica ou
professor do curso da instituição de ensino. No entanto, esses critérios de
avaliação são genéricos e não padronizadas dificultando, portanto, o consenso
do mesmo; foi realizado um levantamento teórico para a definição dos critérios
de avaliação no ambiente NetClass. Esta pesquisa gerou o conhecimento
sobre o desenvolvimento de um modelo de um mecanismo denominado de
Mecanismo de Apoio no Acompanhamento do Aprendiz (MAAA) o qual é
aplicado ao ambiente NetClass de ensino-aprendizagem.
|
37 |
多樣需求與資源環境中垃圾桶模式之e化服務決策研究 / Manifold Needs and Resources:Garbage Can Model of e-Service Perspective呂知穎, Lu, Chih-Ying Unknown Date (has links)
為因應人類生理或心理上的需求,而產生了形形色色之服務。隨著高科技不斷地發展,人類的未來生活,將會是充滿e化服務的生活環境。在此環境中,並非所有人均能了解各應用服務,更不知該選擇何服務才能滿足自身之多重需求。本研究擬設計一決策機制,當人們有多重需求時,能考慮有形及無形資源之有效利用,並考量不同個體之使用偏好及興趣,提供適合個人的e化服務建議。本研究之應用環境,符合垃圾桶模式中的無政府狀態之三大特性,然而原垃圾桶決策方式卻不適用於個人。因此,本研究之主體,為一智慧代理人,將以垃圾桶模式的決策原理做為基礎,並對其加以修改,分為二階段的決策過程。在第一階段,將使用一考量資源使用效率之task-chosen演算法,並搭配增強式學習中之AH-learning演算法;在第二階段,則是使用BDI代理人的架構。本研究所提出之提供e化服務建議的決策機制,預期將促使應用服務能不斷地創新及進步,並使資源獲得更有效之利用,使得人類擁有高品質的生活環境。 / There are manifold services, in order to fulfill people’s physical and mental needs. Through the continuous development of high technique, people will live in the environment surrounding e-services in the future. In this environment, it is hart for everyone to understand all e-services and choose a service to fulfill selves multiple needs. Therefore, the paper presents a decision mechanism which providing suitable e-service suggestion for everyone when they have multiple needs, considering the using utility of resources include tangible and intangible, and different preferences and interests for different people. This paper’s applying environment satisfies the three general properties of organized anarchies of “Garbage Can Model”. However, the decision method in garbage can model is not suitable to individual. The most important part of the paper is an intelligent agent, based on garbage can model theory but modify it appropriately. This intelligent agent uses two phase decision process. First phase, use a task-chosen algorism considering resource utility and AH-learning in reinforcement learning. Second phase, use the architecture of BDI agent. This paper presents a decision strategy providing e-service suggestion, and expects to promote innovative application services and use resource effectively. Finally, all people will enjoy high quality life.
|
38 |
語意式構思學習模式於協同式腦力激盪決策 / Semantic Ideation Learning for Collective Brainstorming陳延全, Chen,Yen-Chuan Unknown Date (has links)
「知識經濟」時代下,知識汰舊換新速度極快,單打獨鬥不及於團隊合作的成效,因此,不論組織或個人均須講求團隊合作。腦力激盪法(Brainstorming)即是透過團隊合作、協同決策的方式產生具有創意的解決方案。本研究結合智慧型代理人的技術與人類獨特的腦力激盪思考方式,利用智慧型代理人的自主性、溝通能力、適應力與學習能力等特性,讓智慧型代理人能在適當的時候代替腦力激盪會議的與會者出席會議,達成會議目標。為了讓智慧型代理人也能模仿人類進行創意思考,本研究以人類主要用來產生創意構思的三種聯想能力做為代理人之推論機制,並結合增強式學習的概念,設計出能根據以本體論表達之概念(Ontology-Based Concept)進行構思激盪之語意式構思學習代理人( Semantic Ideation Learning Agent,SILA ),並架構一個能讓多個SILA進行知識分享與學習的系統環境-腦力激盪式協同決策系統(Collective Brainstorming Decision System, CBDS)。本研究以傳統的腦力激盪決策模式為基礎,結合現代之網路語意表達與代理人技術,期望讓在網路上代表不同角色、身份的代理人,基於其所擁有之構思知識庫 (Idea Knowledge Base),透過代理人之間的溝通與知識分享,達成代理人自動化協同決策(Collective Decision)之目標。 / In Knowledge Economy Era, the organization and individual are emphasizing on the teamwork instead of single play because of better effectiveness. Brainstorming is a solution that can help organization to generate creative ideas through teamwork and collaboration. This research combines human’s unique brainstorming thinking and the intelligent agent technique for devising an automated decision agent called Semantic Ideation Learning Agent (SILA) (that can represent a session participant to engage the action of brainstorming). In order to make a SILA thinking like human, our research presents a method of Reinforcement Learning grounded on three capabilities of human’s association (similarity, contiguity, contrast) as the SILA’s inference mechanism. Furthermore, the Collective Brainstorming Decision System was build to provide an environment where SILAs can learn and share their knowledge. The aim of this research is to reach automatic collective decision in a brainstorming session through the collaboration of the agents based on the brainstorming decision model and some modern information techniques including knowledge base, semantic web and intelligent agents.
|
39 |
A Logical Theory of Joint Ability in the Situation CalculusGhaderi, Hojjat 17 February 2011 (has links)
Logic-based formalizations of dynamical systems are central to the field of knowledge representation and reasoning. These formalizations can be used to model agents that act, reason,and perceive in a changing and incompletely known environment. A key aspect of reasoning about agents and their behaviors is the notion of joint ability. A team of agents is jointly able
to achieve a goal if despite any incomplete knowledge or even false beliefs about the world or each other, they still know enough to be able to get to a goal state, should they choose to do so. A particularly challenging issue associated with joint ability is how team members can coordinate their actions. Existing approaches often require the agents to communicate to agree
on a joint plan. In this thesis, we propose an account of joint ability that supports coordination among agents without requiring communication, and that allows for agents to have incomplete (or even false) beliefs about the world or the beliefs of other agents. We use ideas from game theory to address coordination among agents. We introduce the notion of a strategy for each
agent which is basically a plan that the agent knows how to follow. Each agent compares her strategies and iteratively discards those that she believes are not good considering the strategies that the other agents have kept. Our account is developed in the situation calculus, a logical language suitable for representing and reasoning about action and change that is extended to support reasoning about multiple agents. Through several examples involving public, private, and sensing actions, we demonstrate how symbolic proof techniques allow us to reason about team ability despite incomplete specifications about the beliefs of agents.
|
40 |
[en] METHODS FOR ACCELERATION OF LEARNING PROCESS OF REINFORCEMENT LEARNING NEURO-FUZZY HIERARCHICAL POLITREE MODEL / [pt] MÉTODOS DE ACELERAÇÃO DE APRENDIZADO APLICADO AO MODELO NEURO-FUZZY HIERÁRQUICO POLITREE COM APRENDIZADO POR REFORÇOFABIO JESSEN WERNECK DE ALMEIDA MARTINS 04 October 2010 (has links)
[pt] Neste trabalho foram desenvolvidos e avaliados métodos com o objetivo de melhorar e acelerar o processo de aprendizado do modelo de Reinforcement Learning Neuro-Fuzzy Hierárquico Politree (RL-NFHP). Este modelo pode ser utilizado para dotar um agente de inteligência através de processo de Aprendizado por Reforço (Reinforcement Learning). O modelo RL-NFHP apresenta as seguintes características: aprendizado automático da estrutura do modelo; auto-ajuste dos parâmetros associados à estrutura; capacidade de aprendizado da ação a ser adotada quando o agente está em um determinado estado do ambiente; possibilidade de lidar com um número maior de entradas do que os sistemas neuro-fuzzy tradicionais; e geração de regras linguísticas com hierarquia. Com intenção de melhorar e acelerar o processo de aprendizado do modelo foram implementadas seis políticas de seleção, sendo uma delas uma inovação deste trabalho (Q-DC-roulette); implementado o método early stopping para determinação automática do fim do treinamento; desenvolvido o eligibility trace cumulativo; criado um método de poda da estrutura, para eliminação de células desnecessárias; além da reescrita do código computacional original. O modelo RL-NFHP modificado foi avaliado em três aplicações: o benchmark Carro na Montanha simulado, conhecido na área de agentes autônomos; uma simulação robótica baseada no robô Khepera; e uma num robô real NXT. Os testes efetuados demonstram que este modelo modificado se ajustou bem a problemas de sistemas de controle e robótica, apresentando boa generalização. Comparado o modelo RL-NFHP modificado com o original, houve aceleração do aprendizado e obtenção de menores modelos treinados. / [en] In this work, methods were developed and evaluated in order to improve and accelerate the learning process of Reinforcement Learning Neuro-Fuzzy Hierarchical Politree Model (RL-NFHP). This model is employed to provide an agent with intelligence, making it autonomous, due to the capacity of ratiocinate (infer actions) and learning, acquired knowledge through interaction with the environment by Reinforcement Learning process. The RL-NFHP model has the following features: automatic learning of structure of the model; self-adjustment of parameters associated with its structure, ability to learn the action to be taken when the agent is in a particular state of the environment; ability to handle a larger number of inputs than the traditional neuro-fuzzy systems; and generation of rules with linguistic interpretable hierarchy. With the aim to improve and accelerate the learning process of the model, six selection action policies were developed, one of them an innovation of this work (Q-DC-roulette); implemented the early stopping method for automatically determining the end of the training; developed a cumulative eligibility trace; created a method of pruning the structure, for removing unnecessary cells; in addition to rewriting the original computer code. The modified RL-NFHP model was evaluated in three applications: the simulated benchmark Car-Mountain problem, well known in the area of autonomous agents; a simulated application in robotics based on the Khepera robot; and an application in a real robot. The experiments show that this modified model fits well the problems of control systems and robotics, with a good generalization. Compared the modified RL-NFHP model with the original one, there was acceleration of learning process and smaller structures of the model trained.
|
Page generated in 0.0178 seconds