University of Thessaly (UTH)
Μηχανική μάθηση και εικονικά περιβάλλοντα για ενίσχυση της ανθρώπινης μάθησης: εφαρμογή στην κατανομή φυσικών πόρων
Abstract
dc:descriptionThis PhD thesis studies the problem of natural resource allocation using machine learningprinciples, such as multi-agent modelling and reinforcement learning, in combination withthe use of virtual reality techniques for the visualization of the research findings.In the areas of agent based modelling and game theory, which set the mathematicalfoundation for the study of the problem, existing modeling approaches and multi-agent simulationplatforms are studied. Research is focused on multi-agent simulation models, anad-hoc technique for modelling social and natural phenomena with interacting users. Moreprecisely, a novel multi-agent simulation model (stochastic game) is introduced, that simulatesthe exploitation of a natural resource by a community of users that exploit the resourcein order to survive. The proposed model is capable of simulating various resource allocationpolicies and various user behavours, while its novelty lies in the fact that it encapsulatessocio-economical prameters. In more detail, it is argued that as the resource becomes scarce,possible relations of trust or communication channels that may exist between the users ofthe resource become unreliable, thus users act upon self-lucrative criteria. To encapsulatethis behaviour, the users of the proposed model are not considered to interact (i.e. observeor communicate) with each other. The proposed model was calibrated and validated usingempricial data and measurements from a real life ecosystem, which serves as a representativeexample of the failure of centralized resource allocation policies. To fully investigate theproblem, three real life resource allocation policies (irrigation policies) were implemented,whereas the experimental procedure entailed simulations for several user behaviours (greediness),generated through a Monte Carlo procedure. Experimental results were evaluatedusing appropriate figures of merit, which were defined specifically for the purposes of thisstudy, that investigate the influence of the resource allocation policies to the sustainabilityof the resource, of the users and of the overall ecosystem (users and resource simultaneously).Experimental results confirm the validity of the model and reveal the complexity ofthe problem, as none of the resource allocation policies manages to provide efficient resourceallocations that simulatneously preserve the resource and ensure the survival of its users.In order to investigate viable solutions for the aforementioned problem, research is extendedin the area of multi-agent learning and machine learning in general. Existing multiagentlearning algorithms were studied, and a novel multi-agent learning approach was introducedthat reveals efficient resource allocations without violating the constraints imposedby the problem (i.e. lack of inter-agent communication and observation). The proposedapproach, called Adaptive Rectification - AR employs a parametrized rectification functionthat encapsulates domain knowledge about the problem under study and reveals the wayagents should modify their actions in order to achieve efficient resource allocation. Four AR methods are introduced in this study, that employ different methods for the estimation of theparameters of the rectification function. More precisely the AR-Empirical method is introduced,where the parameters of the rectification function are empirically defined, the AR-RealCoded Genetic Algorithms (AR-RCGA) and AR-Covariance Matric Adaptation (AR-CMA)methods where the rectification function is estimated using stochastic optimization methodsthrough a black box optimization procedure, and finally the AR Fuzzy Particle Swarm Optimization(AR-FPSO) method where the rectification function is approximated using a fuzzyinference system that is trained using particle swarm optimization. The proposed methodsare compared to general purpose multi-agent learning algorithms and to resource allocationpolicies, while the evaluation is conducted using the same figures of merit that were also usedto investigate the influence of resource allocation policies. Experiments entail a Monte Carloprocedure that generates agent populations of varying behaviours, and the produced resultsreveal the complexity of learning in such constrained environments and the superiority ofthe proposed methods in achieving efficient resource allocations that simultaneously preservethe resource and maximize the utilities of its users.Finally, the use of virtual reality techniques for the visualization of the research findings,i.e. the problem, its complexity and its solution, is investigated. Research is focused on threedimensional virtual environments, which immerse the user in a world that replicates the realworld, provide him with means to interact in ways that are not possible in the real world,and thus making learning easier and more pleasant. The novelty of the proposed virtual environmentstems from its integration with the aforementioned multi-agent simulation model,that controls the simulations of the virtual world. In this sense, the virtual environmentacts as an interface between the learner and the multi-agent model, protecting him from themathematical complexity of the environmental model and allowing him to focus only on itsimportant parts. The virtual environment was developed with the active participation ofstudents, and the learning advantage that it provides was investigated through learning goalsthat stem from the analytical cirriculum of the Hellenic Ministry of Education. These goalsare related with the scarcity of water resources, the identification of the problems that arisefrom water overdraining and the realization of efficient solutions to the problem (throughthe proposed Adaptive Rectification methods). The evaluation of the virtual environmentis conducted by 24 students that are evenly divided in a control group, which is educatedusing conventional means, and an experimental group that interacts exclusively with theproposed virtual environment. Experimental results are gathered using a questionaire thatwas developed for the purposes of this study, which is filled by the students of both groupsprior (pre-test) and after (post-test) the learning process. Results reveal the effectivenessof the proposed virtual environment, that aids the students realize complex environmental concepts.
Degree
thesis:*- Grantor dc:publisher
- University of Thessaly (UTH)
- Year dc:date
- 2013
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Barmpalios, Nikolaos
- Μπαρμπαλιός, Νικόλαος
Subjects
dc:subject × 18- Μηχανική μάθηση
- Πολυπρακτορική μάθηση
- Ενισχυτική μάθηση
- Στοχαστική βελτιστοποίηση
- Περιβαλλοντική μοντελοποίηση
- Εικονικά περιβάλλοντα
- Δίλλημα κοινόχρηστων πόρων
- Machine learning
- Multiagent learning
- Reinforcement learning
- Stochastic optimization
- Enviromental modeling
- Virtual environments
- Tragedy of the commons dilemma
- Φυσικές Επιστήμες
- Επιστήμη Ηλεκτρονικών Υπολογιστών και Πληροφορική
- Natural Sciences
- Computer and Information Sciences
Rights
- Language dc:language
- gre
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/41556