Die dreidimensionale Kartierung der Umgebung spielt speziell in der Robotik eine große Rolle und ist Grundlage für nahezu alle Aufgaben, die eine nicht rein reaktive Interaktion mit dieser Umgebung darstellen. Die vorliegende Arbeit beschreibt den Weg zu solchen Karten. Angefangen bei der reinen (2D-)Lokalisierung eines mobilen Roboters, als erster, fundamentaler Schritt in Richtung autonomer Exploration und Kartierung, beschreibt der Text die Registrierung von Scans zur automatischen, effizienten Generierung von 3D-Karten und gleichzeitiger Lokalisierung in sechs Freiheitsgraden (SLAM-Problem). Es folgen Lösungsstrategien für den Umgang mit akkumulierten Fehlern gerade bei großen explorierten Gebieten: Eine GraphSLAM-Variante liefert global konsistente Karten, optional unterstützt durch eine echtzeitfähige Heuristik zur online-Schleifenoptimierung. Den Abschluss bildet ein alternativer Lokalisierungsansatz zur 3D-Kartierung mittels kooperativ agierenden Robotern.

The fields of biologically inspired artificial intelligence, neuroscience, and psychology have had exciting influences on each other over the past decades. Especially recently, with the increased popularity and success of artificial neural networks (ANNs), ANNs have enjoyed frequent use as models for brain function. However, there are still many disparities between the implementation, algorithms, and learning environment used for deep learning and those employed by the brain, which is reflected in their differing abilities. I first briefly introduce ANNs and survey the differences and similarities between them and the brain. I then make a case for designing the learning environment of ANNs to be more similar to that in which brains learn, namely by allowing them to actively interact with the world and decreasing the amount of external supervision. To implement this sensorimotor learning in an artificial agent, I use deep reinforcement learning, which I will also briefly introduce and compare to learning in the brain. In the research presented in this dissertation, I focus on testing the hypothesis that the learning environment matters and that learning in an embodied way leads to acquiring different representations of the world. We first tested this on human subjects, comparing spatial knowledge acquisition in virtual reality to learning from an interactive map. The corresponding two publications are complemented by a methods paper describing eye tracking in virtual reality as a helpful tool in this type of research. After demonstrating that subjects do indeed learn different spatial knowledge in the two conditions, we test whether this transfers to artificial agents. Two further publications show that an ANN learning through interaction learns significantly different representations of the sensory input than ANNs that learn without interaction. We also demonstrate that through end-to-end sensorimotor learning, an ANN can learn visually-guided motor control and navigation behavior in a complex 3D maze environment without any external supervision using curiosity as an intrinsic reward signal. The learned representations are sparse, encode meaningful, action-oriented information about the environment, and can perform few-shot object recognition despite not knowing any labeled data beforehand. Overall, I make a case for increasing the realism of the computational tasks ANNs need to solve (largely self-supervised, sensorimotor learning) to improve some of their shortcomings and make them better models of the brain.

Damit mobile Roboter vielfältige komplexe Aufgaben autonom erfüllen können, benötigen sie Planung, um so entsprechend der Gegebenheiten ihrer Umgebung zu handeln. Durch die stetig zunehmenden Fähigkeiten der Roboterhardware gewinnt die Handlungsplanung und deren Integration in das Gesamtsystem zunehmend an Bedeutung. Die vorliegende Arbeit versucht, einen weiteren Schritt Richtung planbasierter Robotersteuerung zu gehen. Dabei wird zunächst die Verwendung des HTN-Planers SHOP2 in einem Robotersystem, das sich das Lernen aus Erfahrungen zum Ziel gesetzt hat, beschrieben und Wege aufgezeigt, wie die Robustheit des Systems durch die Integration mit anderen Komponenten erhöht werden kann. Mobilen Robotern stehen unterschiedliche Formen von Wissen, wie temporales oder räumliches Wissen oder Informationen über Ressourcen zur Verfügung. Diese können von SHOP2 jedoch nicht genutzt werden. Um diese Anforderung zu erfüllen, wird in dieser Arbeit der hybride hierarchische Planer CHIMP präsentiert, der die Vorteile hierarchischer Planung und der hybriden Planung als Meta-CSP, das die Integration verschiedener Wissensformen erlaubt, kombiniert. Des Weiteren können seine Pläne parallel ausführbare Aktionen enthalten, und zusätzliche Aufgaben können während der Ausführung in den bestehenden Plan integriert werden.

Machine learning plays an increasingly important role in modern systems. The ability to learn from data enhances or enables many applications. Recently, quick in-stream processing of possibly a huge or even infinite amount of data gains more attention. This thesis deals with such on-line learning systems for regression that learn with every example incrementally and are reliable even in presence of uncertainties. A new learning approach, called IRMA, is introduced which directly incorporates knowledge about the model structure into its parameter update. This way it is aggressive to incorporate a new example locally as much as possible and at the same time passive in the sense that the global output is changed as little as possible. It can be applied to any model structure that is linear in its parameters and is proven to minimize the worst case prediction error in each step. Hence, IRMA is reliable in every situation and the investigations show that in every case a bad performance is prevented by inherently averting overfitting even for complex model structures and in high dimensions. An extension of such on-line learning systems monitors the learning process, regarding conflict and ignorance, and estimates the trustworthiness of the learned hypothesis by the means of trust management. This provides insight into the learning system at every step and the designer can adjust its setup if necessary. Additionally, the trust estimation allows to assign a trustworthiness to each individual prediction the learning system makes. This way the overall system can react to uncertain predictions at a higher level and increase its safety, e.g. by reverting to a fallback. Furthermore, the uncertainties are explicitly incorporated into the learning process. The uncertainty of the hypothesis is reflected by allowing less change for more certain regions of the learned system. This way, good learned knowledge is protected and a higher robustness to disturbances is achieved. The uncertainty of each example used for learning is reflected by adapting less to uncertain examples. Thereby, the learning system gets more robust to training examples that are known to be uncertain. All approaches are formally analyzed and their characteristic properties are demonstrated in empirical investigations. In addition, a real world application to forecasting electricity loads shows the benefits of the approaches.

Die Arbeit beschäftigt sich mit Konvergenz- und Stabilitätseigenschaften von Verfahren des Reinforcement Lernens mit Funktionsapproximation. Besonderer Schwerpunkt wird dabei auf die Analyse des TD[0] Lernens gelegt, welches als unendliches Produkt von Matrizen aufgefasst wird. Damit kann man eine Klasse von Approximatoren festlegen, welche für das TD[0] Lernen geeignet ist. Im Allgemeinen ist eine solche Analyse aber schwer durchzuführen (Unentscheidbarkeit der Beschränktheit von unendlichen Matrixprodukten). Um eine breitere Klasse von Approximatoren untersuchen zu können, wird das so genannte synchrone TD[0] Lernen vollständig analysiert (inklusive Aussagen über Konvergenzgeschwindigkeit). Es wird aufgezeigt, dass die Divergenz des synchronen TD[0] Lernens die Divergenz des normalen (asynchronen) TD[0] Lernens impliziert. Es werden verschiedene Klassen von Approximatoren sowie andere Bedingungen für die Stabilität des synchronen TD[0] Lernens untersucht. Eine Anwendung der erzielten Resultate auf gitterbasierte Approximatoren schliesst die Arbeit ab.

There are two disciplines that are concerened with the same object of study, narratives, but that rarely exchange insights and ideas, let alone engage in collaborative research. The first is Narrative Theory (NT), an analytical discipline from the humanities that attempts to analyze literary texts and from these instances derive a general understanding of the concept of narrative. The second is Compuatational Story Composition (CSC), a discipline in the domain of Artificial Intelligence that attempts to enable computers to autonomously compose fictional narratives in a way that could be deemed creative. Several reasons can be found for the lack of collaboration, but one of them stands out: The two disciplines follow decidedly different research methodologies at contradistinct levels of abstraction. This makes it hard to conduct NT and CSC research simultaneously, and also means that CSC researchers have a hard time validating whether they use NT concepts correctly, while NT scholars have no use for the outputs created by work in CSC. At the same time, a close exchance between the two disciplines would be desirebale, not only because of the complementary approach to their object of study, but also because comparable interdisciplinary collaborations have proven to be productive in other fields, like for instance linguistics. The present thesis proposes a research methodology called generative modeling designed to address the methodological differences outlined above, and thus allow to conduct simultaneous NT and CSC research. As a proof of concept it performs several cycles of generative modeling, in which it computationally implements concepts and dynamics described in two frameworks from NT, namely Marie-Laure Ryan's possible worlds approach to plot, and Alan Palmer's fictional minds approach to characters. In detail, the first cycle attempts to implement Ryan's possible worlds semantics and the resulting dynamics of plot, but falls short in a way that suggests that the first principles layed out in the theory are not sufficient to capture an example plot, for a number of reasons. The second cycle resolves these hypothesized problems by extending Ryan's plot understanding with affective dynamics based on Palmer's understanding of fictional minds. With plot dynamics completed, the third cycle implements Ryan's concept of tellability, which represents a quantifiable measure of the structural quality of plots. The last cycle implements a Genetic Algorithm based search heuristic that is capable of searching the plot space spanned by the employed formalism for plots high in tellability, which provides additional insights on properties of tellability. The resulting implementation is a in-depth computational representation of plot ingrained into the CSC System InBloom, which is capable of autonomusly composing novel plots and evaluating their quality. The study reported in this thesis demonstrates, how implementing narratological theories as generative models can lead to insights for NT, and how grounding computational representations of narrative in NT can help CSC systems take over creative responsibilities. Thereby, it shows the feasibility and utility of generative modeling.

In the realm of ever increasing data volume and traffic the processing of data as a stream is key in order to build flexible and scalable data processing engines. On-line machine learning provides powerful algorithms for extracting predictive models from such data streams even if the modeled relation is time-variant in nature. The modeling of real valued data in on-line regression tasks is especially important as it connects to modeling and system identification tasks in engineering domains and bridges to other fields of machine learning like classification and reinforcement learning. Therefore, this thesis considers the problem of on-line regression on time variant data streams and introduces a new multi resolution perspective for tackling it. The proposed incremental learning system, called AS-MRA, comprises a new interpolation scheme for symmetric simplicial input segmentations, a layered approximation structure of sequential local refinement layers and a learning architecture for efficiently training the layer structure. A key concept for making these components work together in harmony is a differential parameter encoding between subsequent refinement layers which allows to decompose the target function into independent additional components represented as individual refinement layers. The whole AS-MRA approach is designed to form a smooth approximation while having its computational demands scaling linearly towards the input dimension and the overall expressiveness and therefore potential storage demands scaling exponentially towards input dimension. The AS-MRA provides no mandatory design parameters, but offers opportunities for the user to state tolerance parameters for the expected prediction performance which automatically and adaptively shape the resulting layer structure during the learning process. Other optional design parameters allow to restrict the resource consumption with respect to computational and memory demands. The effect of these parameters and the learning behavior of the AS-MRA as such are investigated with respect to various learning issues and compared to different related on-line learning approaches. The merits and contributions of the AS-MRA are experimentally shown and linked to general considerations about the relation between key concepts of the AS-MRA and fundamental results in machine learning.

In der Arbeit wird ein neuartiges Lernverfahren für Gauß-Mischverteilungen entwickelt. Es basiert auf der Technik der Markov-Chain Monte-Carlo Verfahren und ist in der Lage, in einem Zuge die Größe der Mischverteilung sowie deren Parameter zu bestimmen. Das Verfahren zeichnet sich sowohl durch eine gute Anpassung an die Trainingsdaten als auch durch eine gute Generalisierungsleistung aus. Ausgehend von einer Beschreibung der stochastischen Grundlagen und einer Analyse der Probleme, die beim Lernen von Gauß-Mischverteilungen auftreten, wird in der Abeit das neue Lernverfahren schrittweise entwickelt und seine Eigenschaften untersucht. Ein experimenteller Vergleich mit bekannten Lernverfahren für Gauß-Mischverteilungen weist die Eignung des neuen Verfahrens auch empirisch nach.

Managing agricultural processes with significant logistics sub-processes is a challenge because coordinating a distributed fleet in a dynamic environment is difficult without proper oversight in terms of qualitative and quantitative process information. Digital assistance systems are thought to aid agricultural practitioners by providing process-related information and thus support operational decision-making or even control the logistic flow (semi-)automatically. However, their development is currently stifled by a lack of monitoring capabilities during process execution. This thesis concerns the topic of online process monitoring for ongoing agricultural logistic processes. It discusses how to extract process knowledge from the telemetry of agricultural machines by applying spatio-semantic reasoning techniques. Our method combines spatial analysis for identifying spatial relationships between machines and their environment with semantic inference to derive formal process knowledge through ontological and rule-based reasoning. To test our concepts, we implemented a domain-agnostic semantic mapping framework and applied it in the context of forage maize harvesting. We present custom-made ontological models and rules to represent agricultural environments and to reason about machine actors and their process states. Based on our prototype, we demonstrate how to implement automated process and service tracking in near-real-time. Finally, we discuss the role of online process analytics systems in the context of other agricultural assistance systems for farm and fleet management.

This (cumulative) dissertation is concerned with mechanisms and models of information processing and transmission by individual neurons and small neural assemblies. In this document, I first provide historical context for these ideas and highlight similarities and differences to related concepts from machine learning and neuromorphic engineering. With this background, I then discuss the four main themes of my work, namely dendritic filtering and delays, homeostatic plasticity and adaptation, rate-coding with spiking neurons, and spike-timing based alternatives to rate-coding. The content of this discussion is in large part derived from several of my own publications included in Appendix C, but it has been extended and revised to provide a more accessible and broad explanation of the main ideas, as well as to show their inherent connections. I conclude that fundamental differences remain between our understanding of information processing and transmission in machine learning on the one hand and theoretical neuroscience on the other, which should provide a strong incentive for further interdisciplinary work on the domain boundaries between neuroscience, machine learning and neuromorphic engineering.