Sequences of neural activity representing paths in an environment are expressed in the rodent hippocampus at three distinct time scales, with different hypothesized roles in hippocampal function. As an animal moves through an environment and passes through a series of place fields, place cells activate and deactivate in sequence, at the time scale of the animal’s movement (i.e., the behavioral time scale). Moreover, at each moment in time, as the animal’s location in the environment overlaps with the firing fields of many place cells, the active place cells fire in sequence during each cycle of the 4-12 Hz theta oscillation observed in the hippocampal local field potentials (i.e., the theta time scale), such that the neural activity, in general, represents a short path that begins slightly behind the animal and ends slightly ahead of the animal. These sequences have been hypothesized to play a role in the encoding and recall of episodes of behavior. Sequences of neural activity occurring at the third time scale are observed during both sleep and awake but restful states, when animals are paused and generally inattentive, and are associated with sharp wave ripple complexes (SWRs) observed in the hippocampal local field potentials. During the awake state, these sequences have been shown to begin near the animal’s location and extend forward (forward replay) or backward (backward replay), and have been hypothesized to play a role in memory consolidation, path planning, and reinforcement learning. This thesis uses a novel sequence detection method and a novel behavioral spatial decision task to study the functional significance of theta sequences and SWR sequences. The premise of the thesis is that by investigating the behavioral content represented by these sequences, we may further our understanding of how these sequences contribute to hippocampal function. The first part of the thesis presents an analysis of SWR sequences or replays, revealing several novel properties of these sequences. In particular it was found that instead of preferentially representing the more recently experienced parts of the maze, as might be expected for memory consolidation, paths that were not recently experienced were more likely to be replayed. Additionally, paths that were never experienced, including shortcut paths, were observed. These observations suggest that hippocampal replay may play a role in constructing and maintaining a "cognitive map" of the environment. The second part of the thesis investigates the properties of theta sequences. A recent study found that theta sequences extend further forward at choice points on a maze and suggested that these sequences may be partly under cognitive control. In this part of the thesis I present an analysis of theta sequences showing that there is diversity in theta sequences, with some sequences extending more forward and others beginning further backward. Furthermore, certain components of the environment are preferentially represented by theta sequences, suggesting that theta sequences may reflect the cognitive "chunking" of the animal’s environment. The third part of the thesis describes a computational model of the hippocampus which explores how synaptic learning due to neural activity during navigation (i.e., theta sequences) may enable the hippocampal network to produce forward, backward, and shortcut sequences during awake rest states (i.e., SWR sequences).

In this thesis, we study the topic of Lifelong Robotic Object Perception. We propose, as a long-term goal, a framework to recognize known objects and to discover unknown objects in the environment as the robot operates, for as long as the robot operates. We build the foundations for Lifelong Robotic Object Perception by focusing our study on the two critical components of this framework: 1) how to recognize and register known objects for robotic manipulation, and 2) how to automatically discover novel objects in the environment so that we can recognize them in the future. Our work on Object Recognition and Pose Estimation addresses two main challenges in computer vision for robotics: robust performance in complex scenes, and low latency for real-time operation. We present MOPED, a framework for Multiple Object Pose Estimation and Detection that integrates single-image and multi-image object recognition and pose estimation in one optimized, robust, and scalable framework. We extend MOPED to leverage RGBD images using an adaptive image-depth fusion model based on maximum likelihood estimates. We incorporate this model to each stage of MOPED to achieve object recognition robust to imperfect depth data. In Robotic Object Discovery, we address the challenges of scalability and robustness for long-term operation. As a first step towards Lifelong Robotic Object Perception, we aim to automatically process the raw video stream of an entire workday of a robotic agent to discover novel objects. The key to achieve this goal is to incorporate non-visual information| robotic metadata|in the discovery process. We encode the natural constraints and nonvisual sensory information in service robotics to make long-term object discovery feasible. We introduce an optimized implementation, HerbDisc, that processes a video stream of 6 h 20 min of challenging human environments in under 19 min and discovers 206 novel objects. We tailor our solutions to the sensing capabilities and requirements in service robotics, with the goal of enabling our service robot, HERB, to operate autonomously in human environments.

Surgeons increasingly need to perform complex operations on extremely small anatomy. Many existing and promising new surgeries are effective, but difficult or impossible to perform because humans lack the extraordinary control required at sub-millimeter scales. Using micromanipulators, surgeons gain higher positioning accuracy and additional dexterity as the instrument removes tremor and scales hand motions. While these aids are advantageous, they do not actively consider the goals or intentions of the operator and thus cannot provide context-specific behaviors, such as motion scaling around anatomical targets, prevention of unwanted contact with pre-defined tissue areas, compensation for moving anatomy, and other helpful task-dependent actions. This thesis explores the fusion of visual information with micromanipulator control and enforces task-specific behaviors that respond in synergy with the surgeon’s intentions and motions throughout surgical procedures. By exploiting real-time microscope view observations, a-priori knowledge of surgical operations, and pre-operative data prepared before the surgery, we hypothesize that micromanipulators can employ individualized and targeted aids to further help the surgeon. Specifically, we propose a vision-based control framework of virtual fixtures for handheld micromanipulator robots that naturally incorporates tremor suppression and motion scaling. We develop real-time vision systems to track the surgeon and anatomy and design fast, new algorithms for analysis of the retina. Virtual fixtures constructed from visually tracked anatomy allows for complex task-specific behaviors that monitor the surgeon’s actions and react appropriately to cooperatively accomplish the procedure. Particular focus is given to vitreoretinal surgery as a good choice for vision-based control because several new and promising surgical techniques in the eye depend on fine manipulations of tiny and delicate retinal structures. Experiments with Micron, the fully handheld micromanipulator developed in our lab, show that vision-based virtual fixtures significantly increase pointing precision by reducing positioning error during synthetic, but medically relevant hold-still and tracing tasks. To evaluate the proposed framework in realistic environments, we consider three demanding retinal procedures: membrane peeling, laser photocoagulation, and vessel cannulation. Preclinical trials on artificial phantoms, ex vivo, and in vivo animal models demonstrate that vision-based control of a micromanipulator significantly improves surgeon performance (p < 0.05).

Enabling computers to understand human and animal behavior has the potential to revolutionize many areas that benefit society such as clinical diagnosis, human-computer interaction, and social robotics. Critical to the understanding of human and animal behavior, and any temporally-varying phenomenon in general, is the capability to segment, classify, and cluster time series data. This thesis proposes segment-based Support Vector Machines (Seg-SVMs), a framework for supervised, weakly-supervised, and unsupervised time series analysis. Seg-SVMs outperform state-of-the-art approaches by combining three powerful ideas: energy-based structure prediction, bag-of-words representation, and maximum-margin learning. Energy-based structure prediction provides a principled mechanism for concurrent top-down recognition and bottom-up temporal localization. Bag-of-words representation provides segment-based features that tolerate misalignment errors and are computationally efficient. Maximum-margin learning, such as SVM and Structure Output SVM, has a convex learning formulation; it produces classifiers that are discriminative and less prone to over-fitting. In this thesis, we show how Seg-SVMs outperform state-of-the-art approaches for segmenting, classifying, and clustering human and animal behavior in video and accelerometer data of varying complexity. We illustrate these benefits in the problems of facial event detection, sequence labeling of human actions, and temporal clustering of animal behavior. In addition, the Seg-SVMs framework naturally provides solutions to two novel problems: early detection of human actions and weakly-supervised discovery of discriminative events.

This research focuses on tracking generic non-rigid objects at close range to an infrared triangulation-based RGB-D sensor. The work was motivated by direct industry demand for a foundation for a low-cost application to operate in a surveillance setting. There are several novel components of this research that build on classical and state-of-the-art literature to extend into this real-world environment with limited constraints. The initialization is automatic with no a priori knowledge of the object and there are no restrictions on object appearance or transformation. There are no assumptions on object placement and only a very general physical model is applied to object trajectory. The tracking is performed using a Kalman filter and polynomial predictor to hypothesize the next location and a particle filter with colour, edge, depth edge, and absolute depth features to pinpoint object location. This work deals with challenges that are not explored in other work including highly variable object motion characteristics and generality with respect to the object tracked. It also explores the potential for multiple objects to occupy the same x-y location and have the same appearance. The result is a basic model for generic single object tracking that can be extended to any scenario with tailored occlusion-handling and augmented with behavioural analysis to confront a real-world problem. / Cette recherche implique le suivi des objets génériques non-rigides qui passent à courte distance à un capteur RVB-D utilisant une caméra infrarouge avec triangulation. Le travail a été motivé par un besoin de partenaires de l'industrie pour une application à faible coût pour fonctionner dans un cadre de surveillance. Il y a plusieurs éléments nouveaux de ce recherche qui utilisent la technologie classique et nouvelle pour rendre l'application faisable dans le monde réel avec peu de contraintes. L'initialisation est automatique sans connaissance à l'avance de l'objet et il n'y a aucune restriction sur l'apparence ou transformation de l'objet. Il n'y a pas d'hypothèses sur le placement de l'objet et seulement une modèle physique très général est appliqué à la trajectoire de l'objet. Le suivi est effectué en utilisant un filtre Kalman avec une fonctionne polynomial pour prédire l'emplacement de l'objet dans le prochain cadre. Un filtre à particules utilise ce prédiction pour placé ces particules et focalisé sur l'objet utilisant l'information de la couleur, les bords d'intensité, les bords de profondeur, et la profondeur absolue. Cette recherche traite des défis qui ne sont pas explorées dans d'autres travaux, notamment le mouvement d'objets très variable et la généralité par rapport à l'objet suivi. Il explore aussi la possibilité de plusieurs objets qui occupent le même emplacement x-y ayant la même apparence. Le résultat est un modèle pour le suivi d'un objet unique générique qui peut être étendue à n'importe quel scénario avec l'ajout d'un processus pour les occlusions et l'analyse comportementale.

The bottleneck concept in reinforcement learning has played a prominent role in automatically finding temporal abstractions from experience. Lacking significant theory, it has however been regarded by some as being merely a trick. This thesis attempts to gain better intuition about this approach using spectral graph theory. A connection to the theory of Nearly Completely Decomposable Markov Chains (NCD) is also drawn and shows great promise. An options discovery algorithm is proposed and is the first of its kind to be applicable in continuous state spaces. As opposed to other similar approaches, this one can have running time O(n^2 log n) rather than O(n^3) making it suitable to much larger domains than the typical grid worlds. / L'identification automatique de goulots d'étranglement dans la structure de solution a joué un rôle important en apprentissage par renforcement hiérarchique au cours des dernières années. Bien que populaire, cette approche manque toujours de fondements théoriques adaptés. Ce mémoire tente de pallier ces lacunes en établissant des liens en théorie spectrale des graphes, espérant ainsi obtenir une meilleure compréhension des conditions garantissant son applicabilité. Une revue des efforts réalisés concernant les chaines de Markov presque complètement décomposable (NCD) permet de croire qu'elles pourraient être utiles au problème ici considéré. Un algorithme de découverte d'options motivé par la théorie spectrale des graphes est proposé et semble être le premier du genre à pouvoir être aussi appliqué dans un espace d'états continu. Contraire- ment à d'autres approches similaires, la complexité algorithmique en temps peut être de l'ordre de O(n^2 log n) plutôt que O(n^3), rendant possible la résolution de problèmes de plus grande envergure.

<p> This thesis addresses the problem of performing probabilistic inference in stochastic systems where the probability mass is far from uniformly distributed among all possible outcomes. Such <i>near-deterministic</i> systems arise in several real-world applications. For example, in human physiology, the widely varying evolution rates of physiological variables make certain trajectories much more likely than others; in natural language, a very small fraction of all possible word sequences accounts for a disproportionately high amount of probability under a language model. In such settings, it is often possible to obtain significant computational savings by focusing on the outcomes where the probability mass is concentrated. This contrasts with existing algorithms in probabilistic inference---such as junction tree, sum product, and belief propagation algorithms---which are well-tuned to exploit conditional independence relations. </p><p> The first topic addressed in this thesis is the structure of discrete-time temporal graphical models of near-deterministic stochastic processes. We show how the structure depends on the ratios between the size of the time step and the effective rates of change of the variables. We also prove that accurate approximations can often be obtained by sparse structures even for very large time steps. Besides providing an intuitive reason for causal sparsity in discrete temporal models, the sparsity also speeds up inference. </p><p> The next contribution is an eigenvalue algorithm for a linear factored system (e.g., dynamic Bayesian network), where existing algorithms do not scale since the size of the system is exponential in the number of variables. Using a combination of graphical model inference algorithms and numerical methods for spectral analysis, we propose an approximate spectral algorithm which operates in the factored representation and is exponentially faster than previous algorithms. </p><p> The third contribution is a temporally abstracted Viterbi (TAV) algorithm. Starting with a spatio-temporally abstracted coarse representation of the original problem, the TAV algorithm iteratively refines the search space for the Viterbi path via spatial and temporal refinements. The algorithm is guaranteed to converge to the optimal solution with the use of admissible heuristic costs in the abstract levels and is much faster than the Viterbi algorithm for near-deterministic systems. </p><p> The fourth contribution is a hierarchical image/video segmentation algorithm, that shares some of the ideas used in the TAV algorithm. A supervoxel tree provides the abstraction hierarchy for this application. The algorithm starts working with the coarsest level supervoxels, and refines portions of the tree which are likely to have multiple labels. Several existing segmentation algorithms can be used to solve the energy minimization problem in each iteration, and admissible heuristic costs once again guarantee optimality. Since large contiguous patches exist in images and videos, this approach is more computationally efficient than solving the problem at the finest level of supervoxels. </p><p> The final contribution is a family of Markov Chain Monte Carlo (MCMC) algorithms for near-deterministic systems when there exists an efficient algorithm to sample solutions for the corresponding deterministic problem. In such a case, a generic MCMC algorithm's performance worsens as the problem becomes more deterministic despite the existence of the efficient algorithm in the deterministic limit. MCMC algorithms designed using our methodology can bridge this gap. </p><p> The computational speedups we obtain through the various new algorithms presented in this thesis show that it is indeed possible to exploit near-determinism in probabilistic systems. Near-determinism, much like conditional independence, is a potential (and promising) source of computational savings for both exact and approximate inference. It is a direction that warrants more understanding and better generalized algorithms.</p>

It is estimated that by 2020, more than 50 billion devices will be interconnected, to form what is called the Internet of Things. Those devices range from consumer electronics to utility meters, including vehicles. Provided with sensory capabilities, those objects will be able to transmit valuable information about their environment, not only to humans, but even more importantly to other machines, which should ultimately be able to interpret and take decisions based on the information received. This “smartness” implies gifting those devices with a certain degree of automation. This Master’s Thesis investigates how recent advances in artificial intelligence planning can be helpful in building such systems. In particular, an artificial intelligence planner able to generate workflows for most of IoT-related use cases has been connected to an IoT platform. A performance study of a state-of-the planner, Fast Downward, on one of the most challenging IoT application, Smart Garbage Collection (which is similar to the Traveling Salesman Problem) has also been carried out. Eventually, different pre-processing and clustering techniques are suggested to tackle the latest AI planners’ inefficiency on quickly finding plans for the most difficult tasks.

No description available.

The ink and stylus tablets discovered at the Roman Fort of Vindolanda have provided a unique resource for scholars of ancient history. However, the stylus tablets in particular have proved extremely difficult to read. The aim of this thesis is to explore the extent to which techniques from Artificial Intelligence can be used to develop a system that could aid historians in reading the stylus texts. This system would utilise image processing techniques that have been developed in Engineering Science to analyse the stylus tablets, whilst incorporating knowledge elicited from experts working on the texts, to propagate possible suggestions of the text contained within the tablets. This thesis reports on what appears to be the first system developed to aid experts in the process of reading an ancient document. There has been little previous research carried out to see how papyrologists actually carry out their task. This thesis studies closely how experts working with primary sources, such as the Vindolanda Texts, operate. Using Knowledge Elicitation Techniques, a model is proposed for how they read a text. Information regarding the letter forms and language used at Vindolanda is collated, A corpus of annotated images is built up, to provide a data set regarding the letter forms used in the ink and stylus texts. In order to relate this information to the work done on image processing, a stochastic Minimum Description Length (MDL) architecture is adopted, and adapted, to form the basis of a system that can propagate interpretations of the Vindolanda texts. In doing so a system is constructed that can read in image data and output textual interpretations of the writing that appears on the documents. It is demonstrated that knowledge elicitation techniques can be used to capture and mobilise expert information. The process of reading ancient, and ambiguous texts, is made explicit. It is also shown that MDL can be used as a basis to build large systems that reason about complex information effectively. This research presents the first stages towards developing a cognitive visual system that can propagate realistic interpretations from image data, and so aid the papyrologists in their task.