This thesis is about a design process for creating educationally relevant tools. I submit that the key to creating tools that are educationally relevant is to focus on ensuring a high degree of alignment between the designed tool and the broader educational context into which the tool will be integrated. The thesis presents methods and processes for creating a tool that is both well aligned and relevant. The design domain of the thesis is described by a set of tools I refer to as “Configurable Embodied Interfaces”. Configurable embodied interfaces have a number of key features, they: Can sense their local surroundings through the detection of such environmental and physical parameters as light, sound, imagery, device acceleration, etc. Act on their local environment by outputting sound, light, imagery, motion of the device, etc. Are configurable in such a way as to link these inputs and outputs in a nearly unlimited number of ways. Contain active ways for users to either directly create new programs linking input and output, or to easily re-configure them by running different programs on them. Are user focused; they assume that a human being is manipulating them in some way, through affecting input and observing output of the interface. Spurred by the growth of cheap computation and sensing, a large number of educational programs have been built around use of configurable embodied interfaces in the last three decades. These programs exist in both formal and informal educational settings and are in use from early childhood through adult and community education. Typically, configurable embodied interfaces are used as tools in three major and sometimes overlapping areas: computer Science education, creative and engineering design education, and traditional science and math education. This work details three examples of collaborations between technologists and educators that led to the creation of educationally successful tools; these three examples share a focus on creating a configurable embodied interface to tackle a specific cognitive and affective set of learning goals, but differ completely in the location of the learning environment, the age and interests of the learners, and the nature of the learning goals. Through the exploration of the methods used, an analysis of the general and context-specific features of the design processes of the three accounts, and a comparison of the process used in this thesis to a conventional engineering design process, this work provides case studies and a set of guidelines that can inform technologists interested in designing educationally relevant embodied interfaces

Mobile robots that encounter people on a regular basis must react to them in some way. While traditional robot control algorithms treat all unexpected sensor readings as objects to be avoided, we argue that robots that operate around people should react socially to those people, following the same social conventions that people use around each other. This thesis presents our COMPANION framework: a Constraint-Optimizing Method for Person–Acceptable NavigatION. COMPANION is a generalized framework for representing social conventions as components of a constraint optimization problem, which is used for path planning and navigation. Social conventions, such as personal space and tending to the right, are described as mathematical cost functions that can be used by an optimal path planner. These social conventions are combined with more traditional constraints, such as minimizing distance, in a flexible way, so that additional constraints can be added easily. We present a set of constraints that specify the social task of traveling around people. We explore the implementation of this task first in simulation, where we demonstrate a robot’s behavior in a wide variety of scenarios. We also detail how a robot’s behavior can be changed by using different relative weights between the constraints or by using constraints representing different sociocultural conventions. We then explore the specific case of passing a person in a hallway, using the robot Grace. Through a user study, we show that people interpret the robot’s behavior according to human social norms, and also that people ascribe different personalities to the robot depending on its level of social behavior. In addition, we present an extension of the COMPANION framework that is able to represent joint tasks between the robot and a person. We identify the constraints necessary to represent the task of having a robot escort a person while traveling side-by-side. In simulation, we show the capability of this representation to produce behaviors such as speeding up or slowing down to travel together around corners, as well as complex maneuvers to travel through narrow chokepoints and return to a side-by-side formation. Finally, we present a newly designed robot, Companion, that is intended as a platform for general social human–robot research. Companion is a holonomic robot, able to move sideways without turning first, which we believe is an important social capability. We detail the design and capabilities of this new platform. As a whole, this thesis demonstrates both a need for, and an implementation and evaluation of, robots that navigate around people according to social norms.

Currently available wind-wave prediction models require a prohibitive amount of computing time for simulating non-linear wave-wave interactions. Moreover, some parts of wind-wave generation processes are not fully understood yet. For this reason accurate predictions are not always guaranteed. In contrast, Artificial Neural Network (ANN) techniques are designed to recognize the patterns between input and output so that they can save considerable computing time so that real-time wind-wave forecast can be available to the navy and commercial ships. For this reason, this study tries to use ANN techniques to predict waves for winter storms and hurricanes with much less computing time at the five National Oceanic and Atmospheric Administration (NOAA) wave stations along the East Coast of the U.S. from Florida to Maine (station 44007, 44013, 44025, 44009, and 41009). In order to identify prediction error sources of an ANN model, the 100% known wind-wave events simulated from the SMB model were used. The ANN predicted even untrained wind-wave events accurately, and this implied that it could be used for winter-storm and hurricane wave predictions. For the prediction of winter-storm waves, 1999 and 2001 winter-storm events with 403 data points had 1998 winter-storm events with 78 points were prepared for training and validation data sets, respectively. In general, because winter-storms are relatively evenly distributed over a large area and move slowly, wind information (u and v wind components) over a large domain was considered as ANN inputs. When using a 24-hour time-delay to simulate the time required for waves to be fully developed seas, the ANN predicted wave heights (r = 0.88) accurately, but the prediction accuracy of zero-crossing wave periods was much less (r = 0.61). For the prediction of hurricane waves, 15 hurricanes from 1995 to 2001 and Hurricane Bertha in 1998 were prepared for training and validation data sets, respectively. Because hurricanes affect a relatively small domain, move quickly, and change dramatically with time, the location of hurricane centers, the maximum wind speed, central pressure of hurricane centers, longitudinal and latitudinal distance between wave stations and hurricane centers were used as inputs. The ANN predicted wave height accurately when a 24-hour time-delay was used (r = 0.82), but the prediction accuracy of peak-wave periods was much less (r = 0.50). This is because the physical processes of wave periods are more complicated than those of wave heights. This study shows a possibility of an ANN technique as the winter-storm and hurricane-wave prediction model. If more winter-storm and hurricane data can be available, and the prediction of hurricane tracks is possible, we can forecast real-time wind-waves more accurately with less computing time.

Sequences of neural activity representing paths in an environment are expressed in the rodent hippocampus at three distinct time scales, with different hypothesized roles in hippocampal function. As an animal moves through an environment and passes through a series of place fields, place cells activate and deactivate in sequence, at the time scale of the animal’s movement (i.e., the behavioral time scale). Moreover, at each moment in time, as the animal’s location in the environment overlaps with the firing fields of many place cells, the active place cells fire in sequence during each cycle of the 4-12 Hz theta oscillation observed in the hippocampal local field potentials (i.e., the theta time scale), such that the neural activity, in general, represents a short path that begins slightly behind the animal and ends slightly ahead of the animal. These sequences have been hypothesized to play a role in the encoding and recall of episodes of behavior. Sequences of neural activity occurring at the third time scale are observed during both sleep and awake but restful states, when animals are paused and generally inattentive, and are associated with sharp wave ripple complexes (SWRs) observed in the hippocampal local field potentials. During the awake state, these sequences have been shown to begin near the animal’s location and extend forward (forward replay) or backward (backward replay), and have been hypothesized to play a role in memory consolidation, path planning, and reinforcement learning. This thesis uses a novel sequence detection method and a novel behavioral spatial decision task to study the functional significance of theta sequences and SWR sequences. The premise of the thesis is that by investigating the behavioral content represented by these sequences, we may further our understanding of how these sequences contribute to hippocampal function. The first part of the thesis presents an analysis of SWR sequences or replays, revealing several novel properties of these sequences. In particular it was found that instead of preferentially representing the more recently experienced parts of the maze, as might be expected for memory consolidation, paths that were not recently experienced were more likely to be replayed. Additionally, paths that were never experienced, including shortcut paths, were observed. These observations suggest that hippocampal replay may play a role in constructing and maintaining a "cognitive map" of the environment. The second part of the thesis investigates the properties of theta sequences. A recent study found that theta sequences extend further forward at choice points on a maze and suggested that these sequences may be partly under cognitive control. In this part of the thesis I present an analysis of theta sequences showing that there is diversity in theta sequences, with some sequences extending more forward and others beginning further backward. Furthermore, certain components of the environment are preferentially represented by theta sequences, suggesting that theta sequences may reflect the cognitive "chunking" of the animal’s environment. The third part of the thesis describes a computational model of the hippocampus which explores how synaptic learning due to neural activity during navigation (i.e., theta sequences) may enable the hippocampal network to produce forward, backward, and shortcut sequences during awake rest states (i.e., SWR sequences).

In this thesis, we study the topic of Lifelong Robotic Object Perception. We propose, as a long-term goal, a framework to recognize known objects and to discover unknown objects in the environment as the robot operates, for as long as the robot operates. We build the foundations for Lifelong Robotic Object Perception by focusing our study on the two critical components of this framework: 1) how to recognize and register known objects for robotic manipulation, and 2) how to automatically discover novel objects in the environment so that we can recognize them in the future. Our work on Object Recognition and Pose Estimation addresses two main challenges in computer vision for robotics: robust performance in complex scenes, and low latency for real-time operation. We present MOPED, a framework for Multiple Object Pose Estimation and Detection that integrates single-image and multi-image object recognition and pose estimation in one optimized, robust, and scalable framework. We extend MOPED to leverage RGBD images using an adaptive image-depth fusion model based on maximum likelihood estimates. We incorporate this model to each stage of MOPED to achieve object recognition robust to imperfect depth data. In Robotic Object Discovery, we address the challenges of scalability and robustness for long-term operation. As a first step towards Lifelong Robotic Object Perception, we aim to automatically process the raw video stream of an entire workday of a robotic agent to discover novel objects. The key to achieve this goal is to incorporate non-visual information| robotic metadata|in the discovery process. We encode the natural constraints and nonvisual sensory information in service robotics to make long-term object discovery feasible. We introduce an optimized implementation, HerbDisc, that processes a video stream of 6 h 20 min of challenging human environments in under 19 min and discovers 206 novel objects. We tailor our solutions to the sensing capabilities and requirements in service robotics, with the goal of enabling our service robot, HERB, to operate autonomously in human environments.

Surgeons increasingly need to perform complex operations on extremely small anatomy. Many existing and promising new surgeries are effective, but difficult or impossible to perform because humans lack the extraordinary control required at sub-millimeter scales. Using micromanipulators, surgeons gain higher positioning accuracy and additional dexterity as the instrument removes tremor and scales hand motions. While these aids are advantageous, they do not actively consider the goals or intentions of the operator and thus cannot provide context-specific behaviors, such as motion scaling around anatomical targets, prevention of unwanted contact with pre-defined tissue areas, compensation for moving anatomy, and other helpful task-dependent actions. This thesis explores the fusion of visual information with micromanipulator control and enforces task-specific behaviors that respond in synergy with the surgeon’s intentions and motions throughout surgical procedures. By exploiting real-time microscope view observations, a-priori knowledge of surgical operations, and pre-operative data prepared before the surgery, we hypothesize that micromanipulators can employ individualized and targeted aids to further help the surgeon. Specifically, we propose a vision-based control framework of virtual fixtures for handheld micromanipulator robots that naturally incorporates tremor suppression and motion scaling. We develop real-time vision systems to track the surgeon and anatomy and design fast, new algorithms for analysis of the retina. Virtual fixtures constructed from visually tracked anatomy allows for complex task-specific behaviors that monitor the surgeon’s actions and react appropriately to cooperatively accomplish the procedure. Particular focus is given to vitreoretinal surgery as a good choice for vision-based control because several new and promising surgical techniques in the eye depend on fine manipulations of tiny and delicate retinal structures. Experiments with Micron, the fully handheld micromanipulator developed in our lab, show that vision-based virtual fixtures significantly increase pointing precision by reducing positioning error during synthetic, but medically relevant hold-still and tracing tasks. To evaluate the proposed framework in realistic environments, we consider three demanding retinal procedures: membrane peeling, laser photocoagulation, and vessel cannulation. Preclinical trials on artificial phantoms, ex vivo, and in vivo animal models demonstrate that vision-based control of a micromanipulator significantly improves surgeon performance (p < 0.05).

Enabling computers to understand human and animal behavior has the potential to revolutionize many areas that benefit society such as clinical diagnosis, human-computer interaction, and social robotics. Critical to the understanding of human and animal behavior, and any temporally-varying phenomenon in general, is the capability to segment, classify, and cluster time series data. This thesis proposes segment-based Support Vector Machines (Seg-SVMs), a framework for supervised, weakly-supervised, and unsupervised time series analysis. Seg-SVMs outperform state-of-the-art approaches by combining three powerful ideas: energy-based structure prediction, bag-of-words representation, and maximum-margin learning. Energy-based structure prediction provides a principled mechanism for concurrent top-down recognition and bottom-up temporal localization. Bag-of-words representation provides segment-based features that tolerate misalignment errors and are computationally efficient. Maximum-margin learning, such as SVM and Structure Output SVM, has a convex learning formulation; it produces classifiers that are discriminative and less prone to over-fitting. In this thesis, we show how Seg-SVMs outperform state-of-the-art approaches for segmenting, classifying, and clustering human and animal behavior in video and accelerometer data of varying complexity. We illustrate these benefits in the problems of facial event detection, sequence labeling of human actions, and temporal clustering of animal behavior. In addition, the Seg-SVMs framework naturally provides solutions to two novel problems: early detection of human actions and weakly-supervised discovery of discriminative events.

The job of classifying or annotating brain tumors from MRI images can be time-consuming and difficult, even for radiologists. To increase the survival chances of a patient, medical practitioners desire a means for quick and accurate diagnosis. While datasets like CIFAR, ImageNet, and SVHN have tens of thousands, hundreds of thousands, or millions of samples, an MRI dataset may not have the same luxury of receiving accurate labels for each image containing a tumor. This work covers three models that classify brain tumors using a combination of convolutional neural networks and of the concept of capsule layers. Each network utilizes a hit-or-miss capsule layer to relate classes to capsule vectors in a one-to-one relationship. Additionally, this work proposes the use of deep active learning for picking the samples that can give the best model, PSP-HitNet, the most information when adding mini-batches of unlabeled data into the master, labeled training dataset. By using an uncertainty estimated querying strategy, PSP-HitNet approaches the best validation accuracy possible within the first 12-24% of added data from the unlabeled dataset, whereas random choosing takes until 30-50% of the unlabeled to reach the same performance.

This thesis presents optical characterization of the static and dynamic magnetic interactions in ferromagnetic and multiferroic heterostructures with time-resolved and interface-specific optical techniques. The focus of the thesis is on elucidating the underlying physics of key physical parameters and novel approaches, crucial to the performance of magnetic recording and spintronic devices.;First, time-resolved magneto-optical Kerr effect (TRMOKE) is applied to investigate the spin dynamics in L10 ordered FePt thin films, where perpendicular magnetic anisotropy Ku and intrinsic Gilbert damping alpha0 are determined. Furthermore, the quadratic dependence of Ku and alpha0 on spin-orbit coupling strength xi is demonstrated, where xi is continuously controlled through chemical substitution of Pt with Pd element. In addition, a linear correlation between alpha0 and electron scattering rate 1/T e is experimentally observed through modulating the anti-site disorder c in the L10 ordered structure. The results elucidate the basic physics of magnetic anisotropy and Gilbert damping, and facilitate the design and fabrication of new magnetic alloys with large perpendicular magnetic anisotropy and tailored damping properties.;Second, ultrafast excitation of coherent spin precession is demonstrated in Fe/CoO heterostructures and La0.67Ca0.33MnO 3 thin films using TRMOKE technique. In the Fe/CoO thin films, Instant non-thermal ferromagnet (FM) -- antiferromagnet (AFM) exchange torque on Fe magnetization through ultrafast photo-excited charge transfer possesses in the CoO layer is experimentally demonstrated at room temperature. The efficiency of spin precession excitation is significantly higher and the recovery is notably faster than the demagnetization procedure. In the La0.67Ca 0.33MnO3 thin films, pronounced spin precessions are observed in a geometry with negligible canting of the magnetization, indicating that the transient exchange field is generated by the emergent AFM interactions due to charge transfer and modification of the kinetic energy of eg electrons under optical excitation. The results will help promoting the development of novel device concepts for ultrafast spin manipulation.;Last, the interfacial spin state of the multiferroic heterostructure PbZr0.52Ti0.4803/La0.67Sr0.33MnO 3 and its dependence on ferroelectric polarization is investigated with interface specific magnetization induced second harmonic generation (MSHG). The spin alignment of Mn ions in the first unit cell layer at the heterointerface can be tuned from FM to AFM exchange coupled, while the bulk magnetization remains unchanged as probed with MOKE. The discovery provides new insights into the basic physics of interfacial magneto-electric (ME) coupling.

We have designed and implemented a physics-based adaptive Bayesian pattern classification model that uses a passive thermal infrared imaging system to automatically characterize non-heat generating objects in unstructured outdoor environments for mobile robots. In the context of this research, non-heat generating objects are defined as objects that are not a source for their own emission of thermal energy, and so exclude people, animals, vehicles, etc. The resulting classification model complements an autonomous bot's situational awareness by providing the ability to classify smaller structures commonly found in the immediate operational environment. Since GPS depends on the availability of satellites and onboard terrain maps which are often unable to include enough detail for smaller structures found in an operational environment, bots will require the ability to make decisions such as "go through the hedges" or "go around the brick wall." A thermal infrared imaging modality mounted on a small mobile bot is a favorable choice for receiving enough detailed information to automatically interpret objects at close ranges while unobtrusively traveling alongside pedestrians. The classification of indoor objects and heat generating objects in thermal scenes is a solved problem. A missing and essential piece in the literature has been research involving the automatic characterization of non-heat generating objects in outdoor environments using a thermal infrared imaging modality for mobile bots. Seeking to classify non-heat generating objects in outdoor environments using a thermal infrared imaging system is a complex problem due to the variation of radiance emitted from the objects as a result of the diurnal cycle of solar energy. The model that we present will allow bots to "see beyond vision" to autonomously assess the physical nature of the surrounding structures for making decisions without the need for an interpretation by humans.;Our approach is an application of Bayesian statistical pattern classification where learning involves labeled classes of data (supervised classification), assumes no formal structure regarding the density of the data in the classes (nonparametric density estimation), and makes direct use of prior knowledge regarding an object class's existence in a bot's immediate area of operation when making decisions regarding class assignments for unknown objects. We have used a mobile bot to systematically capture thermal infrared imagery for two categories of non-heat generating objects (extended and compact) in several different geographic locations. The extended objects consist of objects that extend beyond the thermal camera's field of view, such as brick walls, hedges, picket fences, and wood walls. The compact objects consist of objects that are within the thermal camera's field of view, such as steel poles and trees. We used these large representative data sets to explore the behavior of thermal-physical features generated from the signals emitted by the classes of objects and design our Adaptive Bayesian Classification Model. We demonstrate that our novel classification model not only displays exceptional performance in characterizing non-heat generating outdoor objects in thermal scenes but it also outperforms the traditional KNN and Parzen classifiers.