yes / Applications of autonomous systems are becoming increasingly common across the field of engineered systems from cars, drones, manufacturing systems and medical devices, addressing prevailing societal changes, and, increasingly, consumer demand. Autonomous systems are expected to self-manage and self-certify against risks affecting the mission, safety and asset integrity. While significant progress has been achieved in relation to the modelling of safety and safety assurance of autonomous systems, no similar approach is available for resilience that integrates coherently across the cyber and physical parts. This paper presents a comprehensive discussion of resilience in the context of robotic autonomous systems, covering both resilience by design and resilience by reaction, and proposes a conceptual model of a system of learning for resilience assurance in a continuous product development framework. The resilience assurance model is proposed as a composable digital artefact, underpinned by a rigorous model-based resilience analysis at the system design stage, and dynamically monitored and continuously updated at run time in the system operation stage, with machine learning based knowledge extraction and validation.

ITC/USA 2006 Conference Proceedings / The Forty-Second Annual International Telemetering Conference and Technical Exhibition / October 23-26, 2006 / Town and Country Resort & Convention Center, San Diego, California / Recent emphasis has been placed on mobile robotics performing in unstructured environments. This realm of operations requires many different algorithms to interpret the various situations. This not only requires a system that is able to support, and facilitate, the fusion of the results, but it also needs to be tolerant of system errors. In modern operating systems, separate processes are able to fail without affecting other processes. Using this ability, along with fault tolerant inter-process communications, and supervisory process managers, allows the total system to continue to operate under adverse conditions. While this paper focuses primarily on the challenges faced by mobile robotics, the approach can be extended to a wide range of systems which must autonomously identify and adapt to failures/situations.

ITC/USA 2009 Conference Proceedings / The Forty-Fifth Annual International Telemetering Conference and Technical Exhibition / October 26-29, 2009 / Riviera Hotel & Convention Center, Las Vegas, Nevada / Emerging and next-generation test instrumentation increasingly relies on network communication to manage complex and dynamic test scenarios, particularly for uninhabited autonomous systems. Adapting wireless communication infrastructure to accommodate challenging testing needs can benefit from reconfigurable radio technology. Frequency agility is one characteristic of reconfigurable radios that to date has seen only limited progress toward programmability. This paper overviews an ongoing project to validate a promising chipset that performs conversion of RF signals directly into digital data for the wireless receiver and, for the transmitter, converts digital data into RF signals. The Software Configurable Multichannel Transceiver (SCMT) enables four transmitters and four receivers in a single unit, programmable for any frequency band between 1 MHz and 6 GHz.

Operation of Unmanned Air Vehicles (UAVs) has increased significantly over the past few years. However, routine operation in non-segregated airspace remains a challenge, primarily due to nature of the environment and restrictions and challenges that accompany this. Currently, tight human control is envisaged as a means to achieve the oft quoted requirements of transparency , equivalence and safety. However, the problems of high cost of human operation, potential communication losses and operator remoteness remain as obstacles. One means of overcoming these obstacles is to devolve authority, from the ground controller to an on-board system able to understand its situation and make appropriate decisions when authorised. Such an on-board system is known as an Autonomous System. The nature of the autonomous system, how it should be designed, when and how authority should be transferred and in what context can they be allowed to control the vehicle are the general motivation for this study. To do this, the system must overcome the negative aspects of differentiators that exist between UASs and manned aircraft and introduce methods to achieve required increases in the levels of versatility, cost, safety and performance. The general thesis of this work is that the role and responsibility of an airborne autonomous system are sufficiently different from those of other conventionally controlled manned and unmanned systems to require a different architectural approach. Such a different architecture will also have additional requirements placed upon it in order to demonstrate acceptable levels of Transparency, Equivalence and Safety. The architecture for the system is developed from an analysis of the basic requirements and adapted from a consideration of other, suitable candidates for effective control of the vehicle under devolved authority. The best practices for airborne systems in general are identified and amalgamated with established principles and approaches of robotics and intelligent agents. From this, a decision architecture, capable of interacting with external human agencies such as the UAS Commander and Air Traffic Controllers, is proposed in detail. This architecture has been implemented and a number of further lessons can be drawn from this. In order to understand in detail the system safety requirements, an analysis of manned and unmanned aircraft accidents is made. Particular interest is given to the type of control moding of current unmanned aircraft in order to make a comparison, and prediction, with accidents likely to be caused by autonomously controlled vehicles. The effect of pilot remoteness on the accident rate is studied and a new classification of this remoteness is identified as a major contributor to accidents A preliminary Bayesian model for unmanned aircraft accidents is developed and results and predictions are made as an output of this model. From the accident analysis and modelling, strategies to improve UAS safety are identified. Detailed implementations within these strategies are analysed and a proposal for more advanced Human-Machine Interaction made. In particular, detailed analysis is given on exemplar scenarios that a UAS may encounter. These are: Sense and Avoid , Mission Management Failure, Take Off/Landing, and Lost Link procedures and Communications Failure. These analyses identify the nature of autonomous, as opposed to automatic, operation and clearly show the benefits to safety of autonomous air vehicle operation, with an identifiable decision architecture, and its relationship with the human controller. From the strategies and detailed analysis of the exemplar scenarios, proposals are made for the improvement of unmanned vehicle safety The incorporation of these proposals into the suggested decision architecture are accompanied by analysis of the levels of benefit that may be expected. These suggest that a level approaching that of conventional manned aircraft is achievable using currently available technologies but with substantial architectural design methodologies than currently fielded.

This thesis deals with the challenge of autonomous navigation of the ExoMars rover. The absence of global positioning systems (GPS) in space, added to the limitations of wheel odometry makes autonomous navigation based on these two techniques - as done in the literature - an inviable solution and necessitates the use of other approaches. That, among other reasons, motivates this work to use solely visual data to solve the robot’s Egomotion problem. The homogeneity of Mars’ terrain makes the robustness of the low level image processing technique a critical requirement. In the first part of the thesis, novel solutions are presented to tackle this specific problem. Detection of robust features against illumination changes and unique matching and association of features is a sought after capability. A solution for robustness of features against illumination variation is proposed combining Harris corner detection together with moment image representation. Whereas the first provides a technique for efficient feature detection, the moment images add the necessary brightness invariance. Moreover, a bucketing strategy is used to guarantee that features are homogeneously distributed within the images. Then, the addition of local feature descriptors guarantees the unique identification of image cues. In the second part, reliable and precise motion estimation for the Mars’s robot is studied. A number of successful approaches are thoroughly analysed. Visual Simultaneous Localisation And Mapping (VSLAM) is investigated, proposing enhancements and integrating it with the robust feature methodology. Then, linear and nonlinear optimisation techniques are explored. Alternative photogrammetry reprojection concepts are tested. Lastly, data fusion techniques are proposed to deal with the integration of multiple stereo view data. Our robust visual scheme allows good feature repeatability. Because of this, dimensionality reduction of the feature data can be used without compromising the overall performance of the proposed solutions for motion estimation. Also, the developed Egomotion techniques have been extensively validated using both simulated and real data collected at ESA-ESTEC facilities. Multiple stereo view solutions for robot motion estimation are introduced, presenting interesting benefits. The obtained results prove the innovative methods presented here to be accurate and reliable approaches capable to solve the Egomotion problem in a Mars environment.

In this thesis, the development of a stereo camera system for an intelligent tool is presented. The task of the system is to identify and localise objects so that the tool can guide a robot. Different approaches to object detection have been implemented and evaluated and the systems ability to localise objects has been tested. The results show that the system can achieve a localisation accuracy below 5 mm.

Thermal spectrum cameras are gaining interest in many applications due to their long wavelength which allows them to operate under low light and harsh weather conditions. One disadvantage of thermal cameras is their limited visual interpretability for humans, which limits the scope of their applications. In this thesis, we try to address this problem by investigating the possibility of transforming thermal infrared (TIR) images to perceptually realistic visible spectrum (VIS) images by using Convolutional Neural Networks (CNNs). Existing state-of-the-art colorization CNNs fail to provide the desired output as they were trained to map grayscale VIS images to color VIS images. Instead, we utilize an auto-encoder architecture to perform cross-spectral transformation between TIR and VIS images. This architecture was shown to quantitatively perform very well on the problem while producing perceptually realistic images. We show that the quantitative differences are insignificant when training this architecture using different color spaces, while there exist clear qualitative differences depending on the choice of color space. Finally, we found that a CNN trained from day time examples generalizes well on tests from night time.

Training data is the bottleneck for training Convolutional Neural Networks. A larger dataset gives better accuracy though also needs longer training time. It is shown by finetuning neural networks on synthetic rendered images, that the mean average precision increases. This method was applied to two different datasets with five distinctive objects in each. The first dataset consisted of random objects with different geometric shapes. The second dataset contained objects used to assemble IKEA furniture. The neural network with the best performance, trained on 5400 images, achieved a mean average precision of 0.81 on a test which was a sample of a video sequence. Analysis of the impact of the factors dataset size, batch size, and numbers of epochs used in training and different network architectures were done. Using synthetic images to train CNN’s is a promising path to take for object detection where access to large amount of annotated image data is hard to come by.

Mobile robots have found numerous applications in recent years, in areas such as consumer robotics, environmental monitoring, security and transportation. For information dissemination, multi-robot cooperation or operator intervention, reliable communications are important. The combination of communication constraints with other requirements in robotics, such as navigation and obstacle avoidance is called communication-aware motion planning. To facilitate integration, communication-aware methods should fit into traditional layered architectures of motion planning. This thesis contains two main contributions, applicable to such an architecture. The first contribution is to develop strategies for exploiting multipath fading while following a reference trajectory. By deviating from the reference, a robot can stop and communicate at positions with high signal strength, trading tracking performance for link quality. We formulate this problem in three different ways: First we maximize the link quality, subject to deterministic bounds on the tracking error. We control the velocity based on the position and channel quality. Second, we consider probabilistic tracking error bounds and develop a cascaded control architecture that performs time-triggered stopping while regulating the tracking error. Third, we formulate a hybrid optimal control problem, switching between standing still to communicate and driving to improve tracking. The resulting channel quality is analyzed and we perform extensive experiments to validate the communication model and compare the proposed methods to the nominal case of driving at constant velocity. The results show good agreement with the model and improvements of over 100% in the throughput when the channel quality is low. The second contribution is to plan velocities for a group of N robots, moving along pre-determined paths through an obstacle field. Robots can only communicate if they have an unobstructed line of sight, and the problem is to maintain connectivity while traversing the paths. This is mapped to motion planning in an N-dimensional configuration space. We propose and investigate two solutions, using a rapidly exploring random tree (RRT) and an exact method inspired by cell decomposition. The RRT method scales better with the problem size than the exact method, which has a worst-case time complexity that is exponential in the number of obstacles. But the randomization in the RRT method makes it difficult to set a timeout for the solver, which runs forever if a problem instance is unsolvable. The exact method, on the other hand, detects unsolvable problem instances in finite time. The thesis demonstrates, both in theory and experiments, that mobile robots can improve communications by planning trajectories that maintain visual connectivity, or by exploiting multipath fading when there is no line of sight. The proposed methods are well suited for integration in a layered motion planning architecture. / QC 20120117

The objective of modeling, verification, and synthesis of hierarchical hybrid mission control for underwater vehicle is to (i) propose a hierarchical architecture for mission control for an autonomous system, (ii) develop extended hybrid state machine models for the mission control, (iii) use these models to verify for logical correctness, (iv) check the feasibility of a simulation software to model the mission executed by an autonomous underwater vehicle (AUV) (v) perform synthesis of high-level mission coordinators for coordinating lower-level mission controllers in accordance with the given mission, and (vi) suggest further design changes for improvement. The dissertation describes a hierarchical architecture in which mission level controllers based on hybrid systems theory have been, and are being developed using a hybrid systems design tool that allows graphical design, iterative redesign, and code generation for rapid deployment onto the target platform. The goal is to support current and future autonomous underwater vehicle (AUV) programs to meet evolving requirements and capabilities. While the tool facilitates rapid redesign and deployment, it is crucial to include safety and performance verification into each step of the (re)design process. To this end, the modeling of the hierarchical hybrid mission controller is formalized to facilitate the use of available tools and newly developed methods for formal verification of safety and performance specifications. A hierarchical hybrid architecture for mission control of autonomous systems with application to AUVs is proposed and a theoretical framework for the models that make up the architecture is outlined. An underwater vehicle like any other autonomous system is a hybrid system, as the dynamics of the vehicle as well as its vehicle level control is continuous whereas the mission level control is discrete, making the overall system a hybrid system i.e., one possessing both continuous and discrete states. The hybrid state machine models of the mission controller modules is derived from their implementation done using TEJA, a software for representing hybrid systems with support for auto code generation. The verification of their logical correctness properties has been done using UPPAAL, a software tool for verification of timed automata a special kind of hybrid system. A Teja to Uppaal converter, called dem2xml, has been created at Applied Reserarch Lab that converts a hybrid (timed) autonomous system description in Teja to an Uppaal system description. Verification work involved developing abstract models for the lower level vehicle controllers with which the mission controller modules interact and follow a hierarchical approach: Assuming the correctness of level-zero or vehicle controllers, we establish the correctness of level-one mission controller modules, and then the correctness of level-two modules, etc. The goal of verification is to show that any valid meaning for a mission formalized in our research verifies the safe and correct execution of actions. Simulation of the sequence of actions executed for each of the operations give a better view of the combined working of the mission coordinators and the low level controllers. So we next looked into the feasibility of simulating the operations executed during a mission. A Perl program has been developed to convert the UPPAAL files in .xml format to OpenGL graphic files. The graphic files simulate the steps involved in the execution of a sequence of operations executed by an AUV. The highest level coordinators send mission orders to be executed by the lower level controllers. So a more generalized design of the highest level controllers would help to incorporate the execution of a variety of missions for a vast field of applications. Initially, we consider manually synthesized mission coordinator modules. Later we design automated synthesis of coordinators. This method synthesizes mission coordinators which coordinate the lower level controllers for the execution of the missions ordered and can be used for any autonomous system.