Exploring the advancement of soft and variable impedance actuators (VIAs), the research focuses on their potential for enhancing safety and adaptability in physical human-robot interactions (pHRI). Despite the promising attributes of these technologies, their adoption in portable applications is still emerging. Addressing the challenges hindering the widespread implementation of soft actuators and VIAs, a multidisciplinary approach is employed, spanning materials science, chemistry, thermodynamics, and more. Novel compliant actuators utilizing phase-change materials and flexible thermoelectric devices are introduced, offering improved safety, adaptability, and efficiency. Thermo-active phase change soft actuators, integrating Peltier junctions, achieve precise thermal control and reversible actuation, overcoming traditional Joule heating limitations for more efficient and controlled thermal responses. The research also delves into thermal variable impedance actuators, using viscoelastic polymers like polycaprolactone (PCL) for variable stiffness and damping. This innovation enables rapid adaptation to changing load conditions, enhancing the dynamic performance of VIAs. Key contributions encompass the design of an agonist-antagonist system using thermo-active phase change materials, applications in soft robotic devices such as grippers and locomotion mechanisms, and the implementation of bidirectional heating elements within these actuators. The work also outlines the challenges encountered, such as gravity's influence on actuation and the frequency-dependent properties of PCL, setting the stage for future research directions to advance the field of soft robotics. Through these contributions, the research demonstrates practical applications of soft and variable impedance actuators in pHRI, paving the way for future innovations in soft robotics.

This thesis investigates two well defined problems in image segmentation, viz. interactive and semantic image segmentation. Interactive segmentation involves power assisting a user in cutting out objects from an image, whereas semantic segmentation involves partitioning pixels in an image into object categories. We investigate various models and energy formulations for both these problems in this thesis. In order to improve the performance of interactive systems, low level texture features are introduced as a replacement for the more commonly used RGB features. To quantify the improvement obtained by using these texture features, two annotated datasets of images are introduced (one consisting of natural images, and the other consisting of camouflaged objects). A significant improvement in performance is observed when using texture features for the case of monochrome images and images containing camouflaged objects. We also explore adding mid-level cues such as shape constraints into interactive segmentation by introducing the idea of geodesic star convexity, which extends the existing notion of a star convexity prior in two important ways: (i) It allows for multiple star centres as opposed to single stars in the original prior and (ii) It generalises the shape constraint by allowing for Geodesic paths as opposed to Euclidean rays. Global minima of our energy function can be obtained subject to these new constraints. We also introduce Geodesic Forests, which exploit the structure of shortest paths in implementing the extended constraints. These extensions to star convexity allow us to use such constraints in a practical segmentation system. This system is evaluated by means of a “robot user” to measure the amount of interaction required in a precise way, and it is shown that having shape constraints reduces user effort significantly compared to existing interactive systems. We also introduce a new and harder dataset which augments the existing GrabCut dataset with more realistic images and ground truth taken from the PASCAL VOC segmentation challenge. In the latter part of the thesis, we bring in object category level information in order to make the interactive segmentation tasks easier, and move towards fully automated semantic segmentation. An algorithm to automatically segment humans from cluttered images given their bounding boxes is presented. A top down segmentation of the human is obtained using classifiers trained to predict segmentation masks from local HOG descriptors. These masks are then combined with bottom up image information in a local GrabCut like procedure. This algorithm is later completely automated to segment humans without requiring a bounding box, and is quantitatively compared with other semantic segmentation methods. We also introduce a novel way to acquire large quantities of segmented training data relatively effortlessly using the Kinect. In the final part of this work, we explore various semantic segmentation methods based on learning using bottom up super-pixelisations. Different methods of combining multiple super-pixelisations are discussed and quantitatively evaluated on two segmentation datasets. We observe that simple combinations of independently trained classifiers on single super-pixelisations perform almost as good as complex methods based on jointly learning across multiple super-pixelisations. We also explore CRF based formulations for semantic segmentation, and introduce novel visual words based object boundary description in the energy formulation. The object appearance and boundary parameters are trained jointly using structured output learning methods, and the benefit of adding pairwise terms is quantified on two different datasets.

Stroke is the largest cause of severe adult complex disability, caused when the blood supply to the brain is interrupted, either by a clot or a burst blood vessel. It is characterised by deficiencies in movement and balance, changes in sensation, impaired motor control and muscle tone, and bone deformity. Clinically applied stroke management relies heavily on the observational opinion of healthcare workers. Despite the proven validity of a few clinical outcome measures, they remain subjective and inconsistent, and suffer from a lack of standardisation. Motion capture of the upper limb has also been used in specialised laboratories to obtain accurate and objective information, and monitor progress in rehabilitation. However, it is unsuitable in environments that are accessible to stroke patients (for example at patients’ homes or stroke clubs), due to the high cost, special set-up and calibration requirements. The aim of this research project was to validate and assess the sensitivity of a relatively low cost, wearable, compact and easy-to-use monitoring system, which uses inertial sensors in order to obtain detailed analysis of the forearm during simple functional exercises, typically used in rehabilitation. Forearm linear and rotational motion were characterised for certain movements on four healthy subjects and a stroke patient using a motion capture system. This provided accuracy and sensitivity specifications for the wearable monitoring system. With basic signal pre-processing, the wearable system was found to report reliably on acceleration, angular velocity and orientation, with varying degrees of confidence. Integration drift errors in the estimation of linear velocity were unresolved. These errors were not straightforward to eliminate due to the varying position of the sensor accelerometer relative to gravity over time. The cyclic nature of rehabilitation exercises was exploited to improve the reliability of velocity estimation with model-based Kalman filtering, and least squares optimisation techniques. Both signal processing methods resulted in an encouraging reduction of the integration drift in velocity. Improved sensor information could provide a visual display of the movement, or determine kinematic quantities relevant to the exercise performance. Hence, the system could potentially be used to objectively inform patients and physiotherapists about progress, increasing patient motivation and improving consistency in assessment and reporting of outcomes.

In this thesis, we present three main contributions to laser-based detection and tracking of dynamic objects, from both a model-based point of view and a model-free point of view, with an emphasis on applications to autonomous driving. A segmentation-based detector is first proposed to provide an end-to-end detection of the classes car, pedestrian and bicyclist in 3D laser data amongst significant background clutter. We postulate that, for the particular classes considered, solving a binary classification task outperforms approaches that tackle the multi-class problem directly. This is confirmed using custom and third-party datasets gathered of urban street scenes. The sliding window approach to object detection, while ubiquitous in the Computer Vision community, is largely neglected in laser-based object detectors, possibly due to its perceived computational inefficiency. We give a second thought to this opinion in this thesis, and demonstrate that, by fully exploiting the sparsity of the problem, exhaustive window searching in 3D can be made efficient. We prove the mathematical equivalence between sparse convolution and voting, and devise an efficient algorithm to compute exactly the detection scores at all window locations, processing a complete Velodyne scan containing 100K points in less than half a second. Its superior performance is demonstrated on the KITTI dataset, and compares commensurably with state of the art vision approaches. A new model-free approach to detection and tracking of moving objects with a 2D lidar is then proposed aiming at detecting dynamic objects of arbitrary shapes and classes. Objects are modelled by a set of rigidly attached sample points along their boundaries whose positions are initialised with and updated by raw laser measurements, allowing a flexible, nonparametric representation. Dealing with raw laser points poses a significant challenge to data association. We propose a hierarchical approach, and present a new variant of the well-known Joint Compatibility Branch and Bound algorithm to handle large numbers of measurements. The system is systematically calibrated on real world data containing 7.5K labelled object examples and validated on 6K test cases. Its performance is demonstrated over an existing industry standard targeted at the same problem domain as well as a classical approach to model-free tracking.

In this thesis, we investigate the problem of human layout estimation in unconstrained still images. This involves predicting the spatial configuration of body parts. We start our investigation with pictorial structure models and propose an efficient method of model fitting using skin regions. To detect the skin, we learn a colour model locally from the image by detecting the facial region. The resulting skin detections are also used for hand localisation. Our next contribution is a comprehensive dataset of 2D hand images. We collected this dataset from publicly available image sources, and annotated images with hand bounding boxes. The bounding boxes are not axis aligned, but are rather oriented with respect to the wrist. Our dataset is quite exhaustive as it includes images of different hand shapes and layout configurations. Using our dataset, we train a hand detector that is robust to background clutter and lighting variations. Our hand detector is implemented as a two-stage system. The first stage involves proposing hand hypotheses using complementary image features, which are then evaluated by the second stage classifier. This improves both precision and recall and results in a state-of-the-art hand detection method. In addition we develop a new method of non-maximum suppression based on super-pixels. We also contribute an efficient training algorithm for structured output ranking. In our algorithm, we reduce the time complexity of an expensive training component from quadratic to linear. This algorithm has a broad applicability and we use it for solving human layout estimation and taxonomic multiclass classification problems. For human layout, we use different body part detectors to propose part candidates. These candidates are then combined and scored using our ranking algorithm. By applying this bottom-up approach, we achieve accurate human layout estimation despite variations in viewpoint and layout configuration. In the multiclass classification problem, we define the misclassification error using a class taxonomy. The problem then reduces to a structured output ranking problem and we use our ranking method to optimise it. This allows inclusion of semantic knowledge about the classes and results in a more meaningful classification system. Lastly, we substantiate our ranking algorithm with theoretical proofs and derive the generalisation bounds for it. These bounds prove that the training error reduces to the lowest possible error asymptotically.

High Intensity Focused Ultrasound (HIFU) is a new cancer thermal therapy method which has achieved encouraging results in clinics recently. However, the lack of a temperature monitoring makes it hard to apply widely, safely and efficiently. Conventional ultrasonic temperature estimation based on echo strain suffers from artifacts caused by signal distortion over time, leading to poor estimation and visualization of the 2D temperature map. This thesis presents a novel model-based stochastic framework for ultrasonic temperature estimation, which combines the temperature information from the ultrasound images and a theoretical model of the heat diffusion. Consequently the temperature estimation is more consistent over time and its visualisation is improved. There are 3 main contributions of this thesis related to: improving the conventional echo strain method to estimate temperature, developing and applying approximate heat models to model temperature, and finally combining the estimation and the models. First in the echo strain based temperature estimation, a robust displacement estimator is first introduced to remove displacement outliers caused by the signal distortion over time due to the thermo-acoustic lens effect. To transfer the echo strain to temperature more accurately, an experimental method is designed to model their relationship using polynomials. Experimental results on a gelatine phantom show that the accuracy of the temperature estimation is of the order of 0.1 ◦C. This is better than results reported previously of 0.5 ◦C in a rubber phantom. Second in the temperature modelling, heat models are derived approximately as Gaussian functions which are mathematically simple. Simulated results demonstrate that the approximate heat models are reasonable. The simulated temperature result is analytical and hence computed in much less than 1 second, while the conventional simulation of using finite element methods requires about 25 minutes under the same conditions. Finally, combining the estimation and the heat models is the main contribution of this thesis. A 2D spatial adaptive Kalman filter with the predictive step defined by the shape model from the heat models is applied to the temperature map estimated from ultrasound images. It is shown that use of the temperature shape model enables more reliable temperature estimation in the presence of distorted or blurred strain measurements which are typically found in practice. The experimental results on in-vitro bovine liver show that the visualisation on the temperature map over time is more consistent and the iso-temperature contours are clearly visualised.

This thesis dissertation mainly compares and investigates laboratory study of different implementation methodologies of applied control systems and how they can be adopted in industrial, as well as commercial, automation applications. Namely the research paper aims to assess or evaluate eventual feedback control loops' performance and robustness over multiple conventional or state-of-the-art technologies in the field of applied industrial automation and instrumentation by implementing a laboratory case study setup: the ball on beam system. Hence, the paper tries to close the gap between industry and academia by: first, conducting a historical study and background information of main evolutional and technological eras in the field of industrial process control automation and instrumentation. Then, some related basic theoretical as well as practical concepts are reviewed in Chapter 2 of the report before displaying the detailed design. After that, the next Chapter, analyses the ball on beam control system problem as the case studied in the context of this research through reviewing previous literature, modeling and simulation. The following Chapter details the proposed design and implementation of the ball on beam case study as if it is under the introduced distributed industrial automation architecture. Finally, Chapter 5 concludes this work by listing several points leaned, remarks, and observations, and stating possible development and the future vision of this research.

Les moyens numériques ont transformé les pratiques en architecture notamment en facilitant la continuité des flux d’information entre la conception et la fabrication. À travers ce mémoire, nous explorons les contributions possibles de ces moyens dans un contexte de pénurie d’artisans et de perte de savoir-faire artisanaux en architecture. Nous nous sommes appuyés sur les modèles de la tectonique informée pour la conception à base des matériaux, documentés par R. Oxman (2014), qui regroupent un ensemble d’approches et d’outils numériques, afin d’étudier, dans un premier temps, les apports et limites de ces derniers à partir de plusieurs cas expérimentaux; c’est-à-dire expérimental. Ainsi, nous avons pu identifier certains savoir-faire artisanaux dont le déclin est possiblement compensable par les moyens numériques. Dans un deuxième temps, nous avons décelé les contributions des moyens numériques dans un projet à partir de l’analyse des exemples de la construction de la Sagrada Familia et la restauration du parlement canadien à Ottawa. Les principaux résultats de notre recherche sont : • Nous avons distingué que la modélisation paramétrique est plus adaptée au processus de conception tandis que la modélisation à base de relevés est plus appropriée pour la production des répliques. • Les outils numériques de fabrication permettent d’employer plusieurs types de matériaux. Elles assurent une rapidité et une précision dans la réalisation des artéfacts. Cependant, l’intervention des artisans demeure incontournable dans la majorité des cas. Les moyens numériques ont un grand potentiel pour compenser la perte des savoir-faire artisanaux et pour développer de nouveaux processus créatifs. Toutefois, à ce jour, ils se limitent au rôle d’assistants et devraient être perfectionnés. / Digital tools have transformed the architectural practices due to the information flow continuity between design and manufacturing. Through this these, we explore their possible contributions in the context of artisans’ shortage and craft skills’ losses in architecture. We relied on informed tectonic models of materials based design, documented by R. Oxman (2014), which combines a variety of digital approaches and tools to study their contributions and limits based on multiple case studies. Thus, we identified some craft skills whose decline is possibly compensated by digital tools. Then, we analyzed the digital tools’ contributions in a project based on the Sagrada Familia and Canadian parliament examples. The main results of our research are: • Digital in architecture is based on digital models. The parametric approach is more suited to design process while models based on lasergrammetry and photogrammetry surveys are more appropriate for replica production. • Digital manufacturing tools allow the use of several materials types. Similarly, they provide speed and accuracy. However, the artisans’ participation remains unavoidable in most cases. Digital media have great potential to overcome the craft skills’ loss and even exploit them in new creative processes. However, to date, they are limited to the role of assistants and should be perfected.

Les systèmes de dialogues sont à même de révolutionner l'interaction entre l'homme et la machine. Pour autant, les efforts pour concevoir des agents conversationnels se sont souvent révélés infructueux, et ceux, malgré les dernières avancées en apprentissage profond et par renforcement. Les systèmes de dialogue palissent de devoir opérer sur de nombreux domaines d'application mais pour lesquels aucune mesure d'évaluation claire n'a été définie. Aussi, cette thèse s'attache à étudier les dialogues débouchant sur un objectif clair (goal-oriented dialogue) permettant de guider l'entrainement, et ceci, dans des environnements multimodaux. Plusieurs raisons expliquent ce choix : (i) cela contraint le périmètre de la conversation, (ii) cela introduit une méthode d'évaluation claire, (iii) enfin, l'aspect multimodal enrichie la représentation linguistique en reliant l'apprentissage du langage avec des expériences sensorielles. En particulier, nous avons développé GuessWhat?! (Qu-est-ce donc?!), un jeu imagé coopératif où deux joueurs tentent de retrouver un objet en posant une série de questions. Afin d’apprendre aux agents de répondre aux questions sur les images, nous avons développés une méthode dites de normalisation conditionnée des données (Conditional Batch Nornalization). Ainsi, cette méthode permet d'adapter simplement mais efficacement des noyaux de convolutions visuels en fonction de la question en cours. Enfin, nous avons étudié les tâches de navigation guidée par dialogue, et introduit la tâche Talk the Walk (Raconte-moi le Chemin) à cet effet. Dans ce jeu, deux agents, un touriste et un guide, s'accordent afin d'aider le touriste à traverser une reconstruction virtuelle des rues de New-York et atteindre une position prédéfinie. / While dialogue systems have the potential to fundamentally change human-machine interaction, developing general chatbots with deep learning and reinforce-ment learning techniques has proven difficult. One challenging aspect is that these systems are expected to operate in broad application domains for which there is not a clear measure of evaluation. This thesis investigates goal-oriented dialogue tasks in multi-modal environments because it (i) constrains the scope of the conversa-tion, (ii) comes with a better-defined objective, and (iii) enables enriching language representations by grounding them to perceptual experiences. More specifically, we develop GuessWhat, an image-based guessing game in which two agents cooper-ate to locate an unknown object through asking a sequence of questions. For the subtask of visual question answering, we propose Conditional Batch Normalization layers as a simple but effective conditioning method that adapts the convolutional activations to the specific question at hand. Finally, we investigate the difficulty of dialogue-based navigation by introducing Talk The Walk, a new task where two agents (a “tourist” and a “guide”) collaborate to have the tourist navigate to target locations in the virtual streets of New York City.

The demand for automated decision making, learning and inference in uncertain, risk sensitive and dynamically changing situations presents a challenge: to design computational approaches that promise to be widely deployable and flexible to adapt on the one hand, while offering reliable guarantees on safety on the other. The tension between these desiderata has created a gap that, in spite of intensive research and contributions made from a wide range of communities, remains to be filled. This represents an intriguing challenge that provided motivation for much of the work presented in this thesis. With these desiderata in mind, this thesis makes a number of contributions towards the development of algorithms for automated decision-making and inference under uncertainty. To facilitate inference over unobserved effects of actions, we develop machine learning approaches that are suitable for the construction of models over dynamical laws that provide uncertainty bounds around their predictions. As an example application for conservative decision-making, we apply our learning and inference methods to control in uncertain dynamical systems. Owing to the uncertainty bounds, we can derive performance guarantees of the resulting learning-based controllers. Furthermore, our simulations demonstrate that the resulting decision-making algorithms are effective in learning and controlling under uncertain dynamics and can outperform alternative methods. Another set of contributions is made in multi-agent decision-making which we cast in the general framework of optimisation with interaction constraints. The constraints necessitate coordination, for which we develop several methods. As a particularly challenging application domain, our exposition focusses on collision avoidance. Here we consider coordination both in discrete-time and continuous-time dynamical systems. In the continuous-time case, inference is required to ensure that decisions are made that avoid collisions with adjustably high certainty even when computation is inevitably finite. In both discrete-time and finite-time settings, we introduce conservative decision-making. That is, even with finite computation, a coordination outcome is guaranteed to satisfy collision-avoidance constraints with adjustably high confidence relative to the current uncertain model. Our methods are illustrated in simulations in the context of collision avoidance in graphs, multi-commodity flow problems, distributed stochastic model-predictive control, as well as in collision-prediction and avoidance in stochastic differential systems. Finally, we provide an example of how to combine some of our different methods into a multi-agent predictive controller that coordinates learning agents with uncertain beliefs over their dynamics. Utilising the guarantees established for our learning algorithms, the resulting mechanism can provide collision avoidance guarantees relative to the a posteriori epistemic beliefs over the agents' dynamics.