Integrating computer vision into a robotic system can provide a closed-loop controlled platform that increases the robustness of a robot's motion. This integration is also known as visual servo control or visual servoing. Visual servoing of a robot manipulator in real-time presents complex engineering problems with respect to both control and image processing particularly when we want the robot arm to perform complicated tasks such as portrait drawing. In my research, the implementation of torque feedback control and Image-based Visual Servoing (IBVS) approaches are proposed to improve previous open-loop portrait drawing tasks performed by Betty, a humanoid robot in the Autonomous Agent Lab, University of Manitoba. The implementations and evaluations of hardware, software and kinematic models are discussed in this document. I examined the problem of estimating ideal edges joining points in a pixel reduction image for an existing point-to-point portrait drawing humanoid robot, Betty. To solve this line drawing problem, two automatic sketch generators are presented. First, a modified Theta-graph, called Furthest Neighbour Theta-graph (FNTG). Second, an extension of the Edge Drawing Lines algorithm (EDLines), called Extended Edge Drawing Lines (eEDLines). The results show that the number of edges in the resulting drawing is significantly reduced without degrading the detail of the output image. The other main objective of this research is to propose the extension of the drawing robot project to further develop a robust visual servoing system for Betty to correct any drawing deviation in real-time as a human does. This is achieved by investigating and developing robust feature (lines and shading) extraction approaches for real-time feature tracking of IBVS in combination with adequate torque feedback in the drawing task.

No description available.

the 18th World Congress The International Federation of Automatic Control, Milano (Italy), August 28 - September 2, 2011

Impact at each leg transition is one of the main causes of energy dissipation in most of the current bipedal walking robots. Minimizing impact can reduce the energy loss. Instead of controlling the joint angle profiles to reduce the impact which requires significant amount of energy, installing elastic mechanisms on the robots structure is proposed in this research, enabling the robot to reduce the impact, and to store part of the energy in the elastic form which returns the energy to the robot. Practically, this motivates the development of the bipedal walking robots with adjustable stiffness elasticity which itself creates new challenging problems. This thesis addresses some of the challenges through five consecutive stages. Firstly, an adjustable compliant series elastic actuator (named ACSEA in this thesis) is developed. The velocity control mode of the electric motor is used to accurately control the output force of the ACSEA. Secondly, three different conceptual designs of the adjustable stiffness artificial tendons (ASAT) are proposed each of which is added at the ankle joint of a bipedal walking robot model. Simulation results of the collision phase (part of the gait between the heel-strike and the foot-touch-down in bipedal walking) demonstrate significant improvements in the energetics of the bipedal walking robot by proper stiffness adjustment of ASAT. In the third stage, in order to study the effects of ASATs on reducing the energy loss during the stance phase, a simplified model of bipedal walking is introduced consisting of a foot, a leg and an ASAT which is installed parallel to the ankle joint. A linear spring, with adjustable stiffness, is included in the model to simulate the generated force by the trailing leg during the double support phase. The concept of impulsive constraints is used to establish the mathematical model of impacts in the collision phase which includes the heel-strike and the foot-touch-down. For the fourth stage, an energy-feedback-based controller is designed to automatically adjust the stiffness of the ASAT which reduces the energy loss during the foot-touch-down. In the final stage, a speed tracking (ST) controller is developed to regulate the velocity of the biped at the midstance. The ST controller is an event-based time-independent controller, based on geometric progression with exponential decay in the kinetic energy error, which adjusts the stiffness of the trailing-leg spring to control the injected energy to the biped in tracking a desired speed at the midstance. Another controller is also integrated with the ST controller to tune the stiffness of the ASAT when reduction in the speed is desired. Then, the local stability of the system (biped and the combination of the above three controllers) is analyzed by calculating the eigenvalues of the linear approximation of the return map. Simulation results show that the combination of the three controllers is successful in tracking a desired speed of the bipedal walking even in the presence of the uncertainties in the leg’s initial angles. The outcomes of this research show the significant effects of adjustable stiffness artificial tendons on reducing the energy loss during bipedal walking. It also demonstrates the advantages of adding elastic elements in the bipedal walking model which benefits the efficiency and simplicity in regulating the speed. This research paves the way toward developing the dynamic walking robots with adjustable stiffness ability which minimize the shortcomings of the two major types of bipedal walking robots, i.e., passive dynamic walking robots (which are energy efficient but need extensive parameters tuning for gait stability) and actively controlled walking robots (which are significantly energy inefficient).

The number of elderly people around the world is growing rapidly. This has led to an increase in the number of people who are seeking assistance and adequate service either at home or in long-term- care institutions to successfully accomplish their daily activities. Responding to these needs has been a burden to the health care system in terms of labour and associated costs and has motivated research in developing alternative services using new technologies. Various intelligent, and non-intelligent, machines and robots have been developed to meet the needs of elderly and people with upper limb disabilities or dysfunctions in gaining independence in eating, which is one of the most frequent and time-consuming everyday tasks. However, in almost all cases, the proposed systems are designed only for the personal use of one individual and little effort to design a multiple-user feeding robot has been previously made. The feeding requirements of elderly in environments such as senior homes, where many elderly residents dine together at least three times per day, have not been extensively researched before. The aim of this research was to develop a machine to feed multiple elderly people based on their characteristics and feeding needs, as determined through observations at a nursing home. Observations of the elderly during meal times have revealed that almost 40% of the population was totally dependent on nurses or caregivers to be fed. Most of those remaining, suffered from hand tremors, joint pain or lack of hand muscle strength, which made utensil manipulation and coordination very difficult and the eating process both messy and lengthy. In addition, more than 43% of the elderly were very slow in eating because of chewing and swallowing problems and most of the rest were slow in scooping and directing utensils toward their mouths. Consequently, one nurse could only respond to a maximum of two diners simultaneously. In order to manage the needs of all elderly diners, they required the assistance of additional staff members. The limited time allocated for each meal and the daily progression of the seniors??? disabilities also made mealtime very challenging. Based on the caregivers??? opinion, many of the elderly in such environments can benefit from a machine capable of feeding multiple users simultaneously. Since eating is a slow procedure, the idle state of the robot during one user???s chewing and swallowing time can be allotted for feeding another person who is sitting at the same table. The observations and studies have resulted in the design of a food tray, and selection of an appropriate robot and applicable user interface. The proposed system uses a 6-DOF serial articulated robot in the center of a four-seat table along with a specifically designed food tray to feed one to four people. It employs a vision interface for food detection and recognition. Building the dynamic equations of the robotic system and simulation of the system were used to verify its dynamic behaviour before any prototyping and real-time testing.

People and animals use various kinds of motion in a multitude of ways to communicate their ideas and affective states, such as their moods or emotions. Further, people attribute affect and personalities to movements of even abstract entities based solely on the style of their motions, e.g., movement of a geometric shape (how it moves about) can be interpreted as being shy, aggressive, etc. In this thesis, we investigated how flying robots can leverage this locomotion-style communication channel for communicating their states to people. One problem in leveraging this style of communication in robot design is that there are no guidelines, or tools that Human-Robot Interaction (HRI) designers can leverage to author affect communicating locomotion paths for flying robots. Therefore, we propose to adapt the Laban Effort System (LES), a standard method for interpreting human motion commonly used in the performing arts, to develop a set of guidelines that can be leveraged by HRI designers to author affective locomotion paths for flying robots. We further validate our proposed approach by conducting a small design workshop with a group of interaction designers, where they were asked to design robotic behaviors using our design method. We conclude this thesis with an original adaption of LES to the locomotion path of a flying robot, and a set of design guidelines that can be leveraged by interaction designers for building affective locomotion path for a flying robot.

Impact at each leg transition is one of the main causes of energy dissipation in most of the current bipedal walking robots. Minimizing impact can reduce the energy loss. Instead of controlling the joint angle profiles to reduce the impact which requires significant amount of energy, installing elastic mechanisms on the robots structure is proposed in this research, enabling the robot to reduce the impact, and to store part of the energy in the elastic form which returns the energy to the robot. Practically, this motivates the development of the bipedal walking robots with adjustable stiffness elasticity which itself creates new challenging problems. This thesis addresses some of the challenges through five consecutive stages. Firstly, an adjustable compliant series elastic actuator (named ACSEA in this thesis) is developed. The velocity control mode of the electric motor is used to accurately control the output force of the ACSEA. Secondly, three different conceptual designs of the adjustable stiffness artificial tendons (ASAT) are proposed each of which is added at the ankle joint of a bipedal walking robot model. Simulation results of the collision phase (part of the gait between the heel-strike and the foot-touch-down in bipedal walking) demonstrate significant improvements in the energetics of the bipedal walking robot by proper stiffness adjustment of ASAT. In the third stage, in order to study the effects of ASATs on reducing the energy loss during the stance phase, a simplified model of bipedal walking is introduced consisting of a foot, a leg and an ASAT which is installed parallel to the ankle joint. A linear spring, with adjustable stiffness, is included in the model to simulate the generated force by the trailing leg during the double support phase. The concept of impulsive constraints is used to establish the mathematical model of impacts in the collision phase which includes the heel-strike and the foot-touch-down. For the fourth stage, an energy-feedback-based controller is designed to automatically adjust the stiffness of the ASAT which reduces the energy loss during the foot-touch-down. In the final stage, a speed tracking (ST) controller is developed to regulate the velocity of the biped at the midstance. The ST controller is an event-based time-independent controller, based on geometric progression with exponential decay in the kinetic energy error, which adjusts the stiffness of the trailing-leg spring to control the injected energy to the biped in tracking a desired speed at the midstance. Another controller is also integrated with the ST controller to tune the stiffness of the ASAT when reduction in the speed is desired. Then, the local stability of the system (biped and the combination of the above three controllers) is analyzed by calculating the eigenvalues of the linear approximation of the return map. Simulation results show that the combination of the three controllers is successful in tracking a desired speed of the bipedal walking even in the presence of the uncertainties in the leg’s initial angles. The outcomes of this research show the significant effects of adjustable stiffness artificial tendons on reducing the energy loss during bipedal walking. It also demonstrates the advantages of adding elastic elements in the bipedal walking model which benefits the efficiency and simplicity in regulating the speed. This research paves the way toward developing the dynamic walking robots with adjustable stiffness ability which minimize the shortcomings of the two major types of bipedal walking robots, i.e., passive dynamic walking robots (which are energy efficient but need extensive parameters tuning for gait stability) and actively controlled walking robots (which are significantly energy inefficient).

Randomly scattered sensors may cause sensing holes and redundant sensors. In carrier-based sensor relocation, mobile robots (with limited capacity to carry sensors) pick up additional or redundant sensors and relocate them at sensing holes. In the only known localized algorithm, robots randomly traverse field and act based on identified pair of spare sensor and coverage hole. We propose a Market-based Sensor Relocation (MSR) algorithm, which optimizes sensor deployment location, and introduces bidding and coordinating among neighboring robots. Sensors along the boundary of each hole elect one of them as the representative, which bids to neighboring robots for hole filling service. Robot randomly explores by applying Least Recently Visited policy. It chooses the best bid according to Cost over Progress ratio and fetches a spare sensor nearby to cover the corresponding sensing hole. Robots within communication range share their tasks to search for better possible solutions. Simulation shows that MSR outperforms the existing competing algorithm G-R3S2 significantly on total robot traversed path and energy, and time to cover holes, slightly on number of sensors needed to cover the hole, and the cost of additional messages for bidding and deployment location sharing.

This thesis looks at the problem of optimally placing binary directional proximity markers to assist a robot as it navigates waypoints through an environment. A simple planar fiducial marker is developed to serve as the binary directional proximity marker. A scoring function is proposed for marker placement as well as a method for random generation of hallway maps. Several common metaheuristic algorithms are run to find optimal marker placements with respect to the scoring function for a number of randomly generated hallway maps. From these results, placements are then evaluated by physical experimentation on an iRobot Create equipped with relatively inexpensive webcams. / Graduate

Studies of minimalist multi-robot systems consider multiple robotic agents, each with limited individual capabilities, but with the capacity for self-organization in order to collectively perform coordinated tasks. Object clustering is a widely studied task in which self-organized robots form piles from dispersed objects. Our work considers a variation of an object clustering derived from the influential ant-inspired work of Beckers, Holland and Deneubourg which proposed stigmergy as a design principle for such multi-robot systems. Since puck mechanics contribute to cluster accrual dynamics, we studied a new scenario with square objects because these pucks into clusters differently from cylindrical ones. Although central clusters are usually desired, workspace boundaries can cause perimeter cluster formation to dominate. This research demonstrates successful clustering of square boxes - an especially challenging instance since flat edges exacerbate adhesion to boundaries - using simpler robots than previous published research. Our solution consists of two novel behaviours, Twisting and Digging, which exploit the objects’ geometry to pry boxes free from boundaries. Physical robot experiments illustrate that cooperation between twisters and diggers can succeed in forming a single central cluster. We empirically explored the significance of different divisions of labor by measuring the spatial distribution of robots and the system performance. Data from over 40 hours of physical robot experiments show that different divisions of labor have distinct features, e.g., one is reliable while another is especially efficient.