This thesis presents a novel bilateral teleoperation algorithm for n degree of freedom nonlinear manipulators connected through time delays. Teleoperation has many practical uses, as there are many benefits that come from being able to operate machines from a distance. For instance, the ability to send a remote controlled robotic vehicle into a hazardous environment can be a great asset in many industrial applications. As well, the field of remote medicine can benefit from these technologies. A highly skilled surgeon could perform surgery on a patient who is located in another city, or even country. Earth to space operations and deep sea exploration are other areas where teleoperation is quite useful. Central to the approach presented in this work is the use of second order sliding mode unknown input observers for estimating the external forces acting on the manipulators. The use of these observers removes the need for both velocity and force sensors, leading to a lower cost hardware setup that provides all of the advantages of a position-force teleoperation algorithm. Stability results for this new algorithm are presented for several cases. Stability of each of the master and slave sides of the teleoperation system is demonstrated, showing that the master and slave are both stabilized by their respective controllers when the unknown input observers are used for state and force estimation. Additionally, closed loop stability results for the teleoperation system connected to a variety of slave side environments are presented. Delay-independent stability results for a linear spring-damper environment as well as a general finite-gain stable nonlinear environment are given. Delay-dependent stability results for the case where the slave environment is a liner spring-damper and the delays are commensurate are also presented. As well, stability results for the closed loop under the assumption that the human operator is modeled as a finite-gain stable nonlinear environment are given. Following the theoretical presentation, numerical simulations illustrating the algorithm are presented, and experimental results verifying the practical application of the approach are given.

During last decades, the Internet teleobotics has been growing at an enormous ratedue to the rapid improvement of Internet technology. This paper presents theinternet-based remote control of mobile robot. To face unpredictable Internet delaysand possible connection rupture, a direct continuous control based teleoperationarchitecture with “Speed Limit Module” (SLM) and “Delay Approximator” (DA) isproposed. This direct continuous control architecture guarantees the path error of therobot motion is restricted within the path error tolerance of the application.Experiment results show the feasibility and effectiveness of this direct Internet controlarchitecture in the real Internet environment.

This thesis presents a novel bilateral teleoperation algorithm for n degree of freedom nonlinear manipulators connected through time delays. Teleoperation has many practical uses, as there are many benefits that come from being able to operate machines from a distance. For instance, the ability to send a remote controlled robotic vehicle into a hazardous environment can be a great asset in many industrial applications. As well, the field of remote medicine can benefit from these technologies. A highly skilled surgeon could perform surgery on a patient who is located in another city, or even country. Earth to space operations and deep sea exploration are other areas where teleoperation is quite useful. Central to the approach presented in this work is the use of second order sliding mode unknown input observers for estimating the external forces acting on the manipulators. The use of these observers removes the need for both velocity and force sensors, leading to a lower cost hardware setup that provides all of the advantages of a position-force teleoperation algorithm. Stability results for this new algorithm are presented for several cases. Stability of each of the master and slave sides of the teleoperation system is demonstrated, showing that the master and slave are both stabilized by their respective controllers when the unknown input observers are used for state and force estimation. Additionally, closed loop stability results for the teleoperation system connected to a variety of slave side environments are presented. Delay-independent stability results for a linear spring-damper environment as well as a general finite-gain stable nonlinear environment are given. Delay-dependent stability results for the case where the slave environment is a liner spring-damper and the delays are commensurate are also presented. As well, stability results for the closed loop under the assumption that the human operator is modeled as a finite-gain stable nonlinear environment are given. Following the theoretical presentation, numerical simulations illustrating the algorithm are presented, and experimental results verifying the practical application of the approach are given.

There is currently a wide variety of six degree-of-freedom (6-DOF) motion capture technologies available. However, these systems tend to be very expensive and thus cost prohibitive. A software system was developed to provide 6-DOF motion capture using the Nintendo Wii remote’s (wiimote) sensors, an infrared beacon, and a novel hierarchical linear-quaternion Kalman filter. The software is made freely available, and the hardware costs less than one hundred dollars. Using this motion capture software, a robotic control system was developed to teleoperate a 6-DOF robotic manipulator via the operator’s natural hand movements. The teleoperation system requires calibration of the wiimote’s infrared cameras to obtain an estimate of the wiimote’s 6-DOF pose. However, since the raw images from the wiimote’s infrared camera are not available, a novel camera-calibration method was developed to obtain the camera’s intrinsic parameters, which are used to obtain a low-accuracy estimate of the 6-DOF pose. By fusing the low-accuracy estimate of 6-DOF pose with accelerometer and gyroscope measurements, an accurate estimation of 6-DOF pose is obtained for teleoperation. Preliminary testing suggests that the motion capture system has an accuracy of less than a millimetre in position and less than one degree in attitude. Furthermore, whole-system tests demonstrate that the teleoperation system is capable of controlling the end effector of a robotic manipulator to match the pose of the wiimote. Since this system can provide 6-DOF motion capture at a fraction of the cost of traditional methods, it has wide applicability in the field of robotics and as a 6-DOF human input device to control 3D virtual computer environments.

In conventional bilateral teleoperation, transmission delay over the Internet can potentially cause instability. The wave variable algorithm guarantees stability under varying transmission delay at the cost of poor transient performance. Adding a predictor on the master side can reduce this undesirable side-effect, but that would require a slave model. An inaccurate slave model used in the predictor as well as variations in transmission delay, both of which are likely under realistic situations, can result in steady state errors. A direct drift control algorithm is used to drive this error to zero regardless of the source of error. A semi-adaptive predictor that can distinguish between free space and rigid contact environment is used to provide more accurate force feedback on the master side. A full adaptive predictor is also used that estimates the slave environment parameters using recursive least squares with a forgetting factor. This research presents the experimental results and evaluations of the wave variable based methods under a realistic operation environment using a real master and slave. The effectiveness of this algorithm is fully evaluated using human subjects with no previous experience in haptics. Three algorithms are tested using PHANTOM brand haptic devices as master and slave: conventional bilateral teleoperation with no transmission delay as control, wave variable teleoperation with approximately 200 ms transmission delay one way, and wave variables with adaptive predictor and direct drift control with approximately 200 ms transmission delay one way. For each algorithm the human subjects are asked to perform three simple tasks: use the master to force the slave to track a reference trajectory in free space with the least amount of error, identify a contour surface on the slave side as accurately as possible using only haptic information from the master, and navigate a simple maze on the slave side in the least amount of time using haptic information from the master.

As humans, we have evolved to see in three dimensions. Our ancestors developed two eyes that only look forward, which allows the visual area that can perceive depth to be most of the field of view. A variety of sensors have been developed which can determine depth in the environment. They range from producing individual points of depth to the depth of everything in the environment. These sensors have become cheap and can now reliably produce accurate depth. Research is needed to determine how to present the proximity information to the people using the sensors. Touch, sound, and vision have all been used to provide depth information to the users. This research focuses on vision and compares methods of visually presenting proximity information to a user. The methods examined are stereovision and false color visual proximity mapping. False color mapping proved most effective while, surprisingly, stereovision was not helpful.

Robot and sensor networks are needed for safety, security, and rescue applicationssuch as port security and reconnaissance during a disaster. These applications rely on realtimetransmission of images, which generally saturate the available wireless networkinfrastructure. Knowledge-based Compression is a strategy for reducing the video frametransmission rate between robots or sensors and remote operators. Because images mayneed to be archived as evidence and/or distributed to multiple applications with differentpost processing needs, lossy compression schemes, such as MPEG, H.26x, etc., are notacceptable. This work proposes a lossless video server system consisting of three classesof filters (redundancy, task, and priority) which use different levels of knowledge (localsensed environment, human factors associated with a local task, and relative globalpriority of a task) at the application layer of the network. It demonstrates the redundancyand task filters for realistic robot search scenarios. The redundancy filter is shown toreduce the overall transmission bandwidth by 24.07% to 33.42%, and when combinedwith the task filter, reduces overall transmission bandwidth by 59.08% to 67.83%. Byitself, the task filter has the capability to reduce transmission bandwidth by 32.95% to33.78%. While Knowledge-based Compression generally does not reach the same levels ofreduction as MPEG, there are instances where the system outperforms MPEG encoding.

This work analyzes human-in-the-loop robotic systems to determine where human input can be most beneficial to a collaborative task. This is accomplished by implementing a pick-and-place task using a human-in-the-loop robotic system and determining which segments of the task, when replaced by human guidance, provide the most improvement to overall task performance and require the least cognitive effort. The first experiment entails implementing a pick and place task on a commercial robotic arm. Initially, we look at a pick-and-place task that is segmented into two main areas: coarse approach towards a goal object and fine pick motion. For the fine picking phase, we look at the importance of user guidance in terms of position and orientation of the end effector. Results from this initial experiment show that the most successful strategy for our human-in-the-loop system is the one in which the human specifies a general region for grasping, and the robotic system completes the remaining elements of the task. We extend this study to include a second experiment, utilizing a more complex robotic system and pick-and-place task to further analyze human impact in a human-in-the-loop system in a more realistic setting. In this experiment, we use a robotic system that utilizes an Xbox Kinect as a vision sensor, a more cluttered environment, and a pick-and-place task that we segment in a way similar to the first experiment. Results from the second experiment indicate that allowing the user to make fine tuned adjustments to the position and orientation of the robotic hand can improve task success in high noise situations in which the autonomous robotic system might otherwise fail. The experimental setups and procedures used in this thesis can be generalized and used to guide similar analysis of human impact in other human-in-the-loop systems performing other tasks.

To facilitate the emerging need for remote robot training and reach back, this thesis describes a system that allows for convenient web browser based robot operation over the Internet, while providing the means for recording and playback of all video, data and user actions. Training of first responder personnel on rescue robots is hindered by the fact that these devices are very expensive and are only affordable by a few specialized organizations that make them available by request at the time of a disaster. The system described in this thesis will allow first responders to practice on the robots without having to be physically present at same location. Having these capabilities of remote presence, the system can also be used in a real world response to transmit robot video and data to persons not present at the site of the incident, such as structural engineers or medical doctors. The recording capability will be used as an aid during training and to help resolve accountability issues in the real world scenario. Similar demands in the area of network video surveillance are met by the use of a network DVR that records and relays video and controls between IP cameras and Internet clients. The server implemented in this thesis is unique in that it extends these capabilities to include data from various robot sensors. All of the mentioned above video, data, and controls are combined into a convenient web browser based graphical user interface. The server was implemented and tested using rescue robots, but could be tailored to any other distributed robot architecture where reliable and convenient web browser based robot operation over the Internet is desired. System testing validated server capabilities of remote multi user robot operation, as well as its unique ability to store and play back external camera view along with robot video and data, to help with situation awareness. Conclusions drawn from the experiments indicate that this system can indeed be used for Internet robot training, as well as for other robotics research such as bandwidth regulation techniques or human-robot interaction studies by non computer science researchers who do not have physical access to robots.

The aim of this thesis is to investigate the development and control of electro-pneumatic actuators in a haptic teleoperation system. For controlling the mass flow rate of such actuators, two types of valve technology are sudied, i.e. solenoid (on/off) valve and proportional servovalve. The servovalves have found widespread applications in which high accuracy of force/position control are needed. They are however typically expensive due to the requirements of high-precision manufacturing. Therefore, the low-cost solenoid valves can be an alternative to the servovalves for achieving acceptable-performance pneumatic control. Generally, the highly nonlinear of the pneumatic actuator is heightened when it uses on/off solenoid valves instead of servovalves. In this case, precise control is challenging due to the discrete-input nature of the system. Our first objective is to demonstrate that it is possible to design an acceptable performance teleoperation system using master-slave robots that have pneumatic actuators equipped with only inexpensive on/off solenoid valves. To control efficiently the switching valves, several control approaches have been proposed, namely pulse width modulation (PWM), hybrid algorithm, and sliding mode control. A hybrid control theory, which includes more switching control modes, than PWM, allows to reduce the chattering problem and improve the energy consumption of the valves. Another strategy (i.e. a sliding mode control), which does not depend on the pneumatic model, is proposed. This control stratgy allows to perform not only the transparent analysis but also the stability analysis. In order to improve the dynamic performance and reduce the chattering problem in solenoid valve actuated pneumatic teleoperation systems, a five-mode sliding control scheme has been used, which can be considered as an extension of the three-mode sliding controller. Our study demonstrates that by increasing the number of possible control actions for the valves, we can reduce the valves' switching activities, hence improving the valve's life times at no cost to teleoperation transparency. The second objective of the thesis involves in implementing the proportional servovalves on the pneumatic teleoperation system. A comparison related to the teleoperation performance between an on/off valve and a servovalve is carried out. In experiments, it is observed that with the bilateral teleoperation architecture employing solenoid valves or servovalves, satisfactory force and position tracking between the master and the slave is obtained. In bilateral teleoperation control, force sensors are often omitted to save cost and to lessen weight and volume. Therefore, another aspect of our work consists in using observers for an estimation of operator and environment forces. Experimental results show that acceptable teleoperation transparency based on a simple Nicosia observer and a tangent linear control approach can be achieved.