Visual servoing (VS) is an automatic control technique which uses vision feedback to control the robot motion. Eye-in-hand VS systems, with the vision sensor mounted directly on the robot end-effector have received significant attention, in particular for the task of steering the vision sensor (usually a camera) from the present position to the desired one identified by image features shown in advance. The servo uses the difference between the present and the desired views (shown a priori) of some objects to develop real-time driving signals. This approach is also known as “teach-by-showing” method. To accomplish such a task, many constraints and limits are required such as camera field of view (FOV), robot joint limits, collision and occlusion avoidance, and etc. Path-planning technologies, as one branch of high-level control strategies, are explored in this thesis to impose these constraints for VS tasks with respect to different types of cameras and objects. First, a VS path-planning strategy is proposed for a class of cameras that include conventional perspective cameras, fisheye cameras, and catadioptric systems. These cameras are described by adopting a unified mathematical model and the strategy consists of designing image trajectories that allow the camera to reach the desired position while satisfying the camera FOV limit and the end-effector collision avoidance. To this end, the proposed strategy introduces the projection of the available image features onto a virtual plane and the computation of a feasible camera trajectory through polynomial programming. The computed image trajectory is hence tracked by an image-based visual servoing (IBVS) controller. Experimental results with a fisheye camera mounted on a 6-degree-of-freedom (6-DoF) robot arm illustrate the proposed strategy. Second, this thesis proposes a path-planning strategy for visual servoing with image moments, in the case of which the observed features are not restrained to points. Image moments of some solid objects such as circle, sphere, and etc. are more intuitive features than the dominant feature points in VS applications. The problem consists of planning a trajectory in order to ensure the convergence of the robot end-effector to the desired position while satisfying workspace (Cartesian space) constraints of the robot end-effector and visibility constraints of these solid objects, in particular including collision and occlusion avoidance. A solution based on polynomial parametrization is proposed and validated by some simulation and experiment results. Third, constrained optimization is combined with robot teach-by-demonstration to address simultaneously visibility constraint, joint limits and whole-arm collisions for robust vision-based control of a robot manipulator. User demonstration data generates safe regions for robot motion with respect to joint limits and potential whole-arm collisions. Constrained optimization uses these safe regions to generate new feasible trajectories under visibility constraint that achieve the desired view of the target (e.g., a pre-grasping location) in new, undemonstrated locations. To fulfill these requirements, camera trajectories that traverse a set of selected control points are modeled and optimized using either quintic Hermite splines or polynomials with C2 continuity. Experiments with a 7-DoF articulated arm validate the proposed method. / published_or_final_version / Electrical and Electronic Engineering / Doctoral / Doctor of Philosophy

published_or_final_version / abstract / toc / Electrical and Electronic Engineering / Master / Master of Philosophy

Designers of automatic target recognition algorithms (ATRs) need to compare the performance of different ATRs on a wide variety of imagery. The task would be greatly facilitated by an image complexity metric that correlates with the performance of a large number of ATRs. The ideal metric is independent of any specific ATR and does not require advance knowledge of true targets in the image. No currently used metric meets both these criteria. Complete independence of ATRs and prior target information is neither possible nor desirable since the metric must correlate with ATR performance. An image complexity metric that derives from the common characteristics of a large set of ATRs and the attributes of typical targets may be sufficiently general for ATR comparison. Many real-time, tactical ATRs operate on forward looking infrared (FLIR) imagery and identify, as potential targets, image regions of a specific size that are highly discernible by virtue of their contrast and edge strength. For such ATRs, an image complexity metric could be based on measurements of the mutual discernibility of image regions on various scales. This paper: (1) reviews ATR algorithms in the public domain literature and investigates the common characteristics of both the algorithms and the imagery on which they operate; (2) shows that complexity measurement requires a complete segmentation of the image based on these commonalities; (3) presents a new method of scale-specific image segmentation that uses the mask-driven close-open transform, a novel implementation of a morphological operator; (4) reviews edge detection for discernibility measurement; (5) surveys image complexity metrics in the current literature and discusses their limitations; (6) proposes a new local feature discernibility metric based on relative contrast and edge strength; (7) derives a new global image complexity metric based on the probability distribution of local metrics; (8) compares the metric to the output of a specific ATR; and (9) makes suggestions for further work.

No description available.

No description available.

No description available.

No description available.

Pose estimation is a common problem in computer vision. The pose is the combination of the position and orientation of a particular object relative to some reference coordinate system. The pose estimation problem involves determining the pose of an object from one or multiple images of the object. This problem often arises in the area of robotics. It is necessary to determine the pose of an object before it can be manipulated by the robot. In particular, this research focuses on pose estimation for initialization of position-based visual servoing. A closely related problem is the correspondence problem. This is the problem of finding a set of features from the image of an object that can be identified as the same feature from a model of the object. Solving for pose without known corre- spondence is also refered to as the simultaneous pose and correspondence problem, and it is a lot more difficult than solving for pose with known correspondence. This thesis explores a number of methods to solve the simultaneous pose and correspondence problem, with focuses on a method called SoftPOSIT. It uses the idea that the pose is easily determined if correspondence is known. It first produces an initial guess of the pose and uses it to determine a correspondence. With the correspondence, it determines a new pose. This new pose is assumed to be a better estimate, thus a better correspondence can be determined. The process is repeated until the algorithm converges to a correspondence pose estimate. If this pose estimate is not good enough, the algorithm is restarted with a new initial guess. An improvement is made to this algorithm. An early termination condition is added to detect conditions where the algorithm is unlikely to converge towards a good pose. This leads to an reduction in the runtime by as much as 50% and improvement in the success rate of the algorithm by approximately 5%. The proposed solution is tested and compared with the RANSAC method and simulated annealing in a simulation environment. It is shown that the proposed solution has the potential for use in commercial environments for pose estimation.

Suturing in laparoscopic surgery is a challenging and time-consuming task that presents haptic, motor and spatial constraints for the surgeon. As a result, there is variability in surgical outcome when performing basic suturing tasks such as knot tying, stitching and tissue dissection (as large as 50\%). This goal of this thesis is to develop a standardized, proof-of-concept, automated robotic suturing system that performs a side-to-side anastomosis with image guidance and dynamic trajectory control. A passive alignment tool is created for rigidly constraining needle pose, and robust computer vision algorithms are used to track surface features and the suture needle. A robotic system integrates these components to autonomously pass a curved suture needle through sequential loops in a tissue pad phantom.

Fetal Alcohol Spectrum Disorder (FASD) is the most common form of preventable mental retardation worldwide. This condition affects children whose mothers excessively consume alcohol whilst pregnant. FASD can be identied by physical and mental defects, such as stunted growth, facial deformities, cognitive impairment, and behavioural abnormalities. Magnetic Resonance Imaging provides a non-invasive means to study the neural correlates of FASD. One such approach aims to detect brain abnormalities through an assessment of volume and shape of sub-cortical structures on high-resolution MR images. Two brain structures of interest are the Caudate Nucleus and Hippocampus. Manual segmentation of these structures is time-consuming and subjective. We therefore present a method for automatically segmenting the Caudate Nucleus and Hippocampus from high-resolution MR images captured as part of an ongoing study into the neural correlates of FASD. Our method incorporates an Active Shape Model (ASM), which is used to learn shape variation from manually segmented training data. A discrete Geometrically Deformable Model (GDM) is rst deformed to t the relevant structure in each training set. The vertices belonging to each GDM are then used as 3D landmark points - effectively generating point correspondence between training models. An ASM is then created from the landmark points. This ASM is only able to deform to t structures with similar shape to those found in the training data. There are many variations of the standard ASM technique - each suited to the segmentation of data with particular characteristics. Experiments were conducted on the image search phase of ASM segmentation, in order to find the technique best suited to segmentation of the research data. Various popular image search techniques were tested, including an edge detection method and a method based on grey prole Mahalanobis distance measurement. A heuristic image search method, especially designed to target Caudate Nuclei and Hippocampi, was also developed and tested. This method was extended to include multisampling of voxel proles. ASM segmentation quality was evaluated according to various quantitative metrics, including: overlap, false positives, false negatives, mean squared distance and Hausdorff distance. Results show that ASMs that use the heuristic image search technique, without multisampling, produce the most accurate segmentations. Mean overlap for segmentation of the various target structures ranged from 0.76 to 0.82. Mean squared distance ranged from 0.72 to 0.76 - indicating sub-1mm accuracy, on average. Mean Hausdorff distance ranged from 2:7mm to 3:1mm. An ASM constructed using our heuristic technique will enable researchers to quickly, reliably, and automatically segment test data for use in the FASD study - thereby facilitating a better understanding of the eects of this unfortunate condition.