A technique for real time object tracking in a mobile computing environment and its application to video see-through Augmented Reality (AR) has been designed, verified through simulation, and implemented and validated on a mobile computing device. Using position based visual position and orientation (POSE) methods and the Extended Kalman Filter (EKF), it is shown how this technique lends itself to be flexible to tracking multiple objects and multiple object models using a single monocular camera on different mobile computing devices. Using the monocular camera of the mobile computing device, feature points of the object(s) are located through image processing on the display. The relative position and orientation between the device and the object(s) is determined recursively by an EKF process. Once the relative position and orientation is determined for each object, three dimensional AR image(s) are rendered onto the display as if the device is looking at the virtual object(s) in the real world. This application and the framework presented could be used in the future to overlay additional informational onto displays in mobile computing devices. Example applications include robotic aided surgery where animations could be overlaid to assist the surgeon, in training applications that could aid in operation of equipment or in search and rescue operations where critical information such as floor plans and directions could be virtually placed onto the display. Current approaches in the field of real time object tracking are discussed along with the methods used for video see-through AR applications on mobile computing devices. The mathematical framework for the real time object tracking and video see-through AR rendering is discussed in detail along with some consideration to extension to the handling of multiple AR objects. A physical implementation for a mobile computing device is proposed detailing the algorithmic approach along with design decisions. The real time object tracking and video see-through AR system proposed is verified through simulation and details around the accuracy, robustness, constraints, and an extension to multiple object tracking are presented. The system is then validated using a ground truth measurement system and the accuracy, robustness, and its limitations are reviewed. A detailed validation analysis is also presented showing the feasibility of extending this approach to multiple objects. Finally conclusions from this research are presented based on the findings of this work and further areas of study are proposed.

In this work, the Traveling Salesman Problem (TSP) is discussed. The Hybrid Genetic Algorithm for solving the TSP is presented. The traveling salesman problem is formulated as follows: given matrix D=(dij)nxn of distances between n objects and the set P of permutations of the integers from 1 to n, find a permutation p=(p(1), p(2), ..., p(n)) P that minimizes. Many heuristic algorithms can be applied for the TSP. Recently, genetic algorithms (GAs) are among the advanced heuristic techniques for the combinatorial problems, like the TSP. genetic algorithms are based on the biological process of natural selection. The original concepts of GAs were developed in 1970s. Many simulations have demonstrated the efficiency of GAs on different optimization problems, among them, bin–packing, generalized assignment problem, graph partitioning, job–shop scheduling problem, set covering problem, vehicle routing. One of the main operators in GAs is the crossover (i.e. solution recombination). This operator plays a very important role by constructing competitive GAs. In this work, we investigate several crossover operators for the TSP, among them, CX (cycle crossover), PMX (partialy mapped crossover), POS (position based crossover), ER (edge recombination crossover), edge-NN (edge recombination crossover, nearest neighbour) and AP (alternating-positions crossover). Comparison of these crossover operators was performed. The results show high efficiency of the edge-NN, ER and PMX crossovers.

A technique for real time object tracking in a mobile computing environment and its application to video see-through Augmented Reality (AR) has been designed, verified through simulation, and implemented and validated on a mobile computing device. Using position based visual position and orientation (POSE) methods and the Extended Kalman Filter (EKF), it is shown how this technique lends itself to be flexible to tracking multiple objects and multiple object models using a single monocular camera on different mobile computing devices. Using the monocular camera of the mobile computing device, feature points of the object(s) are located through image processing on the display. The relative position and orientation between the device and the object(s) is determined recursively by an EKF process. Once the relative position and orientation is determined for each object, three dimensional AR image(s) are rendered onto the display as if the device is looking at the virtual object(s) in the real world. This application and the framework presented could be used in the future to overlay additional informational onto displays in mobile computing devices. Example applications include robotic aided surgery where animations could be overlaid to assist the surgeon, in training applications that could aid in operation of equipment or in search and rescue operations where critical information such as floor plans and directions could be virtually placed onto the display. Current approaches in the field of real time object tracking are discussed along with the methods used for video see-through AR applications on mobile computing devices. The mathematical framework for the real time object tracking and video see-through AR rendering is discussed in detail along with some consideration to extension to the handling of multiple AR objects. A physical implementation for a mobile computing device is proposed detailing the algorithmic approach along with design decisions. The real time object tracking and video see-through AR system proposed is verified through simulation and details around the accuracy, robustness, constraints, and an extension to multiple object tracking are presented. The system is then validated using a ground truth measurement system and the accuracy, robustness, and its limitations are reviewed. A detailed validation analysis is also presented showing the feasibility of extending this approach to multiple objects. Finally conclusions from this research are presented based on the findings of this work and further areas of study are proposed.

Thesis (MScEng (Electrical and Electronic Engineering))--University of Stellenbosch, 2011. / ENGLISH ABSTRACT: This thesis presents work completed on the design of control and vision components for use in a monocular vision-based human-following robot. The use of vision in a controller feedback loop is referred to as vision-based or visual servo control. Typically, visual servo techniques can be categorised into imagebased visual servoing and position-based visual servoing. This thesis discusses each of these approaches, and argues that a position-based visual servo control approach is more suited to human following. A position-based visual servo strategy consists of three distinct phases: target recognition, target pose estimation and controller calculations. The thesis discusses approaches to each of these phases in detail, and presents a complete, functioning system combining these approaches for the purposes of human following. Traditional approaches to human following typically involve a controller that causes platforms to navigate directly towards targets, but this work argues that better following performance can be obtained through the use of a controller that incorporates target orientation information. Although a purely direction-based controller, aiming to minimise both orientation and translation errors, suffers from various limitations, this thesis shows that a hybrid, gain-scheduling combination of two traditional controllers offers better targetfollowing performance than its components. In the case of human following the inclusion of target orientation information requires that a definition and means of estimating a human’s orientation be available. This work presents a human orientation measure and experimental results to show that it is suitable for the purposes of wheeled platform control. Results of human following using the proposed hybrid, gain-scheduling controller incorporating this measure are presented to confirm this. / AFRIKAANSE OPSOMMING: Die ontwerp van ’n visiestelsel en beheer-komponente van ’n enkel-kamera robot vir die volging van mense word hier aangebied. Die gebruik van visuele terugvoer in die beheerlus word visie-gebaseerde of visuele servobeheer genoem. Visuele servobeheer tegnieke kan tipies onderskei word tussen beeld-gebaseerde servobeheer en posisie-gebaseerde visuele servobeheer. Altwee benaderings word hier bespreek. Die posisie-gebaseerde benadering word aanbeveel vir die volging van mense. Die posisie-gebaseerde servobeheertegniek bestaan uit drie duidelike fases: teiken herkenning, teiken oriëntasie bepaling en die beheerder berekeninge. Benaderings tot elk van hierdie fases word hier in detail bespreek. Dan word ’n volledige funksionele stelsel aangebied wat hierdie fases saamvoeg sodat mense gevolg kan word. Meer tradisionele benaderings tot die volging van mense gebruik tipies ’n beheerder wat die platvorm direk laat navigeer na die teikens, maar hier word geargumenteer dat beter werkverrigting verkry kan word deur ’n beheerder wat die teiken oriëntasie inligting ook gebruik. ’n Suiwer rigting-gebaseerde beheerder, wat beide oriëntasie en translasie foute minimeer, is onderhewig aan verskeie beperkings. Hier word egter aangetoon dat ’n hibriede, aanwinsskedulerende kombinasie van die twee tradisionele beheerders beter teikenvolging werkverrigting bied as die onderliggende twee tegnieke. In die geval van die volging van mense vereis die insluiting van teiken oriëntasie inligting dat ’n definisie van die persoon se oriëntasie beskikbaar is en dat dit geskat kan word. ’n Oriëntasie maatstaf vir mense word hier aangebied en dit word eksperimenteel getoon dat dit geskik is om ’n platvorm met wiele te beheer. Die resultate van die volging van mense wat die voorgestelde hibriede, aanwins-skedulerende beheerder gebruik, met hierdie maatstaf, word ter ondersteuning aangebied.

abstract: Toward the ambitious long-term goal of a fleet of cooperating Flexible Autonomous Machines operating in an uncertain Environment (FAME), this thesis addresses various control objectives for ground vehicles. There are two main objectives within this thesis, first is the use of visual information to control a Differential-Drive Thunder Tumbler (DDTT) mobile robot and second is the solution to a minimum time optimal control problem for the robot around a racetrack. One method to do the first objective is by using the Position Based Visual Servoing (PBVS) approach in which a camera looks at a target and the position of the target with respect to the camera is estimated; once this is done the robot can drive towards a desired position (x_ref, z_ref). Another method is called Image Based Visual Servoing (IBVS), in which the pixel coordinates (u,v) of markers/dots placed on an object are driven towards the desired pixel coordinates (u_ref, v_ref) of the corresponding markers. By doing this, the mobile robot gets closer to a desired pose (x_ref, z_ref, theta_ref). For the second objective, a camera-based and noncamera-based (v,theta) cruise-control systems are used for the solution of the minimum time problem. To set up the minimum time problem, optimal control theory is used. Then a direct method is implemented by discretizing states and controls of the system. Finally, the solution is obtained by modeling the problem in AMPL and submitting to the nonlinear optimization solver KNITRO. Simulation and experimental results are presented. The DDTT-vehicle used within this thesis has different components as summarized below: (1) magnetic wheel-encoders/IMU for inner-loop speed-control and outer-loop directional control, (2) Arduino Uno microcontroller-board for encoder-based inner-loop speed-control and encoder-IMU-based outer-loop cruise-directional-control, (3) Arduino motor-shield for inner-loop speed-control, (4) Raspberry Pi II computer-board for outer-loop vision-based cruise-position-directional-control, (5) Raspberry Pi 5MP camera for outer-loop cruise-position-directional control. Hardware demonstrations shown in this thesis are summarized: (1) PBVS without pan camera, (2) PBVS with pan camera, (3) IBVS with 1 marker/dot, (4) IBVS with 2 markers, (5) IBVS with 3 markers, (6) camera and (7) noncamera-based (v,theta) cruise control system for the minimum time problem. / Dissertation/Thesis / Masters Thesis Electrical Engineering 2016

Using an explicit integration method in physically based animations has many advantages including conceptual and computational simplicity, however, it re- quires small time steps to ensure low numerical instability. Simulations with large numbers of individually interacting components such as cloth, hair, and fluid models, are limited by the sections of particles most susceptible to error. This results in the need for smaller time steps than required for the majority of the system. These sections can be diverse and dynamic, quickly changing in size and location based on forces in the system. Identifying and handling these trou- blesome sections could allow for a larger time step to be selected, while preventing a breakdown in the simulation. This thesis presents Smarticles (smart particles), a method of individually de- tecting particles exhibiting signs of instability and stabilizing them with minimal adverse effects to visual accuracy. As a result, higher levels of error introduced from large time steps can be tolerated with minimal overhead. Two separate approaches to Smarticles were implemented. They attempt to find oscillating particles by analyzing a particle’s (1) past behavior and (2) behavior with re- spect to its neighbors along a strand. Both versions of Smarticles attempt to correct unstable particles using velocity dampening. Smarticles was applied to a two dimensional hair simulation modeled as a continuum using smooth particle hydrodynamic. Hair strands are formed by linking particles together using one of two methods: position based dynamics or mass-spring forces. Both versions of Smarticles, as well as a control of normal particles, were directly compared and evaluated based on stability and visual fluidity. Hair particles were exposed to various forms of external forces under increasing time step lengths. Testing showed that both versions of Smarticles working together allowed an average increase of 18.62% in the time step length for hair linked with position based dynamics. In addition, Smarticles was able to significantly reduce visible instability at even larger time steps. While these results suggest Smarticles is successful, the method used to correct particle instability may jeopardize other important aspects of the simulation. A more accurate correction method would likely need to be developed to make Smarticles an advantageous method.

Chaque année en Europe, 1.300.000 accidents de la route ont comme conséquence 1.700.000 blessés. Le coût financier d’accidents de la route est évalué à 160 milliards d’euros (approximativement le même coût aux Etats-Unis). VANET (Vehicular Ad-hoc NETwork) est une des technologies clés qui peut permettre de réduire d’une façon significative le nombre d’accidents de la route (e.g. message d’urgence signalant la présence d’un obstacle ou d’un véhicule en cas de brouillard). En plus de l’amélioration de la sécurité et du confort des conducteurs et des passagers, VANET peut contribuer à beaucoup d’applications potentielles telles que la prévision et la détection d’embouteillages, la gestion d’infrastructure de système de transport urbain (e.g. système de transport intelligent multimodal) etc. Dans cette thèse, je présenterai un système embarqué dédié à la communication inter-véhicule particulièrement pour les applications sécuritaires de passagers et de conducteurs. Nos efforts de recherche et de développement sont centrés sur deux principaux objectifs : minimiser le temps de latence intra-noeud et le délai de communication inter-véhicule en prenant en compte le changement dynamique du VANET. De ce fait pour atteindre ces objectifs, des nouvelles approches (e.g. inter-couche ‘Cross-layering’) ont été explorées pour respecter les contraintes de ressource (QoS, mémoire, CPU et énergie de la communication inter-véhicule) d’un système embarqué à faible coût. Le système de communication embarqué proposé comporte deux composants logiciels principaux : un protocole de communication dénommé CIVIC (Communication Inter Véhicule Intelligente et Coopérative) et un système d’exploitation temps réel appelé HEROS (Hybrid Event-driven and Real-time multitasking Operating System). CIVIC est un protocole de communication géographique à faible consommation énergétique et à faible temps de latence (délai de communication). HEROS gère contextuellement l’ensemble du système (matériel et logiciel) en minimisant le temps de latence et la consommation des ressources (CPU et mémoire). En outre, le protocole de communication CIVIC est équipé d’un système de localisation LCD-GPS (Low Cost Differential GPS). Pour tester et valider les différentes techniques et théories, la plateforme matérielle LiveNode (LImos Versatile Embedded wireless sensor NODE) a été utilisée. En effet, la plateforme LiveNode permet de développer et de prototyper rapidement des applications dans différents domaines. Le protocole de communication CIVIC est basé sur la technique de ‘broadcast’ à un saut ; de ce fait il est indépendant de la spécificité du réseau. Pour les expérimentations, seule la norme d’IEEE 802.15.4 (ZigBee) a été choisie comme médium d’accès sans fil. Il est à noter que le médium d’accès sans fil ZigBee a été adopté comme le médium standard pour les réseaux de capteurs sans fil (RCSFs) et le standard 6LoWPAN ; car il est peu coûteux et peu gourmand en énergie. Bien que le protocole de communication à l’origine soit conçu pour répondre aux exigences de VANET, ses domaines d’application ne sont pas limités à VANET. Par exemple il a été utilisé dans différents projets tels que MOBI+ (système de transport urbain intelligent) et NeT-ADDED (projet européen FP6 : agriculture de précision). Les VANETs et les RCSFs sont les réseaux fortement dynamiques, mais les causes de changement topologique de réseau sont différentes : dans le réseau VANET, il est dû à la mobilité des véhicules, et dans le RCSF, il est dû aux pannes des noeuds sans fil. Il est à noter que le VANET et le RCSF sont généralement considérés comme un sous-ensemble du réseau MANET (réseau ad-hoc mobile). Cependant, ils sont réellement tout à fait différents du MANET classique, et leurs similitudes et différences seront expliquées en détail dans la thèse. La contribution principale de mes travaux est le protocole CIVIC, qui échange des messages en basant sur l’information géographique des noeuds (position). (...) / Each year in Europe, 1,300,000 vehicle accidents result in 1,700,000 personal injuries. The financial cost of vehicle accidents is evaluated at 160 billion Euros (approximately the same cost in the USA). VANET (Vehicular Ad-Hoc NETwork) is a key technology that can enable hazard alarming applications to reduce the accident number. In addition to improve the safety for drivers and passengers, VANET can contribute to many potential applications such as detecting and predicting traffic jams, auto-optimizing the traffic flow, and helping disabled passengers to access public transports.This thesis will present an embedded communication system dedicated to VANET especially for the safety-related applications. Our design mainly tries to achieve two requirements: as one can imagine, the embedded communication system for VANET requires extra effort to deal with the highly dynamic network topology caused by moving vehicles, thus to shorten the intra-node system latency and inter-node network delay is essential requirement for such embedded communication system. Besides, a fundamental requirement for any practical embedded system is resource-awareness. Although the embedded communication system on vehicles may gain better hardware supports, the characteristics of embedded hardware still have to cope with resource constraints in terms of QoS, memory, CPU and energy. The embedded communication system involves two major software components: a routing protocol called CIVIC (Communication Inter Véhicule Intelligente et Coopérative) and an embedded operating system called HEROS (Hybrid Event-driven and Real-time multitasking Operating System). The former is a quick reaction and low resource consumption geographic protocol for inter-vehicle message transmissions; and the latter controls the whole system and assures intra-node resource awareness. In addition, the system can use a localization software solution called LCD-GPS (Low Cost Differential GPS) to improve the accuracy of locations. The hardware platform is LiveNode (LImos Versatile Embedded wireless sensor NODE), which is a versatile wireless sensor node enabling to implement rapidly a prototype for different application domains. The communication system is based on the one-hop broadcast, thus it does not have a strict limitation on network specification. For the experiments only, the IEEE 802.15.4 standard is chosen as the underlying wireless access medium. The standard is well known as a low-power consumption standard requiring low-cost devices. Notice that the IEEE 802.15.4 standard is also the wireless access medium of 6LoWPAN. Although the embedded communication system is originally designed to meet the requirements of VANET, but its application domains are not limited to VANET. For example, another network which can use the embedded communication system is WSN (Wireless Sensor Network). CIVIC was used to implement different real-world projects such MOBI+ (intelligent urban transportation system) and EU-FP6 NeT-ADDED (precision agriculture). Both VANET and WSN are highly dynamic networks, but the causes of changing network topology are different: the former is because of the high-mobility feature of vehicles, and the latter is because of the fault of wireless sensors. Note that, although VANET and WSN are both commonly considered as the subset of MANET (Mobile Ad-hoc NETwork), they are actually quite different from the classical MANET, and the similarities and differences will be further explained in the thesis. The major contribution of my works relates to the CIVIC protocol, which routes messages based on the geographic information. The related works of the thesis will focus on the geographic routing techniques, problems and solutions, but other related techniques will also be addressed. Note that, although some related projects were investigated but their implementation and experiment aspects were not detailed. (...)