The WinCandide application, a platform for face tracking and model based coding, had become out of date and needed to be upgraded. This report is based on the work of investigating possible open source GUIs and computer vision tool kits that could replace the old ones that are unsupported. Multi platform GUIs are of special interest.

Most people are familiar with the BRIO labyrinth game and the challenge of guiding the ball through the maze. The goal of this project was to use this game to create a platform for evaluation of control algorithms. The platform was used to evaluate a few different controlling algorithms, both traditional automatic control algorithms as well as algorithms based on online incremental learning. The game was fitted with servo actuators for tilting the maze. A camera together with computer vision algorithms were used to estimate the state of the game. The evaluated controlling algorithm had the task of calculating a proper control signal, given the estimated state of the game. The evaluated learning systems used traditional control algorithms to provide initial training data. After initial training, the systems learned from their own actions and after a while they outperformed the controller used to provide initial training.

Foreground segmentation is a common first step in tracking and surveillance applications.  The purpose of foreground segmentation is to provide later stages of image processing with an indication of where interesting data can be found.  This thesis is an investigation of how foreground segmentation can be performed in two contexts: as a pre-step to trajectory tracking and as a pre-step in indoor surveillance applications. Three methods are selected and detailed: a single Gaussian method, a Gaussian mixture model method, and a codebook method.  Experiments are then performed on typical input video using the methods.  It is concluded that the Gaussian mixture model produces the output which yields the best trajectories when used as input to the trajectory tracker.  An extension is proposed to the Gaussian mixture model which reduces shadow, improving the performance of foreground segmentation in the surveillance context.

The unit for information security and IT architecture at The Swedish Defence Research Agency (FOI) conducts work with a cyber range called CRATE (Cyber Range and Training Environment). Currently, simulation of user activity involves scripts inside the simulated network. This solution is not ideal because of the traces it leaves in the system and the general lack of standardised GUI API between different operating systems. FOI are interested in testing the use of artificial user agent located outside the virtual environment using computer vision and the virtualisation API to execute actions and extract information from the system. This paper focuses on analysing the reliability of template matching, a computer vision algorithm used to localise objects in images using already identified images of said object as templates. The analysis will evaluate both the reliability of localising objects and the algorithms ability to correctly identify if an object is present in the virtual environment. Analysis of template matching is performed by first creating a prototype of the agent's sensory system and then simulate scenarios which the agent might encounter. By simulating the environment, testing parameters can be manipulated and monitored in a reliable way. The parameters manipulated involves both the amount and type of image noise in the template and screenshot, the agent’s discrimination threshold for what constitutes a positive match, and information about the template such as template generality. This paper presents the performance and reliability of the agent in regards to what type of image noise affects the result, the amount of correctly identified objects given different discrimination thresholds, and computational time of template matching when different image filters are applied. Furthermore the best cases for each study are presented as comparison for the other results. In the end of the thesis we present how for screenshots with objects very similar to the templates used by the agent, template matching can result in a high degree of accuracy in both object localization and object identification and that a small reduction of similarity between template and screenshot to reduce the agent's ability to reliably identifying specific objects in the environment.

Oryx Simulations produce simulators for mining and forestry machinery used for educational and promotional purposes. The simulators use motion platforms to reflect how the vehicle moves within the simulator. This platform tilts and accelerates the driver in order to enhance the experience. Previously a classical washout filter algorithm has been used to control the platform which leaves something to be desired regarding how well it reflects the vehicles movement, how easy it is to tune and how it handles the limits of the platform. This thesis aims to produce a model that accurately reflects angles, velocities and accelerations while in the mean time respecting the limits of the platform. In addition to this the developed model should be easy to modify and tune. This is achieved using so-called Model Predictive Control which achieves the wanted behaviour by predicting how the platform will move based on its current state while implementing the constraints of the platform directly into the model. Since all of the parameters in the model are actual physical quantities, this makes the model easier to tune. A key component in this solution is the so-called tilt coordination which consists of substituting a lateral/longitudinal acceleration with the acceleration of gravity by tilting the driver. Constructing and implementing this model in Matlab we verify it by using data extracted from the simulator environment. We see that the parameters consisting of angles, rotational velocities and linear accelerations are tracked very well while respecting the constraints for the platform, constraints that can be easily changed to fit the current simulator.We also see that the model successfully implements tilt coordination into the behaviour of the platform. This model performs extraordinarily well in theory, what remains is to implement this to the motion platform and fine-tune it. / Oryx Simulations tillverkar simulatorer i huvudsak för gruv- och skogsindustrinvilket används i utbildnings- och marknadsföringssyfte. Simulatorerna använder en röorelseplattform för att spegla hur fordonet i simulatormiljön rör sig. Denna plattform lutar och accelererar föraren för att förstarka upplevelsen. Tidigare har ett så kallat klassiskt washout-filter använts för att kontrollera plattformen som lämnar en del i övrigt att onska vad gäller hur väl fordonets rörelser speglas, hur lätt det ar att justera samt hur det hanterar plattformens begränsningar. Detta projekt ämnar producera en modell som väl speglar vinklar,hastigheter och accelerationer samtidigt som den respekterar plattformens gränser. I tillägg till detta bör modellen vara enkel att modifiera och justera. Detta uppnås genom så kallad Model Predictive Control som förutsager hur plattformen kommer röra sig utifrån dess aktuella tillstånd samtidigt som den respekterar de tvång som finns på plattformen direkt i modellen. Då alla parametrar i modellen är faktiska fysiska kvantiteter blir modellen märkbart lättare att justera. En viktig komponent i denna lösning är så kallad tilt coordination vilket består i att substituera lateral/longtudinell acceleration med en komposant av tyngdaccelerationen genom att luta föraren. Denna modell konstrueras och implementeras i Matlab och verifieras genom att använda extraherat data från den simulerade miljön. Vi kan se att parametrarna som består av vinklar, rotationella hastigheter och linjära accelerationer speglas väldigt väl, samtidigt som tvången på plattformen respekteras. Dessa tvång kan enkelt modieras for att passa den aktuella simulatorn. Vi ser även att modellen framgångsrikt implementerar tilt coordination i plattformens beteende. I teorin har denna modell väldigt bra prestanda; vad som kvarstår är att implementera den på en rörelseplattform och finjustera modellen.

Many methods have been developed for visual tracking of generic objects. The vast majority of these assume the world is two-dimensional, either ignoring the third dimension or only dealing with it indirectly. This causes difficulties for the tracker when the target approaches or moves away from the camera, is occluded or moves out of the camera frame. Unmanned aerial vehicles (UAVs) are increasingly used in civilian applications and some of these will undoubtedly carry tracking systems in the future. As they move around, these trackers will encounter both scale changes and occlusions. To improve the tracking performance in these cases, the third dimension should be taken into account. This thesis extends the capabilities of a 2D tracker to three dimensions, with the assumption that the target moves on a ground plane. The position of the tracker camera is established by matching the video it produces to a sparse point-cloud map built with off-the-shelf structure-from-motion software. A target is tracked with a generic 2D tracker and subsequently positioned on the ground. Should the target disappear from view, its motion on the ground is predicted. In combination, these simple techniques are shown to improve the robustness of a tracking system on a moving platform under target scale changes and occlusions.

The Next-Best-View (NBV) problem plays an important part in automatic 3D object reconstruction and exploration applications. This thesis presents a novel approach of ray-casting in Occupancy Grid Maps (OGM) in the context of solving the NBV problem in a 3D-exploration setting. The proposed approach utilizes the structure of an octree-based OGM to perform calculations of potential information gain. The computations are significantly faster than current methods, without decreasing mapping quality. Performance, both in terms of mapping quality, coverage and computational complexity, is experimentally verified through a comparison with existing state-of-the-art methods using high-resolution point cloud data generated using time-of-flight laser range scanners. Current methods for viewpoint ranking focus either heavily on mapping performance or computation speed. The results presented in this thesis indicate that the proposed method is able to achieve a mapping performance similar to the performance-oriented approaches while maintaining the same low computation speed as more approximative methods.

In this thesis a method is presented to provide a 3D visualization of the human kidney and surrounding tissue during kidney surgery. The method takes advantage of the high detail of 3D X-Ray Computed Tomography (CT) and the high time resolution of Ultrasonography (US). By extracting the geometry from a single preoperative CT scan and animating the kidney by tracking its position in real time US images, a 3D visualization of the surgical volume can be created. The first part of the project consisted of building an imaging phantom as a simplified model of the human body around the kidney. It consists of three parts: the shell part representing surrounding tissue, the kidney part representing the kidney soft tissue and a kidney stone part embedded in the kidney part. The shell and soft tissue kidney parts was cast with a mixture of the synthetic polymer Polyvinyl Alchohol (PVA) and water. The kidney stone part was cast with epoxy glue. All three parts where designed to look like human tissue in CT and US images. The method is a pipeline of stages that starts with acquiring the CT image as a 3D matrix of intensity values. This matrix is then segmented, resulting in separate polygonal 3D models for the three phantom parts. A scan of the model is then performed using US, producing a sequence of US images. A computer program extracts easily recognizable image feature points from the images in the sequence. Knowing the spatial position and orientation of a new US image in which these features can be found again allows the position of the kidney to be calculated. The presented method is realized as a proof of concept implementation of the pipeline. The implementation displays an interactive visualization where the kidney is positioned according to a user-selected US image scanned for image features. Using the proof of concept implementation as a guide, the accuracy of the proposed method is estimated to be bounded by the acquired image data. For high resolution CT and US images, the accuracy can be in the order of a few millimeters.

In this thesis project techniques for training a rapid image classifier that can recognize an object of a predefined type has been studied. Classifiers have been trained with the AdaBoost algorithm, with and without the use of Viola-Jones cascades. The use of Weight trimming in the classifier training has been evaluated and resulted in a significant speed up of the training, as well as improving the performance of the trained classifier. Different preprocessings of the images have also been tested, but resulted for the most part in worse performance for the classifiers when used individually. Several rectangle shaped Haar-like features including novel versions have been evaluated and the magnitude versions proved to be best at separating the image classes.

Automatic tracking of an object of interest in a video sequence is a task that has been much researched. Difficulties include varying scale of the object, rotation and object appearance changing over time, thus leading to tracking failures. Different tracking methods, such as short-term tracking often fail if the object steps out of the camera’s field of view, or changes shape rapidly. Also, small inaccuracies in the tracking method can accumulate over time, which can lead to tracking drift. Long-term tracking is also problematic, partly due to updating and degradation of the object model, leading to incorrectly classified and tracked objects. This master’s thesis implements a long-term tracking framework called Tracking-Learning-Detection which can learn and adapt, using so called P/N-learning, to changing object appearance over time, thus making it more robust to tracking failures. The framework consists of three parts; a tracking module which follows the object from frame to frame, a learning module that learns new appearances of the object, and a detection module which can detect learned appearances of the object and correct the tracking module if necessary. This tracking framework is evaluated on thermal infrared videos and the results are compared to the results obtained from videos captured within the visible spectrum. Several important differences between visual and thermal infrared tracking are presented, and the effect these have on the tracking performance is evaluated. In conclusion, the results are analyzed to evaluate which differences matter the most and how they affect tracking, and a number of different ways to improve the tracking are proposed.