This thesis proposes a new chroma keying method that can automatically detect background, foreground, and unknown regions. For background color detection, we use K-means clustering in color space to calculate the limited number of clusters of background colors. We use spatial information to clean the background regions and minimize the unknown regions. Our method only needs minimum inputs from user. For unknown regions, we implement the alpha matte based on Wang's robust matting algorithm, which is considered one of the best algorithms in the literature, if not the best. Wang's algorithm is based on modified random walk. We proposed a better color selection method, which improves matting results in the experiments. In the thesis, a detailed implementation of robust matting is provided. The experimental results demonstrate that our proposed method can handle images with one background color, images with gridded background, and images with difficult regions such as complex hair stripes and semi-transparent clothes.

This thesis proposes a new chroma keying method that can automatically detect background, foreground, and unknown regions. For background color detection, we use K-means clustering in color space to calculate the limited number of clusters of background colors. We use spatial information to clean the background regions and minimize the unknown regions. Our method only needs minimum inputs from user. For unknown regions, we implement the alpha matte based on Wang's robust matting algorithm, which is considered one of the best algorithms in the literature, if not the best. Wang's algorithm is based on modified random walk. We proposed a better color selection method, which improves matting results in the experiments. In the thesis, a detailed implementation of robust matting is provided. The experimental results demonstrate that our proposed method can handle images with one background color, images with gridded background, and images with difficult regions such as complex hair stripes and semi-transparent clothes.

In this thesis, the Recursive-Random Sample Consensus (R-RANSAC) multiple target tracking (MTT) algorithm is further developed and applied to video taken from static platforms. Development of R-RANSAC is primarily focused in three areas: data association, the ability to track maneuvering objects, and track management. The probabilistic data association (PDA) filter performs very well in the R-RANSAC framework and adds minimal computation cost over less sophisticated methods. The interacting multiple models (IMM) filter as well as higher-order linear models are incorporated into R-RANSAC to improve tracking of highly maneuverable targets. An effective track labeling system, a more intuitive track merging criteria, and other improvements were made to the track management system of R-RANSAC. R-RANSAC is shown to be a modular algorithm capable of incorporating the best features of competing MTT algorithms. A comprehensive comparison with the Gaussian mixture probability hypothesis density (GM-PHD) filter was conducted using pseudo-aerial videos of vehicles and pedestrians. R-RANSAC maintains superior track continuity, especially in cases of interacting and occluded targets, and has fewer missed detections when compared with the GM-PHD filter. The two algorithms perform similarly in terms of the number of false positives and tracking precision. The concept of a feedback loop between the tracker and sensor processing modules is extensively explored; the output tracks from R-RANSAC are used to inform how video processing is performed. We are able to indefinitely detect stationary objects by zeroing out the background update rate of target-associated pixels in a Gaussian mixture models (GMM) foreground detector. False positive foreground detections are eliminated with a minimum blob area threshold, a ghost suppression algorithm, and judicious tuning of the R-RANSAC parameters. The ability to detect stationary targets also allows R-RANSAC to be applied to a class of problems known as stationary object detection. Additionally, moving camera foreground detection techniques are applied to the static camera case in order to produce measurements with a velocity component; this is accomplished by using sequential-RANSAC to cluster optical flow vectors of FAST feature pairs. This further improves R-RANSAC's track continuity, especially with interacting targets. Finally, a hybrid algorithm composed of R-RANSAC and the Sequence Model (SM), a machine learner, is presented. The SM learns sequences of target locations and is able to assist in data association once properly trained. In simulation, we demonstrate the SM's ability to significantly improve tracking performance in situations with infrequent measurement updates and a high proportion of clutter measurements.

Correct insertion of computer graphics into live-action broadcasts of outdoor sports requires precise knowledge of the foreground, i.e. players present in the scene. This thesis proposes a foreground detection and segmentation- framework with focus on real-time performance for 1080p resolution. A dataset consisting of four scenes; single-, multi-segment-, transcending-foreground and a light-witch scene all with dynamic backgrounds was constructed together with 26 ground-truths. Results show that the framework should run internally at 288p using GPU acceleration with geometrical nearest-neighbour-interpolation to attain real-time-capability. To maximize accuracy of the results, the framework uses two instances of OpenCV MOG2 in parallel on differently downsampled frames that are bitwise-joined to increase robustness. A set of morphological operations provides post-processing to get spatial coherence and a specific turf- consideration gives accurate contours. Thanks to additional camera- operator input, a crude distance-estimate lets foreground segments fade into background at a predetermined depth. The framework suffers from inaccurate segmentation during rapid light-switches, but recovers in a matter of seconds like the 'vanilla' MOG algorithm. For the specific scenes the framework provides excellent performance, especially considering the light-switch scene by comparison to the MOG-algorithm. For non-specific scenes of the 'BMC 2012' performance does not exceed the current state-of-the-art. / Korrekt placering av datorgrafik i video för tv-produktion kräver god känndedom om aktuell förgrund. Denna avhandling föreslår ett förgrundsdetektions- och segmenterings- ramverk med fokus på realtidsbearbetning av full-HD upplöst sport i utomhusmiljö. För utvärdering skapades ett dataset bestående av fyra scener; singel-, multisegment-, avlägsnande-förgrund och en ljusomväxlingsscen tillsammans med 26 referensförgrunder. För att erhålla realtidsbearbetning skall ramverket internt nyttja 288p upplösning med GPU acceleration och geometrisk närmaste-granne-interpolation. Resultaten visade att maximal noggranhet och ökad robusthet erhölls med två instanser av OpenCV MOG2 arbetandes parallellt på olikt nerskalade bilder för att därefter pixelvis förenas. För att erhålla sammanhängande förgrundssegment nyttjades morfologiska operationer på den binära sammansatta förgrunden vilket tillsammans med en specifik gräskantskorrektion ger precisa konturer. Tack vare givna kameraparametrar kan djupet till förgrundselementen uppskattas därmed låts de övergå till bakgrund för ett visst djupt. Ramverket lider av oprecis segmententering vid snabba ljusomväxlingar, men återhämtar sig när bakgrundsmodellen uppdaterats till de nya ljusförutsättningarna. För ovan nämnda specifika scener presterar ramverket utmärkt, speciellt med avseende på ljusomväxlingen, där prestandan är flerfaldigt bättre än den enskilda 'MOG'-metoden. För generella scener ur 'BMC 2012' datasetet presterar vår metod dock inte bättre än state-of-the-art.

In a world reliant on big industries to produce large quantities of more or less every product used, it is of utmost importance that the machines in such industries continue to run with minimum amounts of downtime. One way more and more providers of such industrial machines try to help their customers reduce downtime when a machine stops working or needs maintenance is through the use of remote guidance; a way of knowledge transfer from a technician to a regular employee that aims to allow the regular employee to be guided in real-time by a technician to solve the task himself, thus, not needing the technician to travel to the factory.  One technology that may come to mind if you were to create such a guiding system is to use augmented reality and maybe have a technician record his or her hand and in real-time overlay this upon the videostream the onsite employee sees. This is something available today, however, to separate the hand of the technician from the background can be a complex task especially if the background is not a single colour or the hand has a similar colour to the background. These kinds of limitations to the background separation are what this thesis aims to find a solution to. This thesis addresses this challenge by creating a test dataset containing five different background scenarios that are deemed representative of what a person who would use the product most likely can find something similar to without going out of their way. In each of the five scenarios, there are two videos taken, one with a white hand and one with a hand wearing a black glove. Then a machine learning model is trained in a couple of different configurations and tested on the test scenarios. The best of the models is later also tried to run directly on a mobile phone. It was found that the machine learning model achieved rather promising background segmentation and running on the computer with a dedicated GPU real-time performance was achievable. However, running on the mobile device the processing time proved to be not sufficient.

This thesis presents an innovative agricultural automation technique which focuses on addressing the significant perception challenges posed by occlusion within environments such as farms and greenhouses. Automated systems tasked with duties like pruning face considerable difficulties due to occlusion, complicating the accurate identification of plant features. To tackle these challenges, this work introduces a novel approach utilizing a LiDAR camera mounted on a robot arm, enhancing the system's ability to scan plants and dynamically adjust the arm's trajectory based on machine learning-derived segmentation. This adjustment significantly increases the detection area of plant features, improving identification accuracy and efficiency. Building on foreground isolation and instance segmentation, the thesis then presents an automated method for identifying optimal pruning points using best pose view images of indeterminate tomato plants. By integrating advanced image processing techniques, the proposed method ensures the pruning process by targeting branches with the highest leaf load. Experimental validation of the proposed method was conducted in a simulated environment, where it demonstrated substantially enhanced performance. In terms of pruning point identification, the method achieved impressive results with 94% precision, 90% recall, and 92% F1 score for foreground isolation. Furthermore, the segmentation of isolated images significantly outperformed non-isolated ones, with improvements exceeding 30%, 27%, and 30% in precision, recall, and F1 metrics, respectively. This validation also confirmed the method's effectiveness in accurately identifying pruning points, achieving a 67% accuracy rate when compared against manually identified pruning points. These results underscore the robustness and reliability of the approach in automating pruning processes in agricultural settings. / Master of Science / This thesis explores new methods for improving automated farming systems, particularly focusing on enhancing tasks like pruning where visibility of plant features can be significantly obstructed by overlapping leaves and branches. Central to this study is the development of an innovative approach using a special camera mounted on a robotic arm, which scans plants to determine the best vantage points for precise interactions. This setup not only identifies the optimal positions for viewing but also adjusts the robot's movements in real-time to ensure it can accurately perform pruning task. The innovative approach employed here leverages advanced technology to dynamically adjust the trajectory of the robotic arm based on real-time imaging. This enables the robot to better detect essential features of plants, which is crucial to make informed decision of where to prune the plant. By improving the robot's ability to clearly see and interact with plants, the system facilitates more precise and efficient operations. Tests conducted in simulated environments have demonstrated that this method significantly enhances the robot's capability to isolate and identify plant features accurately. These improvements make it possible for the robot to subsequently identify pruning points, potentially reducing the time and labor typically required in traditional manual operations. Overall, this research indicates that integrating advanced sensing and machine learning technologies into agricultural automation can revolutionize farming practices, making them more efficient and less dependent on human labor, especially in environments where traditional methods are less effective.

Background subtraction is the algorithmic process that segments out the region of interest often known as foreground from the background. Extensive literature and numerous algorithms exist in this domain, but most research have focused on videos captured by static cameras. The proliferation of portable platforms equipped with cameras has resulted in a large amount of video data being generated from moving cameras. This motivates the need for foundational algorithms for foreground/background segmentation in videos from moving cameras. In this dissertation, I propose three new types of background subtraction algorithms for moving cameras based on appearance, motion, and a combination of them. Comprehensive evaluation of the proposed approaches on publicly available test sequences show superiority of our system over state-of-the-art algorithms. The first method is an appearance-based global modeling of foreground and background. Features are extracted by sliding a fixed size window over the entire image without any spatial constraint to accommodate arbitrary camera movements. Supervised learning method is then used to build foreground and background models. This method is suitable for limited scene scenarios such as Pan-Tilt-Zoom surveillance cameras. The second method relies on motion. It comprises of an innovative background motion approximation mechanism followed by spatial regulation through a Mega-Pixel denoising process. This work does not need to maintain any costly appearance models and is therefore appropriate for resource constraint ego-vision systems. The proposed segmentation combined with skin cues is validated by a novel application on authenticating hand-gestured signature captured by wearable cameras. The third method combines both motion and appearance. Foreground probabilities are jointly estimated by motion and appearance. After the mega-pixel denoising process, the probability estimates and gradient image are combined by Graph-Cut to produce the segmentation mask. This method is universal as it can handle all types of moving cameras.

Polarised synchrotron emission from the Milky Way is of interest for its role as a foreground to the polarised CMB and as a probe of the interstellar medium. The Galactic ALFA Continuum Transit Survey (GALFACTS) and the Global Magneto-Ionic Medium Survey (GMIMS) are two ongoing surveys of the diffuse polarised emission around 1.4 GHz, with wide bandwidths and high frequency-resolution. In this thesis, I use early data from GALFACTS to investigate the behaviour of polarised, diffuse Galactic synchrotron emission. I also analyse GMIMS total intensity data. I derive a rotation measure (RM) map of the GALFACTS sky using a combination of RM-synthesis and linear angle fitting, commenting on the structure of the maps in general and on specific regions in particular. Overall I find that the maps are rich in features, and probe the RM structure of the extended Galactic emission with reasonable accuracy. I also derive the Angular Power Spectrum (APS) of the polarised emission for thirty-one 15 by 15 degree subregions across the GALFACTS data. I compute the E- and B-modes (E+B) and the scalar APS of the polarised emission (PI). I parametrise the APS by fitting a power law to the data. Comparing the E+B APS to the PI APS shows that E+B is consistently steeper across the sky. The APS data is also used to estimate the level of foreground contamination of the CMB B-mode by the synchrotron emission. I find that the slope of the APS averaged over high-latitude, low-emission subregions agrees exactly with that of the Planck 30 GHz polarised emission, thus setting an upper limit to the synchrotron contamination of CMB B-modes. Finally, I evaluate the spurious, systematic, temperature zero-level offset and associated uncertainty in preliminary GMIMS total intensity maps, finding a lower limit of ±0.26 K. I also make spectral index maps made using the GMIMS data and the Haslam et al. (1982) 408 MHz map, improving upon previous spectral index maps in the literature.

Cette thèse s'inscrit dans un cadre de vidéo-surveillance, et s'intéresse plus précisément à la détection robustesd'objets mobiles dans une séquence d'images. Une bonne détection d'objets mobiles est un prérequis indispensableà tout traitement appliqué à ces objets dans de nombreuses applications telles que le suivi de voitures ou depersonnes, le comptage des passagers de transports en commun, la détection de situations dangereuses dans desenvironnements spécifiques (passages à niveau, passages piéton, carrefours, etc.), ou encore le contrôle devéhicules autonomes. Un très grand nombre de ces applications utilise un système de vision par ordinateur. Lafiabilité de ces systèmes demande une robustesse importante face à des conditions parfois difficiles souventcausées par les conditions d'illumination (jour/nuit, ombres portées), les conditions météorologiques (pluie, vent,neige) ainsi que la topologie même de la scène observée (occultations). Les travaux présentés dans cette thèsevisent à améliorer la qualité de détection d'objets mobiles en milieu intérieur ou extérieur, et à tout moment de lajournée.Pour ce faire, nous avons proposé trois stratégies combinables :i) l'utilisation d'invariants colorimétriques et/ou d'espaces de représentation couleur présentant des propriétésinvariantes ;ii) l'utilisation d'une caméra stéréoscopique et d'une caméra active Microsoft Kinect en plus de la caméra couleurafin de reconstruire l'environnement 3D partiel de la scène, et de fournir une dimension supplémentaire, à savoirune information de profondeur, à l'algorithme de détection d'objets mobiles pour la caractérisation des pixels ;iii) la proposition d'un nouvel algorithme de fusion basé sur la logique floue permettant de combiner les informationsde couleur et de profondeur tout en accordant une certaine marge d'incertitude quant à l'appartenance du pixel aufond ou à un objet mobile. / This PhD thesis falls within the scope of video-surveillance, and more precisely focuses on the detection of movingobjects in image sequences. In many applications, good detection of moving objects is an indispensable prerequisiteto any treatment applied to these objects such as people or cars tracking, passengers counting, detection ofdangerous situations in specific environments (level crossings, pedestrian crossings, intersections, etc.), or controlof autonomous vehicles. The reliability of computer vision based systems require robustness against difficultconditions often caused by lighting conditions (day/night, shadows), weather conditions (rain, wind, snow...) and thetopology of the observed scene (occultation...).Works detailed in this PhD thesis aim at reducing the impact of illumination conditions by improving the quality of thedetection of mobile objects in indoor or outdoor environments and at any time of the day. Thus, we propose threestrategies working as a combination to improve the detection of moving objects:i) using colorimetric invariants and/or color spaces that provide invariant properties ;ii) using passive stereoscopic camera (in outdoor environments) and Microsoft Kinect active camera (in outdoorenvironments) in order to partially reconstruct the 3D environment, providing an additional dimension (a depthinformation) to the background/foreground subtraction algorithm ;iii) a new fusion algorithm based on fuzzy logic in order to combine color and depth information with a certain level ofuncertainty for the pixels classification.

A COMPARATIVE EVALUATION OF FOREGROUND / BACKGROUND SEGMENTATION ALGORITHMS Pakyurek, Muhammet M.Sc., Department of Electrical and Electronics Engineering Supervisor: Prof. Dr. G&ouml / zde Bozdagi Akar September 2012, 77 pages Foreground Background segmentation is a process which separates the stationary objects from the moving objects on the scene. It plays significant role in computer vision applications. In this study, several background foreground segmentation algorithms are analyzed by changing their critical parameters individually to see the sensitivity of the algorithms to some difficulties in background segmentation applications. These difficulties are illumination level, view angles of camera, noise level, and range of the objects. This study is mainly comprised of two parts. In the first part, some well-known algorithms based on pixel difference, probability, and codebook are explained and implemented by providing implementation details. The second part includes the evaluation of the performances of the algorithms which is based on the comparison v between the foreground background regions indicated by the algorithms and ground truth. Therefore, some metrics including precision, recall and f-measures are defined at first. Then, the data set videos having different scenarios are run for each algorithm to compare the performances. Finally, the performances of each algorithm along with optimal values of their parameters are given based on f measure.