Computer vision approximates human vision using computers. Two subsets are explored in this work: image segmentation and visual tracking. Segmentation involves partitioning an image into logical parts, and tracking analyzes objects as they change over time. The presented research explores a key hypothesis: localizing analysis of visual information can improve the accuracy of segmentation and tracking results. Accordingly, a new class of segmentation techniques based on localized analysis is developed and explored. Next, these techniques are applied to two challenging problems: neuron bundle segmentation in diffusion tensor imagery (DTI) and plaque detection in computed tomography angiography (CTA) imagery. Experiments demonstrate that local analysis is well suited for these medical imaging tasks. Finally, a visual tracking algorithm is shown that uses temporal localization to track objects that change drastically over time.

This thesis aims to contribute to the area of visual tracking, which is the process of identifying an object of interest through a sequence of successive images. The thesis explores kernel-based statistical methods, which map the data to a higher dimensional space. A pre-image framework is provided to find the mapping from the embedding space to the input space for several manifold learning and dimensional learning algorithms. Two algorithms are developed for visual tracking that are robust to noise and occlusions. In the first algorithm, a kernel PCA-based eigenspace representation is used. The de-noising and clustering capabilities of the kernel PCA procedure lead to a robust algorithm. This framework is extended to incorporate the background information in an energy based formulation, which is minimized using graph cut and to track multiple objects using a single learned model. In the second method, a robust density comparison framework is developed that is applied to visual tracking, where an object is tracked by minimizing the distance between a model distribution and given candidate distributions. The superior performance of kernel-based algorithms comes at a price of increased storage and computational requirements. A novel method is developed that takes advantage of the universal approximation capabilities of generalized radial basis function neural networks to reduce the computational and storage requirements for kernel-based methods.

Due to the large extent to which important biological mechanisms are conserved evolutionarily, the study of a simple soil nematode, C. elegans, has provided the template for significant advances in biology. Use of this model organism has accelerated in recent years as developments of advanced reagents such as synapse localized fluorescent markers have provided powerful tools to study the complex process of synapse formation and remodeling. Even as much routine biology work, such as sequencing, has become faster and easier, imaging protocols have remained essentially unchanged over the past forty years of research. This, coupled with the ability to visualize small, complex features as a result of new fluorescent reagents, has resulted in genetic screens in C. elegans becoming increasingly labor intensive and slow because microscopy mainly relies on manual mounting of animals and phenotyping is usually visually done by experts. Genetic screens have become the rate limiting factor for much of modern C. elegans research. Furthermore, phenotyping of fluorescent expression has remained a primarily qualitative process which has prevented statistical analysis of subtle features. To address these issues, a comprehensive system to allow autonomous screening for novel mutants was created. This was done by developing novel microfluidic devices to enable high-throughput screening, systems-level components to allow automated operation, and a computer vision framework for identification and quantitative phenotyping of synaptic patterns. The microfluidic platform allows for imaging and sorting of thousands of animals at high-magnification within hours. The computer vision framework employs a two-stage feature extraction to incorporate local and regional features and allows for synapse identification in near real-time with an extremely low error rate. Using this system thousands of mutagenized animals were screened to indentify numerous novel mutants expressing altered synaptic placement and development. Fully automated screening and analysis of subtle fluorescent phenotypes will allow large scale RNAi and drug screens. Combining microfluidics and computer vision approaches will have a significant impact on the biological community by removing a significant bottleneck and allowing large-scale screens that would have previously been too labor intensive to attempt.

As demand for real-time image processing increases, the need to improve the efficiency of image processing systems is growing. The process of image segmentation is often used in preprocessing stages of computer vision systems to reduce image data and increase processing efficiency. This dissertation introduces a novel image segmentation approach known as leap segmentation, which applies a flexible definition of adjacency to allow groupings of pixels into segments which need not be spatially contiguous and thus can more accurately correspond to large surfaces in the scene. Experiments show that leap segmentation correctly preserves an average of 20% more original scene pixels than traditional approaches, while using the same number of segments, and significantly improves execution performance (executing 10x - 15x faster than leading approaches). Further, leap segmentation is shown to improve the efficiency of a high-level vision application for scene layout analysis within 3D scene reconstruction. The benefits of applying image segmentation in preprocessing are not limited to single-frame image processing. Segmentation is also often applied in the preprocessing stages of video analysis applications. In the second contribution of this dissertation, the fast, single-frame leap segmentation approach is extended into the temporal domain to develop a highly-efficient method for multiple-frame segmentation, called video leap segmentation. This approach is evaluated for use on mobile platforms where processing speed is critical using moving-camera traffic sequences captured on busy, multi-lane highways. Video leap segmentation accurately tracks segments across temporal bounds, maintaining temporal coherence between the input sequence frames. It is shown that video leap segmentation can be applied with high accuracy to the task of salient segment transformation detection for alerting drivers to important scene changes that may affect future steering decisions. Finally, while research efforts in the field of image segmentation have often recognized the need for efficient implementations for real-time processing, many of today’s leading image segmentation approaches exhibit processing times which exceed their camera frame periods, making them infeasible for use in real-time applications. The third research contribution of this dissertation focuses on developing fast implementations of the single-frame leap segmentation approach for use on both single-core and multi-core platforms as well as on both high-performance and resource-constrained systems. While the design of leap segmentation lends itself to efficient implementations, the efficiency achieved by this algorithm, as in any algorithm, is can be improved with careful implementation optimizations. The leap segmentation approach is analyzed in detail and highly optimized implementations of the approach are presented with in-depth studies, ranging from storage considerations to realizing parallel processing potential. The final implementations of leap segmentation for both serial and parallel platforms are shown to achieve real-time frame rates even when processing very high resolution input images. Leap segmentation’s accuracy and speed make it a highly competitive alternative to today’s leading segmentation approaches for modern, real-time computer vision systems.

Thesis (Ph.D.). / Written for the Dept. of Electrical & Computer Engineering. Title from title page of PDF (viewed 2008/01/12). Includes bibliographical references.

Gaze tracking is the estimation of the point in space a person is “looking at”. This is widely used in both diagnostic and interactive applications, such as visual attention studies and human-computer interaction. The most common commercial solution used to track gaze today uses a combination of infrared illumination and one or more cameras. These commercial solutions are reliable and accurate, but often expensive. The aim of this thesis is to construct a simple single-camera gaze tracker from off-the-shelf components. The method used for gaze tracking is based on infrared illumination and a schematic model of the human eye. Based on images of reflections of specific light sources in the surfaces of the eye the user’s gaze point will be estimated. Evaluation is also performed on both the software and hardware components separately, and on the system as a whole. Accuracy is measured in spatial and angular deviation and the result is an average accuracy of approximately one degree on synthetic data and 0.24 to 1.5 degrees on real images at a range of 600 mm.

Att applicera ansiktsigenkänning direkt på en mobiltelefon är en utmanande uppgift, inte minst med tanke på den begränsade minnes- och processorkapaciteten samt den stora variationen med avseende på ansiktsuttryck, hållning och ljusförhållande i inmatade bilder. Det är fortfarande långt kvar till ett färdigutvecklat, robust och helautomatiskt ansiktsigenkänningssystem för den här miljön. Men resultaten i det här arbetet visar att genom att plocka ut feature-värden från lokala regioner samt applicera en välgjord warpstrategi för att minska problemen med variationer i position och rotation av huvudet, är det möjligt att uppnå rimliga och användbara igenkänningsnivåer. Speciellt för ett halvautomatiskt system där användaren har sista ordet om vem personen på bilden faktiskt är. Med ett galleri bestående av 85 personer och endast en referensbild per person nådde systemet en igenkänningsgrad på 60% på en svårklassificerad serie testbilder. Totalt 73% av gångerna var den rätta individen inom de fyra främsta gissningarna. Att lägga till extra referensbilder till galleriet höjer igenkänningsgraden rejält, till nästan 75% för helt korrekta gissningar och till 83,5% för topp fyra. Detta visar att en strategi där inmatade bilder läggs till som referensbilder i galleriet efterhand som de identifieras skulle löna sig ordentligt och göra systemet bättre efter hand likt en inlärningsprocess. Detta exjobb belönades med pris för "Bästa industrirelevanta bidrag" vid Svenska sällskapet för automatiserad bildanalys årliga konferens i Lund, 13-14 mars 2008. / Applying face recognition directly on a mobile phone is a challenging proposal due to the unrestrained nature of input images and limitations in memory and processor capabilities. A robust, fully automatic recognition system for this environment is still a far way off. However, results show that using local feature extraction and a warping scheme to reduce pose variation problems, it is possible to capitalize on high error tolerance and reach reasonable recognition rates, especially for a semi-automatic classification system where the user has the final say. With a gallery of 85 individuals and only one gallery image per individual available the system is able to recognize close to 60 % of the faces in a very challenging test set, while the correct individual is in the top four guesses 73% of the time. Adding extra reference images boosts performance to nearly 75% correct recognition and 83.5% in the top four guesses. This suggests a strategy where extra reference images are added one by one after correct classification, mimicking an online learning strategy.

Within this thesis an algorithm for object recognition called Cluster Matching has been developed, implemented and evaluated. The image information is sampled at arbitrary sample points, instead of interest points, and local image features are extracted. These sample points are used as a compact representation of the image data and can quickly be searched for prior known objects. The algorithm is evaluated on a test set of images and the result is surprisingly reliable and time efficient.

In this thesis an algorithm for producing saliency maps as well as an algorithm for detecting salient regions based on the saliency map was developed. The saliency values are computed as center-surround differences and a local descriptor called the region p-channel is used to represent center and surround respectively. An integral image representation called the integral p-channel is used to speed up extraction of the local descriptor for any given image region. The center-surround difference is calculated as either histogram or p-channel dissimilarities. Ground truth was collected using human subjects and the algorithm’s ability to detect salient regions was evaluated against this ground truth. The algorithm was also compared to another saliency algorithm. Two different center-surround interpretations are tested, as well as several p-channel and histogram dissimilarity measures. The results show that for all tested settings the best performing dissimilarity measure is the so called diffusion distance. The performance comparison showed that the algorithm developed in this thesis outperforms the algorithm against which it was compared, both with respect to region detection and saliency ranking of regions. It can be concluded that the algorithm shows promising results and further investigation of the algorithm is recommended. A list of suggested approaches for further research is provided.

Generic visual tracking is one of the classical problems in computer vision. In this problem, no prior knowledge of the target is available aside from a bounding box in the initial frame of the sequence. The generic visual tracking is a difficult task due to a number of factors such as momentary occlusions, target rotations, changes in target illumination and variations in the target size. In recent years, discriminative correlation filter (DCF) based trackers have shown promising results for visual tracking. These DCF based methods use the Fourier transform to efficiently calculate detection and model updates, allowing significantly higher frame rates than competing methods. However, existing DCF based methods only estimate translation of the object while ignoring changes in size.This thesis investigates the problem of accurately estimating the scale variations within a DCF based framework. A novel scale estimation method is proposed by explicitly constructing translation and scale filters. The proposed scale estimation technique is robust and significantly improve the tracking performance, while operating at real-time. In addition, a comprehensive evaluation of feature representations in a DCF framework is performed. Experiments are performed on the benchmark OTB-2015 dataset, as well as the VOT 2014 dataset. The proposed methods are shown to significantly improve the performance of existing DCF based trackers. / Allmän visuell följning är ett klassiskt problem inom datorseende. I den vanliga formuleringen antas ingen förkunskap om objektet som skall följas, utöver en initial rektangel i en videosekvens första bild.Detta är ett mycket svårt problem att lösa allmänt på grund av occlusioner, rotationer, belysningsförändringar och variationer i objektets uppfattde storlek. På senare år har följningsmetoder baserade på diskriminativea korrelationsfilter gett lovande resultat inom området. Dessa metoder är baserade på att med hjälp av Fourertransformen effektivt beräkna detektioner och modellupdateringar, samtidigt som de har mycket bra prestanda och klarar av många hundra bilder per sekund. De nuvarande metoderna uppskattar dock bara translationen hos det följda objektet, medans skalförändringar ignoreras. Detta examensarbete utvärderar ett antal metoder för att göra skaluppskattningar inom ett korrelationsfilterramverk. En innovativ metod baserad på att konstruera separata skal och translationsfilter. Den föreslagna metoden är robust och har signifikant bättre följningsprestanda, samtidigt som den kan användas i realtid. Det utförs också en utvärdering av olika särdragsrepresentationer på två stora benchmarking dataset för följning.