Problems in automated video surveillance analysis caused by illumination changes are explored, and solutions are presented. Controlled experiments are first conducted to measure the responses of color targets to changes in lighting intensity and spectrum. Surfaces of dissimilar color are found to respond significantly differently. Illumination compensation model error is reduced by 70% to 80% by individually optimizing model parameters for each distinct color region, and applying a model tuned for one region to a chromatically different region increases error by a factor of 15. A background model--called BigBackground--is presented to extract large, stable, chromatically self-similar background features by identifying the dominant colors in a scene. The stability and chromatic diversity of these features make them useful reference points for quantifying illumination changes. The model is observed to cover as much as 90% of a scene, and pixels belonging to the model are 20% more stable on average than non-member pixels. Several illumination compensation techniques are developed to exploit BigBackground, and are compared with several compensation techniques from the literature. Techniques are compared in terms of foreground / background classification, and are applied to an object tracking pipeline with kinematic and appearance-based correspondence mechanisms. Compared with other techniques, BigBackground-based techniques improve foreground classification by 25% to 43%, improve tracking accuracy by an average of 20%, and better preserve object appearance for appearance-based trackers. All algorithms are implemented in C or C++ to support the consideration of runtime performance. In terms of execution speed, the BigBackground-based illumination compensation technique is measured to run on par with the simplest compensation technique used for comparison, and consistently achieves twice the frame rate of the two next-fastest techniques.

Background subtraction is the technique of segmenting moving foreground objects from stationary or dynamic background scenes. Background subtraction is a critical step in many computer vision applications including video surveillance, tracking, gesture recognition etc. This thesis addresses the challenges associated with the background subtraction systems due to the sudden illumination changes happening in an indoor environment. Most of the existing techniques adapt to gradual illumination changes, but fail to cope with the sudden illumination changes. Here, we introduce a Global change reactive background subtraction to model these changes as a regression function of spatial image coordinates. The regression model is learned from highly probable background regions and the background model is compensated for the illumination changes by the model parameters estimated. Experiments were performed in the indoor environment to show the effectiveness of our approach in modeling the sudden illumination changes by a higher order regression polynomial. The results of non-linear SVM regression were also presented to show the robustness of our regression model.