No description available.

No description available.

<p>A Velodyne Puck VLP-16 LiDAR and a Carnegie Robotics Multisense S21 stereo camera were placed in an environmental testing chamber to investigate dust and lighting effects on depth returns. The environmental testing chamber was designed and built with varied lighting conditions with corn dust plumes forming the atmosphere. Specific software employing ROS, Python, and OpenCV were written for point cloud streaming and publishing. Dust chamber results showed while dust effects were present in point clouds produced by both instruments, the stereo camera was able to “see” the far wall of the chamber and did not image the dust plume, unlike the LiDAR sensor. The stereo camera was also set up to measure the volume of total mixed ration (TMR) and shelled grain in various volume scenarios with mixed surface terrains. Calculations for finding actual pixel area based on depth were utilized along with a volume formula exploiting the depth capability of the stereo camera for the results. Resulting accuracy was good for a target of 8 liters of shelled corn with final values between 6.8 and 8.3 liters from three varied surface scenarios. Lessons learned from the chamber and volume measurements were applied to loading large grain vessels being filled from a 750-bushel grain cart in the form of calculating the volume of corn grain and tracking the location of the vessel in near real time. Segmentation, masking, and template matching were the primary software tools used within ROS, OpenCV, and Python. The S21 was the center hardware piece. Resulting video and images show some lag between depth and color images, dust blocking depth pixels, and template matching misses. However, results were sufficient to show proof of concept of tracking and volume estimation. </p>

Visually extracting textual context from lips consists of pattern matching which results in a frequent use of machine learning approaches for the task of classification. Previous research has consisted of mostly audiovisual (multi modal) approaches and conventional cameras. This study isolates the visual medium and uses event-based cameras instead of conventional cameras. Classifying visual features is computationally expensive and the minimisation of excessive data can be of importance for performance which motivates the usage of event cameras. Event cameras are inspired by the biological vision and only capture changes in the scene while offering high temporal resolution (corresponding to frame rate for conventional cameras). This study investigates the importance of temporal resolution for the task of lip reading by modifying the ∆time used for collecting events. No correlation could be observed within the collected data set. The paper is not able to come to any conclusions regarding suitability of the chosen approach for the particular application. There are multiple other variables that could effect the results which makes it hard to dismiss the technology’s potential within the domain. / Visuell bedömning av vilka ord läppar talar består av mönstermatchning vilket resulterar i att maskininlärning ofta används för att klassificera data som text. Tidigare studier har i hög grad varit audiovisuella(multimodala) och konventionella kameror. Visuell analys är beräkningsmässigt dyrt vilket motiverar en minimering av överflödig data för att öka prestandan, vilket motiverar användningen av eventkameror. Eventkameror är inspirerade av biologisk syn och registrerar endast skillnaden i omgivningen, samtidigt som de har en hög tidsupplösning (motsvarande frame rate för konventionella kameror). Studien undersöker relevansen av tidsupplösning för maskinell läppläsning genom att modifiera ∆time som används för att samla events. Ingen korrelation mellan ∆time och träffsäkerheten kunde observeras med det dataset som användes. Studien kan inte avfärda potentialen för tekniken eftersom det finns många fler parametrar som kan påverka träffsäkerheten.

This project explores the design and implementation of synchronized Pan-Tilt-Zoom (PTZ) camera control for panoramic imaging, with a specific focus on enhancing surveillance systems in air traffic management. The motivation for this study comes from the need for better monitoring in air traffic control, where panoramic views can greatly improve situational awareness and safety. The main challenge is coordinating multiple PTZ cameras to capture and stitch images, creating a comprehensive panoramic view despite individual camera limitations. The study uses simulation to test synchronization and image stitching techniques, ensuring camera alignment and seamless panoramic images. Results indicate the system's robustness and potential for real-world applications, though future validation with physical hardware is necessary.

This thesis analyzes the problem of acquiring stereoscopic images in all gazing directions around a reference viewpoint in space with the purpose of creating stereoscopic panoramas of non-static scenes. The generation of immersive stereoscopic imagery suitable to stimulate human stereopsis requires images from two distinct viewpoints with horizontal parallax in all gazing directions, or to be able to simulate this situation in the generated imagery. The available techniques to produce omnistereoscopic imagery for human viewing are not suitable to capture dynamic scenes stereoscopically. This is a not trivial problem when considering acquiring the entire scene at once while avoiding self-occlusion between multiple cameras. In this thesis, the term omnidirectional refers to all possible gazing directions in azimuth and a limited set of directions in elevation. The acquisition of dynamic scenes restricts the problem to those techniques suitable for collecting in one simultaneous exposure all the necessary visual information to recreate stereoscopic imagery in arbitrary gazing directions. The analysis of the problem starts by defining an omnistereoscopic viewing model for the physical magnitude to be measured by a panoramic image sensor intended to produce stereoscopic imagery for human viewing. Based on this model, a novel acquisition model is proposed, which is suitable to describe the omnistereoscopic techniques based on horizontal stereo. From this acquisition model, an acquisition method based on multiple cameras combined with the rendering by mosaicking of partially overlapped stereoscopic images is identified as a good candidate to produce omnistereoscopic imagery of dynamic scenes. Experimental acquisition and rendering tests were performed for different multiple-camera configurations. Furthermore, a mosaicking criterion between partially overlapped stereoscopic images based on the continuity of the perceived depth and the prediction of the location and magnitude of unwanted vertical disparities in the final stereoscopic panorama are two main contributions of this thesis. In addition, two novel omnistereoscopic acquisition and rendering techniques were introduced. The main contributions to this field are to propose a general model for the acquisition of omnistereoscopic imagery, to devise novel methods to produce omnistereoscopic imagery, and more importantly, to contribute to the awareness of the problem of acquiring dynamic scenes within the scope of omnistereoscopic research.

High resolution surveillance video cameras are invaluable resources for effective crime prevention and forensic investigations. However, increasing communication bandwidth requirements of high definition surveillance videos are severely limiting the number of cameras that can be deployed. Higher bitrate also increases operating expenses due to higher data communication and storage costs. Hence, it is essential to develop low complexity algorithms which reduce data rate of the compressed video stream without affecting the image fidelity. In this thesis, a computer vision aided H.264 surveillance video encoder and four associated algorithms are proposed to reduce the bitrate. The proposed techniques are (I) Speeded up foreground segmentation, (II) Skip decision, (III) Reference frame selection and (IV) Face Region-of-Interest (ROI) coding. In the first part of the thesis, a modification to the adaptive Gaussian Mixture Model (GMM) based foreground segmentation algorithm is proposed to reduce computational complexity. This is achieved by replacing expensive floating point computations with low cost integer operations. To maintain accuracy, we compute periodic floating point updates for the GMM weight parameter using the value of an integer counter. Experiments show speedups in the range of 1.33 - 1.44 on standard video datasets where a large fraction of pixels are multimodal. In the second part, we propose a skip decision technique that uses a spatial sampler to sample pixels. The sampled pixels are segmented using the speeded up GMM algorithm. The storage pattern of the GMM parameters in memory is also modified to improve cache performance. Skip selection is performed using the segmentation results of the sampled pixels. In the third part, a reference frame selection algorithm is proposed to maximize the number of background Macroblocks (MB’s) (i.e. MB’s that contain background image content) in the Decoded Picture Buffer. This reduces the cost of coding uncovered background regions. Distortion over foreground pixels is measured to quantify the performance of skip decision and reference frame selection techniques. Experimental results show bit rate savings of up to 94.5% over methods proposed in literature on video surveillance data sets. The proposed techniques also provide up to 74.5% reduction in compression complexity without increasing the distortion over the foreground regions in the video sequence. In the final part of the thesis, face and shadow region detection is combined with the skip decision algorithm to perform ROI coding for pedestrian surveillance videos. Since person identification requires high quality face images, MB’s containing face image content are encoded with a low Quantization Parameter setting (i.e. high quality). Other regions of the body in the image are considered as RORI (Regions of reduced interest) and are encoded at low quality. The shadow regions are marked as Skip. Techniques that use only facial features to detect faces (e.g. Viola Jones face detector) are not robust in real world scenarios. Hence, we propose to initially detect pedestrians using deformable part models. The face region is determined using the deformed part locations. Detected pedestrians are tracked using an optical flow based tracker combined with a Kalman filter. The tracker improves the accuracy and also avoids the need to run the object detector on already detected pedestrians. Shadow and skin detector scores are computed over super pixels. Bilattice based logic inference is used to combine multiple likelihood scores and classify the super pixels as ROI, RORI or RONI. The coding mode and QP values of the MB’s are determined using the super pixel labels. The proposed techniques provide a further reduction in bitrate of up to 50.2%.

This thesis analyzes the problem of acquiring stereoscopic images in all gazing directions around a reference viewpoint in space with the purpose of creating stereoscopic panoramas of non-static scenes. The generation of immersive stereoscopic imagery suitable to stimulate human stereopsis requires images from two distinct viewpoints with horizontal parallax in all gazing directions, or to be able to simulate this situation in the generated imagery. The available techniques to produce omnistereoscopic imagery for human viewing are not suitable to capture dynamic scenes stereoscopically. This is a not trivial problem when considering acquiring the entire scene at once while avoiding self-occlusion between multiple cameras. In this thesis, the term omnidirectional refers to all possible gazing directions in azimuth and a limited set of directions in elevation. The acquisition of dynamic scenes restricts the problem to those techniques suitable for collecting in one simultaneous exposure all the necessary visual information to recreate stereoscopic imagery in arbitrary gazing directions. The analysis of the problem starts by defining an omnistereoscopic viewing model for the physical magnitude to be measured by a panoramic image sensor intended to produce stereoscopic imagery for human viewing. Based on this model, a novel acquisition model is proposed, which is suitable to describe the omnistereoscopic techniques based on horizontal stereo. From this acquisition model, an acquisition method based on multiple cameras combined with the rendering by mosaicking of partially overlapped stereoscopic images is identified as a good candidate to produce omnistereoscopic imagery of dynamic scenes. Experimental acquisition and rendering tests were performed for different multiple-camera configurations. Furthermore, a mosaicking criterion between partially overlapped stereoscopic images based on the continuity of the perceived depth and the prediction of the location and magnitude of unwanted vertical disparities in the final stereoscopic panorama are two main contributions of this thesis. In addition, two novel omnistereoscopic acquisition and rendering techniques were introduced. The main contributions to this field are to propose a general model for the acquisition of omnistereoscopic imagery, to devise novel methods to produce omnistereoscopic imagery, and more importantly, to contribute to the awareness of the problem of acquiring dynamic scenes within the scope of omnistereoscopic research.

Thesis (MScEng (Mechanical and Mechatronic Engineering))--University of Stellenbosch, 2008. / A rapid measurement system is implemented using two digital video cameras, presenting a faster and less expensive solution to certain metrology problems. The cameras are calibrated from one stereo image-pair of a 3D calibration grid that allows an immediate assessment of the achievable metric accuracy of the system. Three different methods, using either laser tracking or structured light patterns, were developed and employed to solve the coordinate extraction and correspondence matching problems. Different image processing techniques were used to speed up the entire measurement process. All software development was accomplished using only freely distributed software packages. The system achieves calibration in less than a minute and accumulates point correspondences at 12 frames per second. Accuracies of greater than 0.4 mm are achieved for a 235 x 190 x 95 mm measurement volume using a single pair of images with 640 x 480 pixel resolution each.

Background subtraction is the algorithmic process that segments out the region of interest often known as foreground from the background. Extensive literature and numerous algorithms exist in this domain, but most research have focused on videos captured by static cameras. The proliferation of portable platforms equipped with cameras has resulted in a large amount of video data being generated from moving cameras. This motivates the need for foundational algorithms for foreground/background segmentation in videos from moving cameras. In this dissertation, I propose three new types of background subtraction algorithms for moving cameras based on appearance, motion, and a combination of them. Comprehensive evaluation of the proposed approaches on publicly available test sequences show superiority of our system over state-of-the-art algorithms. The first method is an appearance-based global modeling of foreground and background. Features are extracted by sliding a fixed size window over the entire image without any spatial constraint to accommodate arbitrary camera movements. Supervised learning method is then used to build foreground and background models. This method is suitable for limited scene scenarios such as Pan-Tilt-Zoom surveillance cameras. The second method relies on motion. It comprises of an innovative background motion approximation mechanism followed by spatial regulation through a Mega-Pixel denoising process. This work does not need to maintain any costly appearance models and is therefore appropriate for resource constraint ego-vision systems. The proposed segmentation combined with skin cues is validated by a novel application on authenticating hand-gestured signature captured by wearable cameras. The third method combines both motion and appearance. Foreground probabilities are jointly estimated by motion and appearance. After the mega-pixel denoising process, the probability estimates and gradient image are combined by Graph-Cut to produce the segmentation mask. This method is universal as it can handle all types of moving cameras.