In the forest products manufacturing industry, recent price increases in the cost of high-quality lumber together with the reduced availability of this resource have forced manufacturers to utilize lower grade hardwood lumber in their manufacturing operations. This use of low quality lumber means that the labor involved in converting this lumber to usable parts is also increased because it takes more time to remove the additional defects that occur in the lower grade material. Simultaneously, labor costs have gone up and availability of skilled workers capable of getting a high yield of usable parts has markedly decreased. To face this increasingly complex and competitive environment, the industry has a critical need for efficient and cost-effective new processing equipment that can replace human operators who locate and identify defects that need to be removed in lumber and then remove these defects when cutting the lumber into rough parts. This human inspection process is laborious, inconsistent and subjective in nature due to the demands of making decisions very rapidly in a noisy and tiring environment. Hence, an automatic sawing system that could remove defects in lumber while creating maximum yield, offers significant opportunities for increasing profits of this industry. The difficult part in designing an automatic sawing system is creating an automatic inspection system that can detect critical features in wood that affect the quality of the rough parts. Many automatic inspection systems have been proposed and studied for the inspection of wood or wood products. But, most of these systems utilize a single sensing modality, e.g., a single optical sensor or an X-ray imaging system. These systems cannot detect all critical defects in wood. This research work reported in this dissertation is the first aimed at creating a vision system utilizes three imaging modalities: a color imaging system, a laser range profiling system and an X-ray imaging system. The objective of in designing this vision system is to detect and identify: 1) surface features such as knots, splits, stains; 2) geometry features such as wane, thin board; and 3) internal features such as voids, knots. The laser range profiling system is used to locate and identify geometry features. The X-ray imaging system is primarily used to detect features such as knots, splits and interior voids. The color imaging system is mainly employed to identify surface features. In this vision system a number of methodologies are used to improve processing speed and identification accuracy. The images from different sensing modalities are analyzed in a special order to offset the larger amount of image data that comes from the multiple sensors and that must be analyzed. The analysis of laser image is performed first. It is used to find defects that have insufficient thickness. These defects are then removed from consideration in the subsequent analysis of the X-ray image. Removing these defects from consideration in the analysis of the X-ray image not only improves the accuracy of detecting and identifying defects but also reduces the amount of time needed to analyze the X-ray image. Similarly, defect areas such as knot and mineral streak that are found in the analysis of the X-ray image are removed from consideration in the analysis of the color image. A fuzzy logic algorithm -- the approaching degree method-- is used to assign defect labels. The fuzzy logic approach is used to mimic human behavior in identifying defects in hardwood lumber. The initial results obtained from this vision system demonstrate the feasibility of locating and identifying all the major defects that occur in hardwood lumber. This was even true during the initial hardware development phase when only images of unsatisfactory quality from a limited lumber of samples were available. The vision system is capable of locating and identifying defects at the production speed of two linear feet per second that is typical in most hardwood secondary manufacturing plants. This vision system software was designed to run on a relative slow computer (200 MHz Pentium processor) with aid of special image processing hardware, i.e., the MORRPH board that was also designed at Virginia Tech. / Ph. D.

Recent research discussed the development of image processing tools as a part of the quality control framework in manufacturing environments. This research could be divided into two image-based fault detection approaches: 1) MVS; and 2) MVS and control charts. Despite the intensive research in both groups, there is a disconnect between research and the actual needs on the shop-floor. This disconnect is mainly attributed to the following: • The literature for the first category has mainly focused on improving fault detection accuracy through the use of special setups without considering its impact on the manufacturing process. Therefore, many of these methods have not been utilized by industry, and these tools lack the capability of using images already present on the shop floor. • The studies presented on the second category have been mainly developed in isolation. In addition, most of these studies have focused more on introducing the concept of utilizing control charts on image data rather than tackling specific industry problems. • In this thesis, these limitations are investigated and are disseminated to the research community through two different journal papers. In the first paper, it was shown that a face-recognition tool could be successfully used to detect faults in real-time in stamped processes, where the changes in image lighting conditions and part location were allowed to emulate actual manufacturing environments. On the other hand, the second paper reviewed the literature on image-based control charts and suggested recommendations for future research. / Master of Science

There is an increasing demand for robotics in dangerous and extreme conditions to limit human exposure and risk. An area in which robots are being considered as a support tool is in firefighting operations to reduce the number of firefighter injuries and deaths. One such application is to increase firefighting performance through localized fire suppression. This research focused on developing an autonomous suppression system for use on a mobile robotic platform. This included a real-time close proximity fire suppression approach, appropriate feature selection and probabilistic classification of water leaks and sprays, real-time trajectory estimation, and a feedback controller for error correction in longer-range firefighting. The close proximity suppression algorithm uses IR fire detection IR stereo processing to localize a fire. Feedback of the fire size and fire target was used to manipulate the nozzle for effective placement of the suppressant onto the fire and experimentally validated with tests in high and low visibility environments. To improve performance of autonomous suppression and for inspection tasks, identification of water sprays and leaks is a critical component. Bayesian classification was used to identify the features associated with water leaks and sprays in thermal images. Appropriate first and second order features were selected by using a multi-objective genetic algorithm optimization. Four textural features were selected as a method of discriminating water sprays and leaks from other non-water, high motion objects. Water classification was implemented into a real-time suppression system as a method of determining the yaw and pitch angle of a water nozzle. Estimation of the angle orientation provided an error estimate between the current path and desired nozzle orientation. A proportional-integral (PI) controller was used to correct for forced errors in fire targeting and performance and response was shown through indoor and outdoor suppression tests with wood-crib fires. The autonomous suppression algorithm was demonstrated through fire testing to be at least three times faster compared with suppression by an operator using tele-operation. / Ph. D.

This thesis describes the development of a commercial machine vision system as a case study for utilizing the Modular Reprogrammable Real-time Processing Hardware (MORRPH) board. The commercial system described in this thesis is based on a prototype system that was developed as a test-bed for developing the necessary concepts and algorithms. The prototype system utilized color linescan cameras, custom framegrabbers, and standard PCs to color-sort red oak parts (staves). When a furniture manufacturer is building a panel, very often they come from edge-glued paneled parts. These are panels formed by gluing several smaller staves together along their edges to form a larger panel. The value of the panel is very much dependent upon the "match" of the individual staves—i.e. how well they create the illusion that the panel came from a single board as opposed to several staves. The prototype system was able to accurately classify staves based on color into classes defined through a training process. Based on Trichromatic Color Theory, the system developed a probability density function in 3-D color space for each class based on the parts assigned to that class during training. While sorting, the probability density function was generated for each scanned piece, and compared with each of the class probability density functions. The piece was labeled the name of the class whose probability density function it most closely matched. A "best-face" algorithm was also developed to arbitrate between pieces whose top and bottom faces did not fall into the same classes. [1] describes the prototype system in much greater detail. In developing a commercial-quality machine vision system based on the prototype, the primary goal was to improve throughput. A Field Programmable Gate Array (FPGA)-based Custom Computing Machine (FCCM) called the MORRPH was selected to assume most of the computational burden, and increase throughput in the commercial system. The MORRPH was implemented as an ISA-bus interface card, with a 3 x 2 array of Processing Elements (PE). Each PE consists of an open socket which can be populated with a Xilinx 4000 series FPGA, and an open support socket which can be populated with support chips such as external RAM, math processors, etc. In implementing the prototype algorithms for the commercial system, a partition was created between those algorithms that would be implemented on the MORRPH board, and those that would be left as implemented on the host PC. It was decided to implement such algorithms as Field-Of-View operators, Shade Correction, Background Extraction, Gray-Scale Channel Generation, and Histogram Generation on the MORRPH board, and to leave the remainder of the classification algorithms on the host. By utilizing the MORRPH board, an industrial machine vision system was developed that has exceeded customer expectations for both accuracy and throughput. Additionally, the color-sorter received the International Woodworking Fair's Challengers Award for outstanding innovation. / Master of Science

Few and Far Between is a music composition for violoncello and live electronics. The project consists of software that uses video data from a webcam to control interactive audio in real time to manipulate audio processing and multichannel diffusion.

Master's thesis deals with the design, assembly, and testing of a camera system for localization of randomly placed and oriented objects on a conveyor belt with the purpose of guiding a robot on those objects. The theoretical part is focused on research in individual components making a camera system and on the field of 2D and 3D localization of objects. The practical part consists of two possible arrangements of the camera system, solution of the chosen arrangement, creating testing images, programming the algorithm for image processing, creating HMI, and testing the complete system.

The inclusion of precision dairy farming technologies in dairy operations is an area of increasing research and industry direction. Machine vision based systems are suitable for the dairy environment as they do not inhibit workflow, are capable of continuous operation, and can be fully automated. The research of this dissertation developed and tested 3 machine vision based precision dairy farming technologies tailored to the latest generation of RGB+D cameras. The first system focused on testing various imaging approaches for the potential use of machine vision for automated dairy cow feed intake monitoring. The second system focused on monitoring the gradual change in body condition score (BCS) for 116 cows over a nearly 7 month period. Several proposed automated BCS systems have been previously developed by researchers, but none have monitored the gradual change in BCS for a duration of this magnitude. These gradual changes infer a great deal of beneficial and immediate information on the health condition of every individual cow being monitored. The third system focused on automated dairy cow feature detection using Haar cascade classifiers to detect anatomical features. These features included the tailhead, hips, and rear regions of the cow body. The features chosen were done so in order to aid machine vision applications in determining if and where a cow is present in an image or video frame. Once the cow has been detected, it must then be automatically identified in order to keep the system fully automated, which was also studied in a machine vision based approach in this research as a complimentary aspect to incorporate along with cow detection. Such systems have the potential to catch poor health conditions developing early on, aid in balancing the diet of the individual cow, and help farm management to better facilitate resources, monetary and otherwise, in an appropriate and efficient manner. Several different applications of this research are also discussed along with future directions for research, including the potential for additional automated precision dairy farming technologies, integrating many of these technologies into a unified system, and the use of alternative, potentially more robust machine vision cameras.

L'utilisation de collections d'images pour les applications de vision par ordinateur devient de plus en plus commune des nos jours. L'objectif principal de cette thèse est d'exploiter et d'extraire des informations importantes d'images de scènes d'extérieur a partir de ce type de collections : l'illumination présente au moment de la prise, les propriétés de reflectance des matériaux composant les objets dans la scène et les propriétés radiométriques des appareils photo utilisés. Pour atteindre notre objectif, cette thèse est composée de deux parties principales. Dans un premier temps nous allons réaliser une analyse de différentes représentations du ciel et une comparaison des images basée sur l'apparence de celui-ci. Une grande partie de l'information visuelle perçue dans les images d'extérieures est due a l'illumination en provenance du ciel. Ce facteur est représenté par les rayons du soleil réfléchis et réfractés dans l'atmosphère en créant une illumination globale de l'environnement. En même temps cet environnement détermine la façon de percevoir les objets du monde réel. Etant donné l'importance du ciel comme source d'illumination, nous formulons un processus générique en trois temps, segmentation, modélisation et comparaison des pixels du ciel, pour trouver des images similaires en se basant sur leurs apparences. Différentes méthodes sont adoptées dans les phases de modélisation et de comparaison. La performance des algorithmes est validée en trouvant des images similaires dans de grandes collections de photos. La deuxième partie de cette thèse consiste a exploiter l'information géométrique additionnelle pour en déduire les caractéristiques photométriques de la scène. A partir d'une structure 3D récupérée en utilisant des méthodes disponibles, nous analysons le processus de formation de l'image a partir de modèles simples, puis nous estimons les paramètres qui les régissent. Les collections de photos sont généralement capturées par différents appareils photos, d'où l'importance d'insister sur leur calibrage radiométrique. Notre formulation estime cet étalonnage pour tous les appareils photos en même temps, en utilisant une connaissance a priori sur l'espace des fonctions de réponse des caméras possibles. Nous proposons ensuite, un cadre d'estimation conjoint pour calculer une représentation de l'illumination globale dans chaque image, l'albedo de la surface qui compose la structure 3D et le calibrage radiométrique pour tous les appareils photos. / The main objective of this thesis is to exploit the photometric information avail- able in large photo collections of outdoor scenes to infer characteristics of the illumination, the objects and the cameras. To achieve this goal two problems are addressed. In a preliminary work, we explore opti- mal representations for the sky and compare images based on its appearance. Much of the information perceived in outdoor scenes is due to the illumination coming from the sky. The solar beams are reflected and refracted in the atmosphere, creating a global illumination ambiance. In turn, this environment determines the way that we perceive objects in the real world. Given the importance of the sky as an illumination source, we formulate a generic 3–step process in order to compare images based on its appearance. These three stages are: segmentation, modeling and comparing of the sky pixels. Different approaches are adopted for the modeling and comparing phases. Performance of the algorithms is validated by finding similar images in large photo collections. A second part of the thesis aims to exploit additional geometric information in order to deduce the photometric characteristics of the scene. From a 3D structure recovered using available multi–view stereo methods, we trace back the image formation process and estimate the models for the components involved on it. Since photo collections are usually acquired with different cameras, our formulation emphasizes the estimation of the radiometric calibration for all the cameras at the same time, using a strong prior on the possible space of camera response functions. Then, in a joint estimation framework, we also propose a robust computation of the global illumination for each image, the surface albedo for the 3D structure and the radiometric calibration for all the cameras.

Several novel structure from motion algorithms are presented that are designed to more effectively manage the problem of noise. In many practical applications, structure from motion algorithms fail to work properly because of the noise in the optical flow values. Most structure from motion algorithms implicitly assume that the noise is identically distributed and that the noise is white. Both assumptions are false. Some points can be track more easily than others and some points can be tracked more easily in a particular direction. The accuracy of each optical flow value can be quantified using an optical flow probability distribution. By using optical flow probability distributions in place of optical flow estimates in a structure from motion algorithm, a better understanding of the noise is developed and a more accurate solution is obtained. Two different methods of calculating the optical flow probability distributions are presented. The first calculates non-Gaussian probability distributions and the second calculates Gaussian probability distributions. Three different methods for calculating structure from motion are presented that use these probability distributions. The first method works on two frames and can handle any kind of noise. The second method works on two frames and is restricted to only Gaussian noise. The final method works on multiple frames and uses Gaussian noise. A simulation was created to directly compare the performance of methods that use optical flow probability distributions and methods that do not. The simulation results show that those methods which use the probability distributions better estimate the camera motion and the structure of the scene.

We investigate the unsupervised K-means clustering and the semi-supervised hidden Markov model (HMM) to automatically detect anomalous motion patterns in groups of people (crowds). Anomalous motion patterns are typically people merging into a dense group, followed by disturbances or threatening situations within the group. The application of K-means clustering and HMM are illustrated with datasets from four surveillance scenarios. The results indicate that by investigating the group of people in a systematic way with different K values, analyze cluster density, cluster quality and changes in cluster shape we can automatically detect anomalous motion patterns. The results correspond well with the events in the datasets. The results also indicate that very accurate detections of the people in the dense group would not be necessary. The clustering and HMM results will be very much the same also with some increased uncertainty in the detections. / <p>Funding Agencies|Vinnova (Swedish Governmental Agency for Innovation Systems) under the VINNMER program||</p>