Three-dimensional (3D) non-contact optical methods for surface inspection are of significant interest to many industrial sectors. Many aspects of manufacturing processes have become fully automated resulting in high production volumes. However, this is not necessarily the case for surface defect inspection. Existing human visual analysis of surface defects is qualitative and subject to varying interpretation. Automated 3D non-contact analysis should provide a robust and systematic quantitative approach. However, different 3D optical measurement technologies use different physical principles, interact with surfaces and defects in diverse ways, leading to variation in measurement data. Instrument s native software processing of the data may be non-traceable in nature, leading to significant uncertainty about data quantisation. Sub-millimetric level surface defect artefacts have been created using Rockwell and Vickers hardness testing equipment on various substrates. Four different non-contact surface measurement instruments (Alicona InfiniteFocus G4, Zygo NewView 5000, GFM MikroCAD Lite and Heliotis H3) have been utilized to measure different defect artefacts. The four different 3D optical instruments are evaluated by calibrated step-height created using slipgauges and reference defect artefacts. The experimental results are compared to select the most suitable instrument capable of measuring surface defects in robust manner. This research has identified a need for an automatic tool to quantify surface defect and thus a mathematical solution has been implemented for automatic defect detection and quantification (depth, area and volume) in 3D. A simulated defect softgauge with a known geometry has been developed in order to verify the implemented algorithm and provide mathematical traceability. The implemented algorithm has been identified as a traceable, highly repeatable, and high speed solution to quantify surface defect in 3D. Various industrial components with suspicious features and solder joints on PCB are measured and quantified in order to demonstrate applicability.

<p>Laser scanners in combination with devices for accurate orientation like Coordinate Measuring Machines (CMM) are often used in Geometric Reverse Engineering (GRE) to measure point data. The industrial robot as a device for orientation has relatively low accuracy but the advantage of being numerically controlled, fast, flexible, rather cheap and compatible with industrial environments. It is therefore of interest to investigate if it can be used in this application.</p><p>This thesis will describe a measuring system consisting of a laser profile scanner mounted on an industrial robot with a turntable. It will also give an introduction to Geometric Reverse Engineering (GRE) and describe an automatic GRE process using this measuring system. The thesis also presents a detailed accuracy analysis supported by experiments that show how 2D profile data can be used to achieve a higher accuracy than the basic accuracy of the robot. The core topic of the thesis is the investigation of a new technique for planar segmentation. The new method is implemented in the GRE system and compared with an implementation of a more traditional method.</p><p>Results from practical experiments show that the new method is much faster while equally accurate or better.</p>

This thesis details the development and validation of a laser Doppler vibrometer-based measurement system that is capable of quantifying not only the normal vibration of a solid body but also the component of vibration that is parallel to the plane containing the surface of interest. LDV manufacturers produce various devices that capture 3D measurements in a beam configuration that can be used to decompose the measured signals into not only the normal velocity of the surface of an object but also two orthogonal in-plane components of the vibration. It was a slightly simplified two-dimensional version of this approach that was chosen for implementation in the Wave Physics Lab using individual components to create a cheaper, more flexible system than those produced by companies such as Polytec. The goal of this system is to facilitate the exploration and discovery of areas and applications where 2D measurement may provide a more complete and precise view of the physics of different phenomena. Upon completion of the system development and validation, a study was done that sought to measure the acoustoelastic effect in an Aluminum plate by measuring how increasing loads alter both the normal and in-plane components of Lamb wave propagation in the plate. The acoustoelastic effect is the effect of stress on wave speed caused by non-linearity in the propagation medium.

Fringe-pattern projection (FPP) techniques are commonly used for surface-shape measurement in a wide range of applications including object and scene modeling, part inspection, and reverse engineering. Periodic intensity fringe patterns with a specific amplitude are projected by the projector onto an object and a camera captures images of the fringe patterns, which appear distorted by the object surface from the perspective of the camera. The images are then used to compute the height or depth of the object at each pixel. One of the problems with FPP is that camera sensor saturation may occur if there is a large change in ambient lighting or a large range in surface reflectivity when measuring object surfaces. Camera sensor saturation occurs when the reflected intensity exceeds the maximum quantization level of the camera. A low SNR occurs when there is a low intensity modulation of the fringe pattern compared to the amount of noise in the image. Camera sensor saturation and low SNR can result in significant measurement error. Careful selection of the camera aperture or exposure time can reduce the error due to camera sensor saturation or low SNR. However, this is difficult to perform automatically, which may be necessary when measuring objects in uncontrolled environments where the lighting may change and objects have different surface reflectivity. This research presents three methods to avoid camera sensor saturation when measuring surfaces subject to changes in ambient lighting and objects with a large range in reflectivity. All these methods use the same novel approach of lowering the maximum input gray level (MIGL) to the projector for saturation avoidance. This approach avoids saturation by lowering the reflected intensity so that formerly saturated intensities can be captured by the camera. The first method of saturation avoidance seeks a trade-off between robustness to intensity saturation and low SNR. Measurements of a flat white plate at different MIGL resulted in a trade-off MIGL that yielded the highest accuracy for a single adjustment of MIGL that is uniform within and across the projected images. The second method used several sets of images, taken at constant steps of MIGL, and combined the images pixel-by-pixel into a single set of composite images, by selecting the highest unsaturated intensities at each pixel. White plate measurements using this method had comparable accuracy to the first method but required more images to form the composite image. Measurement of a checkerboard showed a higher accuracy than the first method since the second method maintains a higher SNR when the object has a large range of reflectivity. The last method also used composite images where the step size was determined dynamically, based on the estimated percentage of pixels that would become unsaturated at the next step. In measurements of a flat white plate and a checkerboard the dynamic step size was found to add flexibility to the measurement system compared to the constant steps using the second method. Using dynamic steps, the measurement system was able to measure objects with either a low or high range of reflectivity with high accuracy and without manually adjusting the step size. This permits fully automated measurement of unknown objects with variable reflectivity in unstructured environments with changing lighting conditions. The methods can be used for measurement in uncontrolled environments, for specular surfaces, and those with a large range of reflectivity or luminance. This would allow a wider range of measurement applications using FPP techniques.

Laser scanners in combination with devices for accurate orientation like Coordinate Measuring Machines (CMM) are often used in Geometric Reverse Engineering (GRE) to measure point data. The industrial robot as a device for orientation has relatively low accuracy but the advantage of being numerically controlled, fast, flexible, rather cheap and compatible with industrial environments. It is therefore of interest to investigate if it can be used in this application. This thesis will describe a measuring system consisting of a laser profile scanner mounted on an industrial robot with a turntable. It will also give an introduction to Geometric Reverse Engineering (GRE) and describe an automatic GRE process using this measuring system. The thesis also presents a detailed accuracy analysis supported by experiments that show how 2D profile data can be used to achieve a higher accuracy than the basic accuracy of the robot. The core topic of the thesis is the investigation of a new technique for planar segmentation. The new method is implemented in the GRE system and compared with an implementation of a more traditional method. Results from practical experiments show that the new method is much faster while equally accurate or better.

<p dir="ltr">Over the last few decades, three-dimensional (3D) optical imaging and sensing techniques have attracted much attention from both academia and industries. Owing to its capability of gathering more information than conventional 2D imaging, it has been successfully adopted in many applications on the macro scale which ranges from sub-meters to meters such as entertainment, commercial electronics, manufacturing, and construction. For example, the iPhone “FaceID” sensor is used for facial recognition, and the Microsoft Kinect is used to track body motion in video games. With recent advances in many technical fields, such as semiconductor packaging, additive manufacturing, and micro-robots, there is an increasing need for microscopic 3D imaging, and several techniques including interferometry, confocal microscopy, focus variation, and structured light have been developed and adopted in these industries. Among these techniques, the structured light 3D imaging technique is considered one of the most promising techniques for in-situ metrology, owing to its advantage of simple configuration and high measurement speed. However, several challenges must be addressed in employing the structured-light 3D imaging technique in these fields.</p><p dir="ltr">The first challenge is the limited measurement range caused by the limited depth of field (DOF). Given the necessity for large magnification in the microscopic structured light system, the DOF becomes notably shallow, especially when pin-hole lenses are adopted. This issue is exacerbated by the fact that the measured objects in the aforementioned industries could contain miniaturized features spanning a broad height range. To address this problem, we introduce the idea of the focus stacking technique, wherein the focused pixels gathered from various focus settings are merged to form an all-in-focus image, into the structured-light 3D imaging. We further developed a computational framework that utilizes the phase information and fringe contrast of the projected fringe patterns to mitigate the influence of object textures.</p><p dir="ltr">The second challenge is the 3D imaging speed. The 3D measurement speed is a crucial factor for in-situ applications. We improved the large DOF 3D imaging speed by reducing the required fringe images from two aspects: 1) We developed a calibration method for multifocus pin-hole mode, which can eliminate the necessity of the 2D image alignment. The conventional method based on circle patterns will be affected during the feature extraction process by the significant camera defocusing. In contrast, our proposed method is more robust since it uses virtual features extracted from a reconstructed white flat surface under a pre-calibrated focus setting. 2)We developed a phase unwrapping method with the assistance of the electrically tunable lens (ETL), which is an optical component we used to capture fringe images under various focus settings. The proposed phase unwrapping method leverages the focal plane position of each focus setting to estimate a rough depth map for the geometric-constraint phase unwrapping algorithm. By doing this, the method eliminates the limitation on the effective working depth range and becomes feasible in large DOF 3D imaging.</p><h4>Even with all previous methodologies, the efficiency of large DOF 3D imaging is still not high enough under certain circumstances. One of the major reasons is that we can still only use a series of pre-defined focus settings to run the focus stacking, since we have no prior on the measured objects. This issue could lead to low measurement efficiency when the depth range of the measured objects does not cover the whole enlarged DOF. To improve the performance of the system under such situations, we developed a method that introduces another computational imaging technique: the focal sweep technique, to help determine the optimal focus settings adapting to different measured objects.</h4><h4>In summary, this dissertation contributed to high-speed, large depth-of-field, and automated 3D imaging, which can be used in micro-scale applications from the following aspects: (1) enlarging the DOF of the microscopic 3D imaging using the focus stacking technique; (2) developing methods to improve the speed of large DOF microscopic 3D imaging; and (3) developing a method to improve the efficiency of the focus stacking under certain circumstances. These contributions can potentially enable the structured-light 3D imaging technique to be an alternative 3D microscopy approach for many academic studies and industry applications.</h4><p></p>

Thesis (MScEng (Mechanical and Mechatronic Engineering))--University of Stellenbosch, 2011. / ENGLISH ABSTRACT: The In the Physiotherapy Division of the Faculty of Health Sciences on the Tygerberg Hospital Campus of the University of Stellenbosch, the challenge arose to develop a portable, affordable and yet accurate 3D measurement machine for the assessment of posture in school children in their classroom environment. Currently Division already uses a state-of-the-art VICON commercial medical measuring machine to measure human posture in 3D in their physiotherapy clinic, but the system is not portable and is too expensive to cart around to different places for testing. To respond to this challenge, this Master’s thesis designed and analyzed a machine and its supporting system through both research on stereo-vision methodologies and empirical appraisal in the field. In the development process, the research was required to overcome the limitations posed by small image resolutions and lens distortions that are typical of cheap cameras. The academic challenge lay in the development of an error prediction model through Jacobian derivation and Error Propagation Law, to predict uncertainties of angular measurement calculated by the system. The research culminated in a system that is comparable in accuracy to the VICON within 3mm, and that has 1.5mm absolute accuracy within its own system for a measurement volume radius of 2.5 m. As such, the developed error model is an exact predictor of the angular error to within 0.02° of arc. These results, for both system accuracy and the error model, exceed the expectations on the basis of the initial challenge of the system. The development of the machine was successful in providing a prototype tool that is suitable for commercial development for use by physiotherapists in human posture measurement and assessment. In its current incarnation, the machine will also serve the Engineering Faculty as the most fundamental form of a three-dimensional measuring apparatus using only basic theories and algorithms of stereo-vision, thereby providing a basic experimental platform from which further scientific research on the theory and application of computer vision can be conducted. / AFRIKAANSE OPSOMMING: Die Fisioterapie Afdeling van die Fakulteit Gesondheidswetenskappe op die Tygerberg kampus van die Universiteit van Stellenbosch gebruik ’n allernuutste VICON kommersiële mediese meettoestel om menslike postuur in drie dimensies te meet. Vanuit hierdie Afdeling het die uitdaging ontstaan om ’n draagbare, bekostigbare, maar tog akkurate, drie-dimensionele meetapparaat geskik vir die meet van die postuur van skoolkinders in die klaskamer te ontwikkel. In aanvaarding van hierdie uitdaging, het hierdie Magistertesis ’n toestel en ondersteuningstels ontwerp en ontleed deur beide navorsing in stereo-visie metodiek en terplaatse beoordeling. In die ontwikkelingsproses moes die navorsing die beperkings wat deur klein-beeld resolusie en lens-distorsie (tipies van goedkoop kameras) meegebring word, oorkom. Die akademiese uitdaging lê in die ontwikkeling van ’n voorspellende foutmodel deur van die Jacobianse-afleiding en die Fout Propageringswet gebruik te maak om onsekerheid van hoeksberekening deur die stelsel te voorspel. Die navorsing het gelei tot ’n stelsel wat binne 3mm vergelykbaar is in akkuraatheid met dié van die VICON en ook 1.5mm absolute interne akkuraatheid het in ’n meet-volume radius van 2.5m radius. Die ontwikkelde foutmodel is dus ’n presiese voorspeller van hoekfout tot binne 0.02° van boog. Die resultate met betrekking tot beide die akkuraatheid en die foutmodel het die oorspronklike verwagtinge van die uitdaging oortref. Die ontwikkeling was suksesvol in die skep van ’n prototipe-toestel geskik vir kommersiële ontwikkeling, vir gebruik deur fisioterapeute in die meting en evaluering van menslike postuur. Die stelsel is in sy fundamentele vorm, deur die gebruik van slegs basiese teorieë en algoritmes van stereo-visie, funksioneer as ’n drie-dimensionele meetapparaat. In die fundamentele vorm sal die stelsel die Ingenieursfakulteit dien as ’n basiese eksperimentele platform waarop verdere wetenskaplike navorsing in die teorie en toepassing van rekenaar-visie gedoen kan word.

La majorité des dispositifs de métrologie basés vision sont équipés de systèmes optiques stéréo ou de systèmes de mesure externes dits actifs. Les méthodes de reconstruction tridimensionnelle (Structure-from-Motion, Shape-from-Shading) applicables à la vision monoculaire souffrent généralement de l’ambiguïté d’échelle. Cette dernière est inhérente au processus d’acquisition d’images qui implique la perte de l’information de profondeur de la scène. La relation entre la taille des objets et la distance de la prise de vue est équivoque.Cette étude a pour objet l’estimation de l’échelle absolue d’une scène par vision passive monofocale. Elle vise à apporter une solution à l’ambiguïté d’échelle uniquement basée vision, pour un système optique monoculaire dont les paramètres internes sont fixes. Elle se destine plus particulièrement à la mesure des lésions en coloscopie. Cette procédure endoscopique (du grec endom : intérieur et scopie : vision) permet l’exploration et l’intervention au sein du côlon à l’aide d’un dispositif flexible (coloscope) embarquant généralement un système optique monofocal. Dans ce contexte, la taille des néoplasies (excroissances anormales de tissu) constitue un critère diagnostic essentiel. Cette dernière est cependant difficile à évaluer et les erreurs d’estimations visuelles peuvent conduire à la définition d’intervalles de temps de surveillance inappropriés. La nécessité de concevoir un système d’estimation de la taille des lésions coloniques constitue la motivation majeure de cette étude. Nous dressons dans la première partie de ce manuscrit un état de l’art synoptique des différents systèmes de mesure basés vision afin de positionner notre étude dans ce contexte. Nous présentons ensuite le modèle de caméra monofocal ainsi que le modèle de formation d’image qui lui a été associé. Ce dernier est la base essentielle des travaux menés dans le cadre de cette thèse. La seconde partie du manuscrit présente la contribution majeure de notre étude. Nous dressons tout d’abord un état de l’art détaillé des méthodes de reconstruction 3D basées sur l’analyse de l’information de flou optique (DfD (Depth-from-Defocus) et DfF (Depth-from-Defocus)). Ces dernières sont des approches passives permettant, sous certaines contraintes d’asservissement de la caméra, de résoudre l’ambiguïté d’échelle. Elles ont directement inspiré le système de mesure par extraction du point de rupture de netteté présenté dans le chapitre suivant. Nous considérons une vidéo correspondant à un mouvement d’approche du système optique face à une région d’intérêt dont on souhaite estimer les dimensions. Notre système de mesure permet d’extraire le point de rupture nette/flou au sein de cette vidéo. Nous démontrons que, dans le cas d’un système optique monofocale, ce point unique correspond à une profondeur de référence pouvant être calibrée. Notre système est composé de deux modules. Le module BET (Blur EstimatingTracking) permet le suivi et l’estimation conjointe de l’information de mise au point d’une région d’intérêt au sein d’une vidéo. Le module BMF (Blur Model Fitting) permet d’extraire de façon robuste le point de rupture de netteté grâce à l’ajustement d’un modèle de flou optique. Une évaluation de notre système appliqué à l’estimation de la taille des lésions coloniques démontre sa faisabilité. Le dernier chapitre de ce manuscrit est consacré à une perspective d’extension de notre approche par une méthode générative. Nous présentons, sous la forme d’une étude théorique préliminaire, une méthode NRSfM (Non-Rigid Structure-from-Motion) permettant la reconstruction à l’échelle de surfaces déformables. Cette dernière permet l’estimation conjointe de cartes de profondeurs denses ainsi que de l’image de la surface aplanie entièrement mise au point. (...) / Vision-based metrology devices generally embed stereoscopic sensors or active measurement systems. Most of the passive 3D reconstruction techniques (Structure-from-Motion, Shape from-Shading) adapted to monocular vision suffer from scale ambiguity. Because the processing of image acquisition implies the loss of the depth information, there is an ambiguous relationship between the depth of a scene and the size of an imaged object. This study deals with the estimation of the absolute scale of a scene using passive monofocal vision. Monofocal vision describes monocular system for which optical parameters are fixed. Such optical systems are notably embedded within endoscopic systems used in colonoscopy. This minimally invasive technique allows endoscopists to explore the colon cavity and remove neoplasias (abnormal growths of tissue). Their size is an essential diagnostic criterion for estimating their rate of malignancy. However, it is difficult to estimate and erroneous visual estimations lead to neoplasias surveillance intervals being inappropriately assigned. The need to design a neoplasia measurement system is the core motivation for our study. In the first part of this manuscript, we review state-of-the-art vision-based metrology devices to provide context for our system. We then introduce monofocal optical systems and the specific image formation model used in our study. The second part deals with the main contribution of our work. We first review in detail state of the art DfD (Depth-from-Defocus) and DfF (Depth-from-Defocus) approaches. They are passive computer vision techniques that enable us to resolve scale ambiguity. Our core contribution is introduced in the following chapter. We define the Infocus-Breakpoint (IB) that allows us to resolve scale from a regular video. The IB is the lower limit of the optical system’s depth of field. Our system relies on two novel technical modules: Blur-Estimating Tracking (BET) and Blur-Model Fitting (BMF). BET allows us to simultaneously track an area of interest and estimate the optical blur information. BMF allows us to robustly extract the IB by fitting an optical blur model to the blur measurement estimated by the BET module. For the optical system is monofocal, the IB corresponds to a reference depth that can be calibrated. In the last chapter, we evaluate our method and propose a neoplasia measurement system adapted to the constraints in colonoscopy examination. The last part of this manuscript is dedicated to a prospect of extension of our method by a generative approach. We present, as a preliminary study, a new NRSfM (Non-Rigid Structure-from-Motion) method allowing the scaled Euclidean 3D reconstruction of deformable surfaces. This approach is based on the simultaneous estimation of dense depth maps corresponding to a set of deformations as well as the in-focus color map of the flattened surface. We first review state-of-the-art methods for 3D reconstruction of deformable surfaces. We then introduce our new generative model as well as an alternation method allowing us to infer it.

Cette thèse consiste en la conception d'un système d'acquisition et d'un algorithme de traitement d'image. Le but de ce travail est la détection de contaminants dans un contexte de contrôle qualité, particulièrement dans l'industrie pharmaceutique. Les contaminants sont des colonies de micro-organismes se développant sur membrane micro-poreuse. Nous avons choisi d'utiliser la mesure tridimensionnelle de surface pour réaliser l'acquisition des données, ce qui n'a jamais été fait pour des données micro-biologiques. Notre apport a de plus consisté à remplacer l'éclairage laser généralement utilisé par un dispositif à LED permettant de réduire le bruit dans les données. Cela permet de diminuer la durée d'incubation des tests de 14 jours à moins de 5. Concernant l'algorithme, nous avons analysé les données de hauteur en combinant une méthode de détection de données aberrantes et un séparateur à vaste marge. La difficulté de la détection réside dans la variété des signaux correspondant aux colonies, et également dans la présence d'artefacts semblables aux colonies. Nous sommes capables de détecter correctement la présence ou l'absence de contaminants dans 98% des cas. / An acquisition system and its algorithm are designed. Their purpose is contaminants detection as quality control in pharmaceutical industry. Contaminants are colonies of micro-organisms growing on micro-porous membrane. We use 3D surface measurement, which has never been done in a microbiological context. In addition, our contribution is to use an LED based lighting instead of a laser. It leads to an important noise reduction. It allows to decrease micro-organisms incubation period from 14 days in current method to 5 days or less. The height map from the system are processed with an outlier detection method combined to a support vector machine. Colonies show varying and various signals, and artifacts are present in the data. Nevertheless, we have been able to detect with success the presence or absence of contaminants with a rate of 98%.

By developing optical measurement technologies of shapes and elements, it is now possible to measure the carbody directly in the production flow of the welding company's line for the production of cars, the so-called „In-line measurement“. When deploying this control process, you also need to process the amount of data measured using the standard described procedure for process control. The thesis presents a proposal for the introduction and control of the dimensional inspection process of the final product of the welded body.