Recently, sensors such as radars and cameras have been widely used in automotives, especially in Advanced Driver-Assistance Systems (ADAS), to collect information about the vehicle's surroundings. Stereo cameras are very popular as they could be used passively to construct a 3D representation of the scene in front of the car. This allowed the development of several ADAS algorithms that need 3D information to perform their tasks. One interesting application is Road Surface Preview (RSP) where the task is to estimate the road height along the future path of the vehicle. An active suspension control unit can then use this information to regulate the suspension, improving driving comfort, extending the durabilitiy of the vehicle and warning the driver about potential risks on the road surface. Stereo cameras have been successfully used in RSP and have demonstrated very good performance. However, the main disadvantages of stereo cameras are their high production cost and high power consumption. This limits installing several ADAS features in economy-class vehicles. A less expensive alternative are monocular cameras which have a significantly lower cost and power consumption. Therefore, this thesis investigates the possibility of solving the Road Surface Preview task using a monocular camera. We try two different approaches: structure-from-motion and Convolutional Neural Networks.The proposed methods are evaluated against the stereo-based system. Experiments show that both structure-from-motion and CNNs have a good potential for solving the problem, but they are not yet reliable enough to be a complete solution to the RSP task and be used in an active suspension control unit.

The thesis work evaluates a method to estimate the volume of stone and gravelpiles using only a cellphone to collect video and sensor data from the gyroscopesand accelerometers. The project is commissioned by Escenda Engineering withthe motivation to replace more complex and resource demanding systems with acheaper and easy to use handheld device. The implementation features popularcomputer vision methods such as KLT-tracking, Structure-from-Motion, SpaceCarving together with some Sensor Fusion. The results imply that it is possible toestimate volumes up to a certain accuracy which is limited by the sensor qualityand with a bias. / I rapporten framgår hur volymen av storskaliga objekt, nämligen grus-och stenhögar,kan bestämmas i utomhusmiljö med hjälp av en mobiltelefons kamerasamt interna sensorer som gyroskop och accelerometer. Projektet är beställt avEscenda Engineering med motivering att ersätta mer komplexa och resurskrävandesystem med ett enkelt handhållet instrument. Implementationen använderbland annat de vanligt förekommande datorseendemetoderna Kanade-Lucas-Tommasi-punktspårning, Struktur-från-rörelse och 3D-karvning tillsammans medenklare sensorfusion. I rapporten framgår att volymestimering är möjligt mennoggrannheten begränsas av sensorkvalitet och en bias.

The energy consumption to make the off-chip memory writing and readings are aknown problem. In the image processing field structure from motion simpler compressiontechniques could be used to save energy. A balance between the detected features suchas corners, edges, etc., and the degree of compression becomes a big issue to investigate.In this thesis a deeper study of this balance are performed. A number of more advancedcompression algorithms for processing of still images such as JPEG is used for comparisonwith a selected number of simpler compression algorithms. The simpler algorithms canbe divided into two categories: individual block-wise compression of each image andcompression with respect to all pixels in each image. In this study the image sequences arein grayscale and provided from an earlier study about rolling shutters. Synthetic data setsfrom a further study about optical flow is also included to see how reliable the other datasets are. / Energikonsumtionen för att skriva och läsa till off-chip minne är ett känt problem. Inombildbehandlingsområdet struktur från rörelse kan enklare kompressionstekniker användasför att spara energi. En avvägning mellan detekterade features såsom hörn, kanter, etc.och grad av kompression blir då en fråga att utreda. I detta examensarbete har en djuparestudie av denna avvägning utförts. Ett antal mer avancerade kompressionsalgoritmer förbearbetning av stillbilder som tex. JPEG används för jämförelse med ett antal utvaldaenklare kompressionsalgoritmer. De enklare algoritmerna kan delas in i två kategorier:individuell blockvis kompression av vardera bilden och kompression med hänsyn tillsamtliga pixlar i vardera bilden. I studien är bildsekvenserna i gråskala och tillhandahållnafrån en tidigare studie om rullande slutare. Syntetiska data set från ytterligare en studie om’optical flow’ ingår även för att se hur pass tillförlitliga de andra dataseten är.

Estimating depth and detection of object instances in 3D space is fundamental in autonomous navigation, localization, and mapping, robotic object manipulation, and augmented reality. RGB-D images and LiDAR point clouds are the most illustrative formats of depth information. However, depth sensors offer many shortcomings, such as low effective spatial resolutions and capturing of a scene from a single perspective. The thesis focuses on reproducing denser and comprehensive 3D scene structure for given monocular RGB images using depth and 3D object detection. The first contribution of this thesis is the pipeline for the depth estimation based on an unsupervised learning framework. This thesis proposes two architectures to analyze structure from motion and 3D geometric constraint methods. The proposed architectures trained and evaluated using only RGB images and no ground truth depth data. The architecture proposed in this thesis achieved better results than the state-of-the-art methods. The second contribution of this thesis is the application of the estimated depth map, which includes two algorithms: point cloud generation and collision avoidance. The predicted depth map and RGB image are used to generate the point cloud data using the proposed point cloud algorithm. The collision avoidance algorithm predicts the possibility of collision and provides the collision warning message based on decoding the color in the estimated depth map. This algorithm design is adaptable to different color map with slight changes and perceives collision information in the sequence of frames. Our third contribution is a two-stage pipeline to detect the 3D objects from a monocular image. The first stage pipeline used to detect the 2D objects and crop the patch of the image and the same provided as the input to the second stage. In the second stage, the 3D regression network train to estimate the 3D bounding boxes to the target objects. There are two architectures proposed for this 3D regression network model. This approach achieves better average precision than state-of-theart for truncation of 15% or fully visible objects and lowers but comparable results for truncation more than 30% or partly/fully occluded objects.

<p>Simultaneous localization and mapping (SLAM) is a general device localization technique that uses realtime sensor measurements to develop a virtualization of the sensor's environment while also using this growing virtualization to determine the position and orientation of the sensor. This is useful for augmented reality (AR), in which a user looks through a head-mounted display (HMD) or viewfinder to see virtual components integrated into the real world. Visual SLAM (i.e., SLAM in which the sensor is an optical camera) is used in AR to determine the exact device/headset movement so that the virtual components can be accurately redrawn to the screen, matching the perceived motion of the world around the user as the user moves the device/headset. However, many potential AR applications may need access to more than device localization data in order to be useful; they may need to leverage environment data as well. Additionally, most SLAM solutions make the naive assumption that the environment surrounding the system is completely static (non-moving). Given these circumstances, it is clear that AR may benefit substantially from utilizing a SLAM solution that detects objects that move in the scene and ultimately provides localization data for each of these objects. This problem is known as the dynamic SLAM problem. Current attempts to address the dynamic SLAM problem often use machine learning to develop models that identify the parts of the camera image that belong to one of many classes of potentially-moving objects. The limitation with these approaches is that it is impractical to train models to identify every possible object that moves; additionally, some potentially-moving objects may be static in the scene, which these approaches often do not account for. Some other attempts to address the dynamic SLAM problem also localize the moving objects they detect, but these systems almost always rely on depth sensors or stereo camera configurations, which have significant limitations in real-world use cases. This dissertation presents a novel approach for registering and localizing unknown moving objects in the context of markerless, monocular, keyframe-based SLAM with no required prior information about object structure, appearance, or existence. This work also details a novel deep learning solution for determining SLAM map initialization suitability in structure-from-motion-based initialization approaches. This dissertation goes on to validate these approaches by implementing them in a markerless, monocular SLAM system called LUMO-SLAM, which is built from the ground up to demonstrate this approach to unknown moving object registration and localization. Results are collected for the LUMO-SLAM system, which address the accuracy of its camera localization estimates, the accuracy of its moving object localization estimates, and the consistency with which it registers moving objects in the scene. These results show that this solution to the dynamic SLAM problem, though it does not act as a practical solution for all use cases, has an ability to accurately register and localize unknown moving objects in such a way that makes it useful for some applications of AR without thwarting the system's ability to also perform accurate camera localization.</p>

Harness racing horses are exposed to high workload and consequently, they are at risk of joint injuries and lameness. In recent years, the interest in applications to improve animal welfare has increased and there is a demand for objective assessment methods that can enable early and robust diagnosis of injuries. In this thesis, experiments were conducted on video recordings collected by a helmet camera mounted on the driver of a sulky. The aim was to take the first steps toward equine gait analysis by investigating how semantic segmentation and 3D reconstruction of such data could be performed. Since these were the first experiments made on this data, no expectations of the results existed in advance. Manual pixel-wise annotations were created on a small set of extracted frames and a deep learning model for semantic segmentation was trained to localize the horse, as well as the sulky and reins. The results are promising and could probably be further improved by expanding the annotated dataset and using a larger image resolution. Structure-from-motion using COLMAP was performed to estimate the camera motion in part of a video recording. A method to filter out dynamic objects based on masks created from predicted segmentation maps was investigated and the results showed that the reconstruction was part-wise successful, but struggled when dynamic objects were not filtered out and when the equipage was moving at high speed along a straight stretch. Overall the results are promising, but further development needs to be conducted to ensure robustness and conclude whether data collected by the investigated helmet camera configuration is suitable for equine gait analysis.

Corrosion damage reduces the load-carrying capacity of bridges which poses a threat to passenger safety. The objective of this research was to reduce the resources involved in conventional bridge inspections which are an important tool in the condition assessment of bridges and to help in determining if live load testing is necessary. This research proposes a framework to link semi-automated damage detection on prestressed concrete bridge girders with the estimation of their residual flexural capacity. The framework was implemented on four full-scale corrosion damaged girders from decommissioned bridges in Virginia. 3D point clouds of the girders reconstructed from images using Structure from Motion (SfM) approach were textured with images containing cracks detected at pixel level using a U-Net (Fully Convolutional Network). Spalls were detected by identifying the locations where normals associated with the points in the 3D point cloud deviated from being perpendicular to the reference directions chosen, by an amount greater than a threshold angle. 3D textured mesh models, overlaid with the detected cracks and spalls were used as 3D damage maps to determine reduced cross-sectional areas of prestressing strands to account for the corrosion damage as per the recommendations of Naito, Jones, and Hodgson (2011). Scaling them to real-world dimensions enabled the measurement of any required dimension, eliminating the need for physical contact. The flexural capacities of a box beam and an I-beam estimated using strain compatibility analysis were validated with the actual capacities at failure sections determined from four destructive tests conducted by Al Rufaydah (2020). Along with the reduction in the cross-sectional areas of strands, limiting the ultimate strain that heavily corroded strands can develop was explored as a possible way to improve the results of the analysis. Strain compatibility analysis was used to estimate the ultimate rupture strain, in the heavily corroded bottommost layer prestressing strands exposed before the box beam was tested. More research is required to associate each level of strand corrosion with an average ultimate strain at which the corroded strands rupture. This framework was found to give satisfactory estimates of the residual strength. Reduction in resources involved in current visual inspection practices and eliminating the need for physical access, make this approach worthwhile to be explored further to improve the output of each step in the proposed framework. / Master of Science / Corrosion damage is a major concern for bridges as it reduces their load carrying capacity. Bridge failures in the past have been attributed to corrosion damage. The risk associated with corrosion damage caused failures increases as the infrastructure ages. Many bridges across the world built forty to fifty years ago are now in a deteriorated condition and need to be repaired and retrofitted. Visual inspections to identify damage or deterioration on a bridge are very important to assess the condition of the bridge and determine the need for repairing or for posting weight restrictions for the vehicles that use the bridge. These inspections require close physical access to the hard-to-reach areas of the bridge for physically measuring the damage which involves many resources in the form of experienced engineers, skilled labor, equipment, time, and money. The safety of the personnel involved in the inspections is also a major concern. Nowadays, a lot of research is being done in using Unmanned Aerial Vehicles (UAVs) like drones for bridge inspections and in using artificial intelligence for the detection of cracks on the images of concrete and steel members. Girders or beams in a bridge are the primary longitudinal load carrying members. Concrete inherently is weak in tension. To address this problem, High Strength steel reinforcement (called prestressing steel or prestressing strands) in prestressed concrete beams is pre-loaded with a tensile force before the application of any loads so that the regions which will experience tension under the service loads would be subjected to a pre-compression to improve the performance of the beam and delay cracking. Spalls are a type of corrosion damage on concrete members where portions of concrete fall off (section loss) due to corrosion in the steel reinforcement, exposing the reinforcement to the environment which leads to accelerated corrosion causing a loss of cross-sectional area and ultimately, a rupture in the steel. If the process of detecting the damage (cracks, spalls, exposed or severed reinforcement, etc.) is automated, the next logical step that would add great value would be, to quantify the effect of the damage detected on the load carrying capacity of the bridges. Using a quantified estimate of the remaining capacity of a bridge, determined after accounting for the corrosion damage, informed decisions can be made about the measures to be taken. This research proposes a stepwise framework to forge a link between a semi-automated visual inspection and residual capacity evaluation of actual prestressed concrete bridge girders obtained from two bridges that have been removed from service in Virginia due to extensive deterioration. 3D point clouds represent an object as a set of points on its surface in three dimensional space. These point clouds can be constructed either using laser scanning or using Photogrammetry from images of the girders captured with a digital camera. In this research, 3D point clouds are reconstructed from sequences of overlapping images of the girders using an approach called Structure from Motion (SfM) which locates matched pixels present between consecutive images in the 3D space. Crack-like features were automatically detected and highlighted on the images of the girders that were used to build the 3D point clouds using artificial intelligence (Neural Network). The images with cracks highlighted were applied as texture to the surface mesh on the point cloud to transfer the detail, color, and realism present in the images to the 3D model. Spalls were detected on 3D point clouds based on the orientation of the normals associated with the points with respect to the reference directions. Point clouds and textured meshes of the girders were scaled to real-world dimensions facilitating the measurement of any required dimension on the point clouds, eliminating the need for physical contact in condition assessment. Any cracks or spalls that went unidentified in the damage detection were visible on the textured meshes of the girders improving the performance of the approach. 3D textured mesh models of the girders overlaid with the detected cracks and spalls were used as 3D damage maps in residual strength estimation. Cross-sectional slices were extracted from the dense point clouds at various sections along the length of each girder. The slices were overlaid on the cross-section drawings of the girders, and the prestressing strands affected due to the corrosion damage were identified. They were reduced in cross-sectional area to account for the corrosion damage as per the recommendations of Naito, Jones, and Hodgson (2011) and were used in the calculation of the ultimate moment capacity of the girders using an approach called strain compatibility analysis. Estimated residual capacities were compared to the actual capacities of the girders found from destructive tests conducted by Al Rufaydah (2020). Comparisons are presented for the failure sections in these tests and the results were analyzed to evaluate the effectiveness of this framework. More research is to be done to determine the factors causing rupture in prestressing strands with different degrees of corrosion. This framework was found to give satisfactory estimates of the residual strength. Reduction in resources involved in current visual inspection practices and eliminating the need for physical access, make this approach worthwhile to be explored further to improve the output of each step in the proposed framework.

Tato práce popisuje problematiku návrhu a vývoje aplikace pro rekonstrukci 3D modelů z 2D obrazových dat, označované jako bundle adjustment. Práce analyzuje proces 3D rekonstrukce a důkladně popisuje jednotlivé kroky. Prvním z kroků je automatizované získání obrazové sady z internetu. Je představena sada skriptů pro hromadné stahování obrázků ze služeb Flickr a Google Images a shrnuty požadavky na tyto obrázky pro co nejlepší 3D rekonstrukci. Práce dále popisuje různé detektory, extraktory a párovací algoritmy klíčových bodů v obraze s cílem najít nejvhodnější kombinaci pro rekonstrukci budov. Poté je vysvětlen proces rekonstrukce 3D struktury, její optimalizace a jak je tato problematika realizovaná v našem programu. Závěr práce testuje výsledky získané z implementovaného programu pro několik různých datových sad a porovnává je s výsledky ostatních podobných programů, představených v úvodu práce.

In this dissertation a novel alternative system concept for monocular-inertial egomotion and landmark position determination is introduced. It is mainly motivated by an in-depth analysis of the observability and consistency of the classic simultaneous localization and mapping (SLAM) approach, which is based on a world-centric model of an agent and its environment. Within the novel system concept - a body-centric agent and environment model, - a pseudo-world centric motion propagation, - and closed-form initialization procedures are introduced. This approach allows for combining the advantageous observability properties of body-centric modeling and the advantageous motion propagation properties of world-centric modeling. A consistency focused and simulation based evaluation demonstrates the capabilities as well as the limitations of the proposed concept. / In dieser Dissertation wird ein neuartiges, alternatives Systemkonzept für die monokular-inertiale Eigenbewegungs- und Landmarkenpositionserfassung vorgestellt. Dieses Systemkonzept ist maßgeblich motiviert durch eine detaillierte Analyse der Beobachtbarkeits- und Konsistenzeigenschaften des klassischen Simultaneous Localization and Mapping (SLAM), welches auf einer weltzentrischen Modellierung eines Agenten und seiner Umgebung basiert. Innerhalb des neuen Systemkonzeptes werden - eine körperzentrische Modellierung des Agenten und seiner Umgebung, - eine pseudo-weltzentrische Bewegungspropagation, - und geschlossene Initialisierungsprozeduren eingeführt. Dieser Ansatz erlaubt es, die günstigen Beobachtbarkeitseigenschaften körperzentrischer Modellierung und die günstigen Propagationseigenschaften weltzentrischer Modellierung zu kombinieren. Sowohl die Fähigkeiten als auch die Limitierungen dieses Ansatzes werden abschließend mit Hilfe von Simulationen und einem starken Fokus auf Schätzkonsistenz demonstriert.

No description available.