This master thesis project was done together with Saab Dynamics in Linköping the spring of 2022 and aims to perform an online IMU-camera calibration using an AprilTag board. Experiments are conducted on two different types of datasets, the public dataset Euroc and internal datasets from Saab. The calibration is done iteratively by solving a series of nonlinear optimization problems without any initial knowledge of the sensor configuration. The method is largely based on work by Huang and collaborators. Other than just finding the transformation between the IMU and the camera, the biases in the IMU, and the time delay between the two sensors are also explored. By comparing the resulting transformation with Kalibr, the current state of the art offline calibration toolbox, it is possible to conclude that the model can find and correct for the biases in the gyroscope. Therefore it is important to include these biases in the model. The model is able to roughly find the time shift between the two sensors but has more difficulties correcting for it. The thesis also aims to explore ways of compiling a good dataset for calibration. Results show that it is desirable to avoid rapid movements as well as images gathered at distances from the AprilTag board that very a lot. Also, having a shorter exposure time is useful to not lose AprilTag detections.

Most current machine vision systems for industrial inspection were developed with one specific task in mind. Due to the requirement for real-time operation, these systems are typically implemented in special purpose hardware that performs very specific operations. Hence, these systems inflexible in the sense that they cannot easily be adapted to other applications. However, current trends in computer technology suggests that low-cost general-purpose computers will be available in the very near future that are fast enough to meet the speed requirements of many industrial inspection problems. If this low-cost computing power is to be effectively utilized on industrial inspection problems, more general-purpose vision systems must be developed, vision systems that can be easily adapted to a variety of applications. Unfortunately, little research has gone into creating such general-purpose industrial inspection systems. In this dissertation, a general vision system framework has been developed that can be easily adapted to a variety of industrial web inspection problems. The objective of this system is to automatically locate and identify "defects" on the surface of the material being inspected. This framework is designed to be robust, to be flexible, and to be as computationally simple as possible. To assure robustness this framework employs a combined strategy of top-down and bottom-up control, hierarchical defect models, and uncertain reasoning methods. To make this framework flexible, a modular Blackboard framework is employed. To minimize computational complexity the system incorporates a simple multi-thresholding segmentation scheme, a fuzzy logic focus of attention mechanism for scene analysis operations, and a partitioning of knowledge that allows concurrent parallel processing during recognition. Based on the proposed vision system framework, a computer vision system for automated lumber grading has been developed. The purpose of this vision system is to locate and identify grading defects on rough hardwood lumber in a species independent manner. This problem seems to represent one of the more difficult and complex web inspection problems. The system has been tested on approximately 100 boards from several different species. Three different methods for performing label verification were tested and compared. These are a rule-based approach, a k-nearest neighbor approach, and a neural network approach. The results of these tests together with other considerations suggest that the neural network approach is the better choice and hence is the one selected for use in the vision system framework. Also, a new back-propagation learning algorithm using a steep activation function was developed that is much faster and more stable than the standard back-propagation learning algorithm. This algorithm was designed to speed the learning process involved in training a neural network to do label verification. However this algorithm seems to have general applicability. / Ph. D.

<p>Automatic brain tumor segmentation in Magnetic Resonance Imaging scans is essential for the diagnosis, treatment, and surgery of cancerous tumors. However, identifying the hardly detectable tumors poses a considerable challenge, which are usually of different sizes, irregular shapes, and vague invasion areas. Current advancements have not yet fully leveraged the dynamics in the multiple modalities of MRI, since they usually treat multi-modality as multi-channel, and the early channel merging may not fully reveal inter-modal couplings and complementary patterns. In this thesis, we propose a novel deep cross-attention learning algorithm that maximizes the subtle dynamics mining from each of the input modalities and then boosts feature fusion capability. More specifically, we have designed a Multimodal Cross-Attention Module (MM-CAM), equipped with a 3D Multimodal Feature Rectification and Feature Fusion Module. Extensive experiments have shown that the proposed novel deep learning architecture, empowered by the innovative MM- CAM, produces higher-quality segmentation masks of the tumor subregions. Further, we have enhanced the algorithm with image matting refinement techniques. We propose to integrate a Progressive Refinement Module (PRM) and perform Cross-Subregion Refinement (CSR) for the precise identification of tumor boundaries. A Multiscale Dice Loss was also successfully employed to enforce additional supervision for the auxiliary segmentation outputs. This enhancement will facilitate effectively matting-based refinement for medical image segmentation applications. Overall, this thesis, with deep learning, transformer-empowered pattern mining, and sophisticated architecture designs, will greatly advance deep brain dynamics and images mining for tumor detection and precision medicine.</p>

<p>Accurate 3D landscape models of cities or mountains have wide applications in mission</p> <p>planning, navigation, geological studies, etc. Lidar scanning using drones can provide high</p> <p>accuracy 3D landscape models, but the data is more expensive to collect as the area of</p> <p>each scan is limited. Thanks to recent maturation of Very-High-Resolution (VHR) optical</p> <p>imaging on satellites, people nowadays have access to stereo images that are collected on a</p> <p>much larger area than Lidar scanning. My research addresses unique challenges in satellite</p> <p>stereo, including stereo rectification with pushbroom sensors, dense stereo matching using</p> <p>image pairs with varied appearance, e.g. sun angles and surface plantation, and rasterized</p> <p>digital surface model (DSM) generation. The key contributions include the Continuous 3D-</p> <p>Label Semi-Global Matching (CoSGM) and a large scale dataset for satellite stereo processing</p> <p>and DSM evaluation.</p>

In the field of cell biology, there is an increasing use of time-lapse data to understand cellular function. Using automated microscopes, large numbers of images can be acquired, delivering videos of cell samples over time. Analysing the images manually is extremely time consuming as there are typically thousands of individual images in any given sequence. Additionally, decisions made by those analysing the images, e.g. labelling a mitotic phase (one of a set of distinct sequential stages of cell division) can be subjective, especially around transition boundaries between phases, leading to inconsistencies in the annotation. There is therefore a need for tools which facilitate automated high-throughput analysis. In this thesis we develop systems to automatically detect, track and analyse sub-cellular structures in image sequences to address biological research needs in three areas: (i) Mitotic phase labelling, (ii) Mitotic defect detection, and (iii) Cell volume estimation. We begin by presenting a system for automated segmentation and mitotic phase labelling using temporal models. This work takes the novel approach of using temporal features evaluated over the whole of the mitotic phases rather than over single frames, thereby capturing the distinctive behaviour over the phases. We compare and contrast three different temporal models: Dynamic Time Warping, Hidden Markov Models, and Semi Markov Models. A new loss function is proposed for the Semi Markov model to make it more robust to inconsistencies in data annotation near transition boundaries. We then present an approach for detecting subtle chromosome segregation errors in mitosis in embryonic stem cells, targeting two cases: misaligned chromosomes in a metaphase cell, and lagging chromosomes between anaphase cells. We additionally explore an unsupervised approach to detect unusual mitotic occurrences and test its applicability to detecting misaligned metaphase chromosomes. Finally, we describe a fully automated method, suited to high-throughput analysis, for estimating the volume of spherical mitotic cells based on a learned membrane classifier and a circular Hough transform. We also describe how it is being used further in biological research.

Ο ταχύτατος ρυθμός εξέλιξης της επιστήμης των υπολογιστών τις τελευταίες δεκαετίες είχε σαν αποτέλεσμα την επέκταση της χρήσης των υπολογιστών σε διάφορους τομείς της καθημερινής μας ζωής, από ένα συνεχώς αυξανόμενο αριθμό ανθρώπων. Ωστόσο παρατηρούνται ακόμα αρκετές δυσκολίες στον τρόπο χειρισμού διάφορων υπολογιστικών συστημάτων, γεγονός που προσανατολίζει την έρευνα στην ανάπτυξη συστημάτων των οποίων η χρήση βασίζεται στα "φυσικά" μέσα επικοινωνίας που χρησιμοποιούνται από τον άνθρωπο, όπως για παράδειγμα οι χειρονομίες. Αντικείμενο της παρούσας ειδικής επιστημονικής εργασίας αποτελεί η διερεύνηση και υλοποίηση ενός συστήματος αναγνώρισης ανθρώπινων χειρονομιών σε ακολουθίες εικόνων (video), με χρήση τεχνικών υπολογιστικής όρασης. Κατόπιν μιας σύντομης αναφοράς στην επικοινωνία ανθρώπου-υπολογιστή (ΕΑΥ), ερευνώνται εκτενώς τρία πεδία της όρασης υπολογιστών: το χρώμα, και ειδικότερα η χρήση του χρώματος για την κατάτμηση μιας εικόνας, η τεχνική της σύμπτωσης προτύπων (template matching) για αναζήτηση πρωτοτύπων εικόνων σε άλλες εικόνες και η μέθοδος αναγνώρισης κινούμενων αντικειμένων. Οι μεθοδολογίες αυτές συνδυάζονται για την ανάπτυξη του συστήματος αναγνώρισης χειρονομιών, το οποίο μπορεί να χρησιμοποιηθεί σε εφαρμογές όπως η αλληλεπίδραση με ηλεκτρονικούς υπολογιστές, σε κονσόλες βιντεοπαιχνιδιών και διάφορων συσκευών που χρησιμοποιούμε καθημερινά, όπως η τηλεόραση και το κινητό τηλέφωνο. / The enormous rate of evolution of computer science in recent decades has resulted in expanding the use of computers in more and more areas of our everyday life, by more and more people. However there are still numerous difficulties in using various computer systems, therefore the research is oriented to the development of the use of which is based on more "natural" means that people use to communicate with each other, such as gestures. The subject of this master thesis is the study and development of a system for the recognition of human gestures in image sequences (video), using computer vision techniques. After a brief mention of Human-Computer Interaction extensively investigated three areas of computer vision: color, particularly the use of color for the segmentation of an image, the technique of template matching to search prototype images other images and recognition method of moving objects. These methodologies combine the development of gesture recognition system, which can be used in applications such as computer interaction, video game consoles and various devices that we use every day, including television and mobile phone.

published_or_final_version / Electrical and Electronic Engineering / Doctoral / Doctor of Philosophy

published_or_final_version / Computer Science / Master / Master of Philosophy

Visual surveillance systems are increasingly common in our society today. There is a conflict between the demands for security of the public and the demands to preserve the personal integrity. This thesis suggests a solution in which parts of the surveillance images are covered in order to conceal the identities of persons appearing in video, but not their actions or activities. The covered parts could be encrypted and unlocked only by the police or another legal authority in case of a crime. This thesis implements a proof-of-concept demonstrator using a combination of image processing techniques such as foreground segmentation, mathematical morphology, geometric camera calibration and region tracking. The demonstrator is capable of tracking a moderate number of moving objects and conceal their identity by replacing them with a mask or a blurred image. Functionality for replaying recorded data and unlocking individual persons are included. The concept demonstrator shows the chain from concealing the identities of persons to unlocking only a single person on recorded data. Evaluation on a publicly available dataset shows overall good performance.

Insects posses an incredible ability to navigate their environment at high speed, despite having small brains and limited visual acuity. Through selective pressure they have evolved computationally efficient means for simultaneously performing navigation tasks and instantaneous control responses. The insect’s main source of information is visual, and through a hierarchy of processes this information is used for perception; at the lowest level are local neurons for detecting image motion and edges, at the higher level are interneurons to spatially integrate the output of previous stages. These higher level processes could be considered as models of the insect's environment, reducing the amount of information to only that which evolution has determined relevant. The scope of this thesis is experimenting with biologically inspired visual control of flying robots through information processing, models of the environment, and flight behaviour. In order to test these ideas I developed a custom quadrotor robot and experimental platform; the 'wasp' system. All algorithms ran on the robot, in real-time or better, and hypotheses were always verified with flight experiments. I developed a new optical flow algorithm that is computationally efficient, and able to be applied in a regular pattern to the image. This technique is used later in my work when considering patterns in the image motion field. Using optical flow in the log-polar coordinate system I developed attitude estimation and time-to-contact algorithms. I find that the log-polar domain is useful for analysing global image motion; and in many ways equivalent to the retinotopic arrange- ment of neurons in the optic lobe of insects, used for the same task. I investigated the role of depth in insect flight using two experiments. In the first experiment, to study how concurrent visual control processes might be combined, I developed a control system using the combined output of two algorithms. The first algorithm was a wide-field optical flow balance strategy and the second an obstacle avoidance strategy which used inertial information to estimate the depth to objects in the environment - objects whose depth was significantly different to their surround- ings. In the second experiment I created an altitude control system which used a model of the environment in the Hough space, and a biologically inspired sampling strategy, to efficiently detect the ground. Both control systems were used to control the flight of a quadrotor in an indoor environment. The methods that insects use to perceive edges and control their flight in response had not been applied to artificial systems before. I developed a quadrotor control system that used the distribution of edges in the environment to regulate the robot height and avoid obstacles. I also developed a model that predicted the distribution of edges in a static scene, and using this prediction was able to estimate the quadrotor altitude.