University of Thessaly (UTH)
Algorithms for automated image and video analysis with applications
Abstract
dc:descriptionImage and Video processing and analysis applications are the core components of Computer Vision. Computer vision is a field of Artificial Intelligence, which has a broad variety of applications in fields including industry, healthcare, human-centric services and assistive environments. This work introduces computer vision applications on preventive medicine that can support monitoring and healthcare for the elderly and chronic ill, as well as doctors in medical reports. Furthermore, this work proposes methods for digitally reconstructing the volume of humans and objects, using a few camera views – which can be embedded in augmented reality applications, or utilized for assessing the human’s posture with silhouette-based pose estimation. Additionally, automated vision-based quality control techniques are presented for industrially manufactured rods that can be incorporated into production. In many computer vision methods, the more optical data available, the better for either the features extraction, or the calculations needed per case. For this reason, in this work, a special type of camera with 180o FoV, the fisheye camera, is exploited and a number of image and/or video processing algorithms are applied, using fisheye camera images and/or videos. The calibration of the fisheye camera is being revisited and applied to the available cameras of this type. Moreover, the calibration of projective camera is analyzed and applied on three projective cameras by a linear and a non-linear estimation of the cameras’ projection matrices. The space carving algorithm for reconstructing the volumetric model of an object/human using images of its silhouette is modified to include different combinations of fisheye and pinhole cameras. We create an experimental setup of several combinations of heterogeneous cameras, and use the calculated calibrations to generate synthetic data, as well as acquire real data, for the tasks of dynamic position estimation, as well as volumetric reconstruction. The classic space carving algorithm suffers from accuracy deterioration as the number of views increases. We propose a method to alleviate this problem. We also investigate the recognition of poses based on the volumetric models, in static and dynamic scenarios (moving objects along any path). Quantitative results are presented in reconstructing synthetic 3D human models and qualitative results for human participants. The results make apparent that the use of fisheye cameras, along with projective cameras, in space carving algorithm implementations reduces the reconstruction error. A methodology for the fall detection in indoor environment, utilizing three uncalibrated fisheye cameras, which set the whole room to be imaged along any direction is presented. The proposed method requires simultaneous frame acquisition by three or more fisheye cameras facing approximately downwards and the segmentation of the silhouettes from the available frames, without the need for more precise camera calibration. The main advantages of this method are: the simplicity of the detecting rule and the speed of execution that stem from the exploitation of a property of omni-directional camera image formation model. In addition the use of fisheye cameras allows simultaneous imaging along any direction. The proposed algorithm is designed and parameterized using an extensive dataset of synthetic frames. Results from real videos are presented using frame statistics and event-based statistics. The proposed algorithm correctly detects fall events within standing or walking, as well as other non-falling activities. In another part of the work, we utilize our fisheye image formation model to modify geodesically the traditional Zernike moments image features (ZMI). The proposed GZMI are tested in the task of pose recognition from omni-directional images, in the form of classifying cropped parts of the image that contain the silhouette. Since it is possible to construct an extensive synthetic dataset with this method, it becomes possible to assess the ability of Convolutional Neural Networks to correctly classify silhouettes. More specifically, the hand-crafted GZMI are compared against the classic ZMI, as well the CNN, in terms of classifying silhouettes in three generic poses. The ability of Convolutional Neural Networks to correctly classify synthetically generated and real silhouettes is assessed, using an extensive synthetic training set of binary silhouettes. Finally CNNs are assessed with enhanced input from the GZMI descriptors at the first fully connected layer, combined with transfer learning (TL). Our initial results show that the CNN trained only on synthetic silhouettes are unable to correctly classify the 3 generic poses in silhouettes segmented from real videos. Similarly, the CNN enhanced with the fusion of GZMI at the FNN level, achieves low classification accuracy, although slightly better than the standard CNN. On the other hand, a simple classifier using the GZMI (Zernike moment invariants, defined using the geodetic properties of the calibration of the specific acquiring camera) achieves significantly better accuracy, although still not high enough to be practically useful. The GZMI features appear to be more immune to the artefacts induced by the segmentation than the CNN. The classification accuracy achieved with transfer learning is close to practically useful levels. Image features are extracted and used for the detection of clinical characteristics, namely linear structures, such as streaks in dermoscopy images. Furthermore, several well-established image processing filter banks are employed for hand-crafted features for image classification with traditional classifiers. The proposed methodologies use image filters to e.g. segment dark linear structures using Hessian-based multi-resolution image features, as well as Convolutional Neural Networks to classify RGB images. Subsequently we investigate the usefulness of these filters for augmenting the input of Convolutional Neural Networks (CNNs) for the task of classification at the image and superpixel level. Melanoma and nevus dermoscopy images classification is applied. The novelty of the proposed methods is the use of (a) these filters also for augmenting the input to CNNs, to increase the size of the subset-of-interest in a meaningful way, as well as (b) transfer learning, i.e. fine-tuning previously trained CNNs using the augmented dataset and (c) an interesting comparison between the hand-crafted streak-based features with the various deep learning architectures. Results show that the accuracy achieved by the CNN with the augmented input is consistently higher than that of the RGB image input, both in terms of different repetitions of the execution, as well as throughout the iterations of each repetition, provided that TL has been utilized. Finally, we present a simple and robust method for curvature measurement of a very commonly manufactured mechanical object, a metal rod. Our initial results show that the proposed method is capable of discriminating between normal and not acceptable objects, based on mono-occular video, without any special requirements for light conditions and experimental geometric setup. The method is computationally very efficient, achieving real-time execution on a low-end laptop.
Degree
thesis:*- Grantor dc:publisher
- University of Thessaly (UTH)
- Year dc:date
- 2023
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Kottari, Konstantina
- Κότταρη, Κωνσταντίνα
Subjects
dc:subject × 88- Τεχνητή νοημοσύνη
- Υπολογιστική όραση
- Διαγνωστικά συστήματα βασισμένα στην υπολογιστική όραση
- Ταξινόμηση εικόνων
- Αλγόριθμοι επεξεργασίας εικόνας και βίντεο
- Ιατρική απεικόνιση
- Απεικονιστική διάγνωση
- Ευρυγώνια κάμερα fisheye
- Μοντέλο παραγωγής εικόνας ευρυγώνιας κάμερας fisheye
- Βαθμονόμηση κάμερας
- Τμηματοποίηση εικόνας
- Τμηματοποίηση σιλλουέττας
- Επεξεργασία βίντεο
- Εκτίμηση θέσης
- Αναγνώριση στάσης/δραστηριότητας
- Εντοπισμός πτώσης ανθρώπου
- Όγκος από σιλλουέττες
- Ανακατασκευή τρισδιάστατου μοντέλου
- Ογκομετρικό μοντέλο
- Ανακατασκευή ογκομετρικού μοντέλου
- Σφάλμα ανακατασκευής
- Ανίχνευση γραμμικών δομών/ραβδώσεων
- Δερματοσκοπικές εικόνες
- Δερματική βλάβη
- Ταξινόμηση δερματικών βλαβών
- Ταξινόμηση σε μελάνωμα και σπίλους
- Αναγνώριση προτύπου
- Τοπικοί περιγραφείς εικόνας
- Αμετάβλητες ροπές Zernike
- Γεωδαισιακές αμετάβλητες ροπές Zernike
- Αμετάβλητες ροπές
- Συνελικτικό νευρωνικό δίκτυο
- Αρχιτεκτονική συνελικτικών νευρωνικών δικτύων
- Μεταφορά εκμάθησης
- Φίλτρα επεξεργασίας εικόνας
- Υπερεικονοστοιχείο
- Επεξεργασία σήματος πραγματικού χρόνου
- Μέτρηση σε πραγματικό χρόνο
- Έλεγχος ποιότητας
- Γραμμή παραγωγής
- Μεταλλική ράβδος
- Αλγορίθμος λάξευσης του χώρου
- Artificial intelligence
- Computer vision
- Computer vision-based diagnostic systems
- Image classification
- Image/Video processing algorithms
- Medical imaging
- Fisheye camera
- Fisheye image generation model
- Projective camera
- Camera calibration
- Image segmentation
- Silhouette segmentation
- Video processing
- Position estimation
- Pose/Activity recognition
- Human fall detection
- Silhouette to shape - S2S
- Shape from silhouettes
- Three-dimensional model reconstruction
- Volumetric model
- Volume reconstruction
- Reconstruction error
- Streak detection
- Dermoscopy images
- Skin lesions
- Melanoma classification
- Pattern recognition
- Local image descriptors
- Zernike moment invariants
- Geodesic Zernike
- Moment invariants
- Convolutional neural networks
- Convolutional neural networks architectures
- Transfer learning
- Image processing filters
- Superpixel
- Real-time signal processing
- Real-time measurement
- Quality control
- Manufacturing industries
- Metal rod
- Space carving algorithm
- Επιστήμη Ηλεκτρονικών Υπολογιστών και Πληροφορική
- Φυσικές Επιστήμες
- Computer and Information Sciences
- Natural Sciences
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/53888