Democritus University of Thrace (DUTH)
Development, implementation and evaluation of methods for the description and the retrieval of multimedia visual content using intelligent techniques
Abstract
dc:descriptionThe thesis you are holding summarizes the author's four and a half yearsof scientific efforts, experimentation and gained knowledge in the field ofvisual content based description and retrieval.The main goal of this thesis was the development of novel techniques andcomponents, that could be utilized for effective retrieval based on multimedia'svisual content. With efficient content based image retrieval (CBIR)being the core of the addressed problem, the main scientific efforts andcontributions lie in the implementation of vectorized image representations(descriptors) based on extracting distinctive image features. On a next level,the entire retrieval framework is evaluated and several improvements areproposed, implemented and tested.Through an in-depth literature review serving as the starting point, theweaknesses, shortcomings and opportunities for improvements were located,and can be summarized as follows:1. Despite the massive amount of proposed approaches, there is still avalid uncertainty concerning the type of features (global or local) thatshould be the basis of the description method to accommodate contentbased image retrieval. The object of debate boils down to the specificsof the domain of application, overall computational cost, and storingneeds for the ever growing data.2. Combining different extracted image features (image attributes) so asto effectively enrich the final representation, and vectorizing it so thatsimilarity can be robustly calculated, usually comes with the respectivecost in terms of complexity, that significantly affects the efficiency ofa method.3. The retrieval frameworks, extensively employed by the scientific community,are unrealistic multiparametric systems, that demand user initializationsand tuning in various steps throughout the retrieval process;from parameters involved in image pre-processing, to selectingscales, vector lengths and weighting factors for the formation of thedescriptors and even demanding hands-on relevance feedback from theuser.4. Last but not least, in their vast majority, proposed methods from theliterature cannot be evaluated, further tested or build upon, due to alack of provided open source implementations, organized in extensibleretrieval platforms and libraries.Motivated by all points raised above, the dissertation starts by providingan extensive literature study on content based image retrieval, covering allthe correlated components and coherent steps. From well-established strategiesand their variants, to recent trends and breakthroughs, this review (a)provides an updated knowledge background for the reader and (b) serves asthe basis for the establishment of the thesis' motivations.During the literature review, we additionally provide a focused study ofthe strategies employed to measure the performance of retrieval systems andpresent original research results evaluating and directly comparing single-valuedmetrics, through a wide range of diverse experimental setups.Moving on to the main core of the thesis, we present novel techniquesfor visual content based retrieval, exploring the matter with varying approaches,aiming in covering all open issues and improvement opportunitiesdiscussed earlier. We organized the presentation of the proposed implementationsbased on the kind of features they employ, and in total we contributewith:a. One complete retrieval system based on a new global-featuredescriptor: It is a fully unsupervised, parameter-free, adaptive method,that extracts and combines contour and color information to effectively representimages, loosely inspired by the Human Visual System.b. Two local-features' retrieval methods: The first fuses the ideaof inserting the spatial relationship of visual words (in terms of their distributionentropy), enhancing the conventional Bag-of-Visual-Words (BOVW)aggregation model and leading to a robust, scale invariant descriptor. Thesecond, presents a three-fold contribution: it eliminates the free parameter ofthe codebook size by dynamically calculating the appropriate size via a self-growing,self-organized gas neural network; it proposes a new soft-weightingtechnique for the assignment of features to visual words that improves theperformance while maintaining a computational advantage over the classic approach; it introduces a novel composite descriptor that combines color and texture information extracted at the features' level.c. An entire family of hybrid descriptors: It is a framework forlocalizing the description mechanism of well-established, suited for retrieval,global-descriptors. Based on the simple idea that we could adopt a varietyof sampling techniques to define different image patches in multiple scalesand then proceed with a descriptor traditionally used for global image descriptionsto individually treat and describe the visual information of thosepatches, we were able to obtain a set of powerful yet light-weighted localfeatures descriptors.Part V presents the hardware and software implementations and tools,developed during the course of the thesis.The hardware implementation concerns the utilization of the multi-coreparallel execution power of GPUs, for the acceleration of the indexing part ofthe retrieval process, so as to achieve real-time indexing of video sequencesand fast execution times for large-scaled image collections.A novel open source and extensible Java image retrieval library, namedGolder Retriever, was implemented. It is complete framework for CBIRbased on the BOVW approach. The scope of the library is to provide solutionson how to integrate CBIR techniques in an easy way, on a widerange of applications. It serves as a testing platform for researchers andcan be used to evaluate existing or new CBIR approaches that can be easilyincorporated via the embedded plug-in system.The contribution, the overall evaluation and the future research directionsare discussed in the Concluding Remarks section.Software implementations of all proposed methods are available in C#, Matlab and Java, and can be used for testing and evaluation under the GNU GPL license. They are also included as extensions to the Img(Rummager) retrieval application and LIRe open source retrieval library
Degree
thesis:*- Grantor dc:publisher
- Democritus University of Thrace (DUTH)
- Year dc:date
- 2015
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Iakovidou, Chryssanthi
- Ιακωβίδου, Χρυσάνθη
Subjects
dc:subject × 6Rights
dc:rights- Statement dc:rights
-
- BY_NC_ND
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/36417