In pattern recognition, &ldquo / bias-variance&rdquo / trade-off is a challenging issue that the scientists has been working to get better generalization performances over the last decades. Among many learning methods, two-layered homogeneous stacked generalization has been reported to be successful in the literature, in different problem domains such as object recognition and image annotation. The aim of this work is two-folded. First, the problems of stacked generalization are attacked by a proposed novel architecture. Then, a set of success criteria for stacked generalization is studied. A serious drawback of stacked generalization architecture is the sensitivity to curse of dimensionality problem. In order to solve this problem, a new architecture named &ldquo / unanimous decision&rdquo / is designed. The performance of this architecture is shown to be comparably similar to two layered homogeneous stacked generalization architecture in low number of classes while it performs better than stacked generalization architecture in higher number of classes. Additionally, a new success criterion for two layered homogeneous stacked generalization architecture is proposed based on the individual properties of the used descriptors and it is verified in synthetic datasets.

Automatic image annotation is the process of assigning keywords to digital images depending on the content information. In one sense, it is a mapping from the visual content information to the semantic context information. In this thesis, we propose a novel approach for automatic image annotation problem, where the annotation is formulated as a multivariate mapping from a set of independent descriptor spaces, representing a whole image, to a set of words, representing class labels. For this purpose, a hierarchical annotation architecture, named as HANOLISTIC (Hierarchical Image Annotation System Using Holistic Approach), is dened with two layers. At the rst layer, called level-0 annotator, each annotator is fed by a set of distinct descriptor, extracted from the whole image. This enables us to represent the image at each annotator by a dierent visual property of a descriptor. Since, we use the whole image, the problematic segmentation process is avoided. Training of each annotator is accomplished by a supervised learning paradigm, where each word is represented by a class label. Note that, this approach is slightly dierent then the classical training approaches, where each data has a unique label. In the proposed system, since each image has one or more annotating words, we assume that an image belongs to more than one class. The output of the level-0 annotators indicate the membership values of the words in the vocabulary, to belong an image. These membership values from each annotator is, then, aggregated at the second layer by using various rules, to obtain meta-layer annotator. The rules, employed in this study, involves summation and/or weighted summation of the output of layer-0 annotators. Finally, a set of words from the vocabulary is selected based on the ranking of the output of meta-layer. The hierarchical annotation system proposed in this thesis outperforms state of the art annotation systems based on segmental and holistic approaches. The proposed system is examined in-depth and compared to the other systems in the literature by means of using several performance criteria.

The increase in the use of digital video/image has showed the need for modeling and querying the semantic content in them. Using manual annotation techniques for defining the semantic content is both costly in time and have limitations on querying capabilities. So, the need for content based information retrieval in multimedia domain is to extract the semantic content in an automatic way. The semantic content is usually defined with the objects in images/videos. In this thesis, a Genetic Algorithm based object extraction and classification mechanism is proposed for extracting the content of the videos and images. The object extraction is defined as a classification problem and a Genetic Algorithm based classifier is proposed for classification. Candidate objects are extracted from videos/images by using Normalized-cut segmentation and sent to the classifier for classification. Objects are defined with the Best Representative and Discriminative Feature (BRDF) model, where features are MPEG-7 descriptors. The decisions of the classifier are calculated by using these features and BRDF model. The classifier improves itself in time, with the genetic operations of GA. In addition to these, the system supports fuzziness by making multiple categorization and giving fuzzy decisions on the objects. Externally from the base model, a statistical feature importance determination method is proposed to generate BRDF model of the categories automatically. In the thesis, a platform independent application for the proposed system is also implemented.

Recently, very large collections of images and videos have grown rapidly. In parallel with this growth, content-based retrieval and querying the indexed collections are required to access visual information. Three main components of the visual information are color, texture and shape. In this thesis, an XML based content-based image retrieval system is presented that combines three visual descriptors of MPEG-7 and measures similarity of images by applying a distance function. An XML database is used for storing these three descriptors. The system is also extended to support high dimensional indexing for efficient search and retrieval from its XML database. To do this, an index structure, called M-Tree, is implemented which uses weighted Euclidean distance function for similarity measure. Ordered Weighted Aggregation (OWA) operators are used to define the weights of the distance function and to combine three features&rsquo / distance functions into one. The system supports nearest neighbor queries and three types of fuzzy queries / feature-based, image-based and color-based queries. Also it is shown through experimental results and analysis of retrieval effectiveness of querying that the content-based retrieval system is effective in terms of retrieval and scalability.

Audiovisuelle Medien, insbesondere Filme und Fernsehsendungen entwickelten sich innerhalb der letzten einhundert Jahre zu bedeutenden Massenmedien. Große Bestände audiovisueller Medien werden heute in Datenbanken und Mediatheken verwaltet und professionellen Nutzern ebenso wie den privaten Konsumenten zur Verfügung gestellt. Eine besondere Herausforderung liegt in der Indexierung, Durchsuchung und Beschreibung der multimedialen Datenbestände. Die Segmentierung audiovisueller Medien, als Teilgebiet der Videoanalyse, bildet die Grundlage für verschiedene Anwendungen im Bereich Multimedia-Information-Retrieval, Content-Browsing und Video-Summarization. Insbesondere die Segmentierung in semantische Handlungsanschnitte bei narrativen Medien gestaltet sich schwierig. Sie setzt ein besonderes Verständnis der filmischen Stilelemente vorraus, die im Rahmen des Schaffensprozesses genutzt wurden, um die Handlung und Narration zu unterstützten. Die Arbeit untersucht die bekannten filmischen Stilelemente und wie sie sich im Rahmen algorithmischer Verfahren für die Analyse nutzen lassen. Es kann gezeigt werden, dass unter Verwendung eines mehrstufigen Analyse-Prozesses semantische Zusammenhänge in narrativen audiovisuellen Medien gefunden werden können, die zu einer geeigneten Sequenz-Segmentierung führen. / Audiovisual media, especially movies and TV shows, developed within the last hundred years into major mass media. Today, large stocks of audiovisual media are managed in databases and media libraries. The content is provided to professional users as well as private consumers. A particular challenge lies in the indexing, searching and description of multimedia assets. The segmentation of audiovisual media as a branch of video analysis forms the basis for various applications in multimedia information retrieval, content browsing and video summarization. In particular, the segmentation into semantic meaningful scenes or sequences is difficult. It requires a special understanding of cinematic style elements that were used to support the narration during the creative process of film production. This work examines the cinematic style elements and how they can be used in the context of algorithmic methods for analysis. For this purpose, an analysis framework was developed as well as a method for sequence-segmentation of films and videos. It can be shown that semantic relationships can be found in narrative audiovisual media, which lead to an appropriate sequence segmentation, by using a multi-stage analysis process, based on visual MPEG-7 descriptors.

Audiovisuelle Medien, insbesondere Filme und Fernsehsendungen entwickelten sich innerhalb der letzten einhundert Jahre zu bedeutenden Massenmedien. Große Bestände audiovisueller Medien werden heute in Datenbanken und Mediatheken verwaltet und professionellen Nutzern ebenso wie den privaten Konsumenten zur Verfügung gestellt. Eine besondere Herausforderung liegt in der Indexierung, Durchsuchung und Beschreibung der multimedialen Datenbestände. Die Segmentierung audiovisueller Medien, als Teilgebiet der Videoanalyse, bildet die Grundlage für verschiedene Anwendungen im Bereich Multimedia-Information-Retrieval, Content-Browsing und Video-Summarization. Insbesondere die Segmentierung in semantische Handlungsanschnitte bei narrativen Medien gestaltet sich schwierig. Sie setzt ein besonderes Verständnis der filmischen Stilelemente vorraus, die im Rahmen des Schaffensprozesses genutzt wurden, um die Handlung und Narration zu unterstützten. Die Arbeit untersucht die bekannten filmischen Stilelemente und wie sie sich im Rahmen algorithmischer Verfahren für die Analyse nutzen lassen. Es kann gezeigt werden, dass unter Verwendung eines mehrstufigen Analyse-Prozesses semantische Zusammenhänge in narrativen audiovisuellen Medien gefunden werden können, die zu einer geeigneten Sequenz-Segmentierung führen. / Audiovisual media, especially movies and TV shows, developed within the last hundred years into major mass media. Today, large stocks of audiovisual media are managed in databases and media libraries. The content is provided to professional users as well as private consumers. A particular challenge lies in the indexing, searching and description of multimedia assets. The segmentation of audiovisual media as a branch of video analysis forms the basis for various applications in multimedia information retrieval, content browsing and video summarization. In particular, the segmentation into semantic meaningful scenes or sequences is difficult. It requires a special understanding of cinematic style elements that were used to support the narration during the creative process of film production. This work examines the cinematic style elements and how they can be used in the context of algorithmic methods for analysis. For this purpose, an analysis framework was developed as well as a method for sequence-segmentation of films and videos. It can be shown that semantic relationships can be found in narrative audiovisual media, which lead to an appropriate sequence segmentation, by using a multi-stage analysis process, based on visual MPEG-7 descriptors.