This dissertation thesis describes the area of multimedia semantics which is a research area that brings together research streams that until recently run separately. The aim of the work is to provide an insight to all areas from this wide discipline and give an outlook on current problems especially to the semantic gab phenomena. Number of findings and outcomes in this work comes from international project K-Space, in which the author took part for three years. The extensive theoretical introduction into problematic is followed by a list state-of-the-art application from this area and overview of KIZI activities and involvements in the European project. The contribution of the work is a research on textual resources complementary to video and experiments with automatic detection of sporting events based on pre-classified examples and trained model. The practical contribution is also a demo web application that shows all the resources together and allows non-linear browsing of events.

Recognition of complex events in consumer uploaded Internet videos, captured under realworld settings, has emerged as a challenging area of research across both computer vision and multimedia community. In this dissertation, we present a systematic decomposition of complex events into hierarchical components and make an in-depth analysis of how existing research are being used to cater to various levels of this hierarchy and identify three key stages where we make novel contributions, keeping complex events in focus. These are listed as follows: (a) Extraction of novel semi-global features – firstly, we introduce a Lie-algebra based representation of dominant camera motion present while capturing videos and show how this can be used as a complementary feature for video analysis. Secondly, we propose compact clip level descriptors of a video based on covariance of appearance and motion features which we further use in a sparse coding framework to recognize realistic actions and gestures. (b) Construction of intermediate representations – We propose an efficient probabilistic representation from low-level features computed from videos, based on Maximum Likelihood Estimates which demonstrates state of the art performance in large scale visual concept detection, and finally, (c) Modeling temporal interactions between intermediate concepts – Using block Hankel matrices and harmonic analysis of slowly evolving Linear Dynamical Systems, we propose two new discriminative feature spaces for complex event recognition and demonstrate significantly improved recognition rates over previously proposed approaches.