Content-based video retrieval research combines expertise from many different areas, such as signal processing, machine learning, pattern recognition, and computer vision. As video extends into both the spatial and the temporal domain, we require techniques for the temporal decomposition of footage so that specific content can be accessed. This content may then be semantically classified - ideally in an automated process - to enable filtering, browsing, and searching. An important aspect that must be considered is that pictorial representation of information may be interpreted differently by individual users because it is less specific than its textual representation. In this thesis, we address several fundamental issues of content-based video retrieval for effective handling of digital footage. Temporal segmentation, the common first step in handling digital video, is the decomposition of video streams into smaller, semantically coherent entities. This is usually performed by detecting the transitions that separate single camera takes. While abrupt transitions - cuts - can be detected relatively well with existing techniques, effective detection of gradual transitions remains difficult. We present our approach to temporal video segmentation, proposing a novel algorithm that evaluates sets of frames using a relatively simple histogram feature. Our technique has been shown to range among the best existing shot segmentation algorithms in large-scale evaluations. The next step is semantic classification of each video segment to generate an index for content-based retrieval in video databases. Machine learning techniques can be applied effectively to classify video content. However, these techniques require manually classified examples for training before automatic classification of unseen content can be carried out. Manually classifying training examples is not trivial because of the implied ambiguity of visual content. We propose an unsupervised learning approach based on latent class modelling in which we obtain multiple judgements per video shot and model the users' response behaviour over a large collection of shots. This technique yields a more generic classification of the visual content. Moreover, it enables the quality assessment of the classification, and maximises the number of training examples by resolving disagreement. We apply this approach to data from a large-scale, collaborative annotation effort and present ways to improve the effectiveness for manual annotation of visual content by better design and specification of the process. Automatic speech recognition techniques along with semantic classification of video content can be used to implement video search using textual queries. This requires the application of text search techniques to video and the combination of different information sources. We explore several text-based query expansion techniques for speech-based video retrieval, and propose a fusion method to improve overall effectiveness. To combine both text and visual search approaches, we explore a fusion technique that combines spoken information and visual information using semantic keywords automatically assigned to the footage based on the visual content. The techniques that we propose help to facilitate effective content-based video retrieval and highlight the importance of considering different user interpretations of visual content. This allows better understanding of video content and a more holistic approach to multimedia retrieval in the future.

Video game development is a high-risk effort with low probability of success. The interactive nature of the resulting artifact increases production complexity, often doing so in ways that are unexpected. New methodologies are needed to address issues in this domain.<p> Video game development has two major phases: preproduction and production. During <i>preproduction</i>, the game designer and other members of the creative team create and capture a vision of the intended player experience in the game design document. The game design document tells the story and describes the game - it does not usually explicitly elaborate all of the details of the intended player experience, particularly with respect to how the player is intended to feel as the game progresses. Details of the intended experience tend to be communicated verbally, on an as-needed basis during iterations of the production effort.<p> During <i>production</i>, the software and media development teams attempt to realize the preproduction vision in a game artifact. However, the game design document is not traditionally intended to capture production-ready requirements, particularly for software development. As a result, there is a communications chasm between preproduction and production efforts that can lead to production issues such as excessive reliance on direct communication with the game designer, difficulty scoping project elements, and difficulty in determining reasonably accurate effort estimates.<p> We posit that defining and capturing the intended player experience in a manner that is influenced and informed by established requirements engineering principles and techniques will help cross the communications chasm between preproduction and production. The proposed experience requirements methodology is a novel contribution composed of:<p> <ol> <li>a model for the elements that compose experience requirements,</li> <li>a framework that provides guidance for expressing experience requirements, and</li> <li>an exemplary process for the elicitation, capture, and negotiation of experience requirements.</li> <ol><p> Experience requirements capture the designer' s intent for the user experience; they represent user experience goals for the artifact and constraints upon the implementation and are not expected to be formal in the mathematical sense. Experience requirements are evolutionary in intent - they incrementally enhance and extend existing practices in a relatively lightweight manner using language and representations that are intended to be mutually acceptable to preproduction and to production.

Scalable image and video has been proposed to transmit image and video signals over lossy networks, such as the Internet and wireless networks. However, scalability alone is not a complete solution since there is a conflict between the unequal importance of the scalable bit stream and the agnostic nature of packet losses in the network. This thesis investigates three methods to combat the detrimental effects of random packet losses to scalable images and video, namely the error resilient method, the error concealment method, and the unequal error protection method within the joint source-channel coding framework. For the error resilient method, an optimal bit allocation algorithm is proposed without considering the distortion caused by packet losses. The allocation algorithm is then extended to accommodate packet losses. For the error concealment method, a simple temporal error concealment mechanism is designed to work for video signals. For the unequal error protection method, the optimal protection allocation problem is formulated and solved. These methods are tested on the wavelet-based Set Partitioning in Hierarchical Trees(SPIHT) scalable image coder. Performance gains and losses in lossy and lossless environments are studied for both the original coder and the error-controlled coders. The results show performance advantages of all three methods over the original SPIHT coder. Particularly, the unequal error protection method and error concealment method are promising for future Internet/wireless image and video transmission, because the former has very good performance even at heavy packet loss (a PSNR of 22.00 dB has been seen at nearly 60% packet loss) and the latter does not introduce any extra overhead.

Scalable image and video has been proposed to transmit image and video signals over lossy networks, such as the Internet and wireless networks. However, scalability alone is not a complete solution since there is a conflict between the unequal importance of the scalable bit stream and the agnostic nature of packet losses in the network. This thesis investigates three methods to combat the detrimental effects of random packet losses to scalable images and video, namely the error resilient method, the error concealment method, and the unequal error protection method within the joint source-channel coding framework. For the error resilient method, an optimal bit allocation algorithm is proposed without considering the distortion caused by packet losses. The allocation algorithm is then extended to accommodate packet losses. For the error concealment method, a simple temporal error concealment mechanism is designed to work for video signals. For the unequal error protection method, the optimal protection allocation problem is formulated and solved. These methods are tested on the wavelet-based Set Partitioning in Hierarchical Trees(SPIHT) scalable image coder. Performance gains and losses in lossy and lossless environments are studied for both the original coder and the error-controlled coders. The results show performance advantages of all three methods over the original SPIHT coder. Particularly, the unequal error protection method and error concealment method are promising for future Internet/wireless image and video transmission, because the former has very good performance even at heavy packet loss (a PSNR of 22.00 dB has been seen at nearly 60% packet loss) and the latter does not introduce any extra overhead.

With the rapid development of network visual communication technologies, digital video has become ubiquitous and indispensable in our everyday lives. Video acquisition, communication, and processing systems introduce various types of distortions, which may have major impact on perceived video quality by human observers. Effective and efficient objective video quality assessment (VQA) methods that can predict perceptual video quality are highly desirable in modern visual communication systems for performance evaluation, quality control and resource allocation purposes. Moreover, perceptual VQA measures may also be employed to optimize a wide variety of video processing algorithms and systems for best perceptual quality. This thesis exploits several novel ideas in the areas of video quality assessment and enhancement. Firstly, by considering a video signal as a 3D volume image, we propose a 3D structural similarity (SSIM) based full-reference (FR) VQA approach, which also incorporates local information content and local distortion-based pooling methods. Secondly, a reduced-reference (RR) VQA scheme is developed by tracing the evolvement of local phase structures over time in the complex wavelet domain. Furthermore, we propose a quality-aware video system which combines spatial and temporal quality measures with a robust video watermarking technique, such that RR-VQA can be performed without transmitting RR features via an ancillary lossless channel. Finally, a novel strategy for enhancing video denoising algorithms, namely poly-view fusion, is developed by examining a video sequence as a 3D volume image from multiple (front, side, top) views. This leads to significant and consistent gain in terms of both peak signal-to-noise ratio (PSNR) and SSIM performance, especially at high noise levels.

El desarrollo y las avances recientes de la tecnología de los ordenadores y de la tecnología de alta velocidad de redes han hecho posible que las aplicaciones de video bajo demanda estén conectadas a "shared-computing" servidores reemplazando los sistemas tradicionales donde cada aplicación tenía su propia máquina dedicada para servirla. La aplicación de video bajo demanda permite a los usuarios seleccionar de una lista de videos su película favorita y ver su reproducción a su gusto.Sin embargo, la aplicación de video bajo demanda se considera como una de las aplicaciones que debería soportar largos "video streams", que consumen muchos recursos como el anch de banda de red y I/O, a gran número de clientes. Por eso, el servidor de video debería asegurar los recursos necesarios para cada "stream" durante un periodo de tiempo largo (e.g. 7200 segundos) para que los clientes reproduzcan el video sin "jitter" y "starvation" en sus búferes.Esta tesis presenta el diseño y la implementación de un Servidor Proxy de Video (VPS) que puede proveer video bajo demanda interactiva. El VPS consiste de tres componentes (partes) principales. La primera parte es el Modulo de Control de Admisión (ACM) que recibe las peticiones de los clientes, negocia los recursos requeridos, y decide si la petición puede ser aceptada o rechazada basado en la disponibilidad de los recursos. La segunda parte es el Modulo de Manejo de los Recursos (RMM) que maneja los recursos del sistema como el CPU, la Memoria, la Red, y el Disco. Este consta de cuatro "brokers" que reservan a los recursos necesarios basado en una política predefinida. La tercera parte es el algoritmo CB_MDA "Credit_Based Media Delivery Algorithm" que controla y regula el flujo de los "streams" del video. La CB_MDA utiliza una combinación de canales unicast y "multicast" para transmitir el video. Los "streams" de "multicast" se inician para empezar a emitir el video desde el principio, mientras los canales unicast se usan para juntar los llegados tardes a un "stream multicast" apropiado. En la implementación, el CB_MDA detecta los momentos cuando el servidor tiene disponibilidad de recursos y les asigna a los usuarios apropiados para crear un trabajo en adelanto. / The recent advances and development of inexpensive computers and high speed networking technology have enabled the Video on Demand (VoD) application to connect to shared-computing servers, replacing the traditional computing environments where each application was having its own dedicated special purpose computing hardware. The VoD application enables the viewer to select, from a list of video files, his favourite video file and watch its reproduction at will.However, the VoD application is known as one of the applications that must provide long-lived video streams which consume high resources such as I/O and network bandwidth to a large number of clients. Therefore, a video server must secure the necessary resources for each stream during a long period of time (e.g. 7200 seconds) so that the clients can reproduce (play) the video data without witnessing jitter or starvation in their buffers.This thesis presents the design and implementation for a video proxy server (VPS) which can provide interactive video on demand. The VPS consists of three main parts. The first part is the Admission Control Module which receives the clients' requests, negotiates the required resources, and decides whether to accept or reject a client based on the available resources. The second part is the Resources Management Module which manages several shared resources such as the CPU, the Memory, the Network and the Disk It consists of four brokers that can reserve the necessary resources based on a predefined policy. The third part is the CB_MDA algorithm which is responsible for regulating the resources assignment and scheduling the video streams. The CB_MDA uses a combination of multicast and unicast channels for transmitting the video data. The multicast streams are initiated to start a video file from the beginning while the unicast channels are used to join the later arrivals to the appropriate multicast stream. In the implementation, the CB_MDA discovers the period of time when the server has plenty of resources an assigns them to appropriate clients in order to create work-ahead video data.The thesis further goes beyond the design of the VPS and presents a video client architecture that can synchronize with the server and work as a plug-in for producing the video data on different players such as MPEG-Berkely player, Xine.etc.

Video game development is a high-risk effort with low probability of success. The interactive nature of the resulting artifact increases production complexity, often doing so in ways that are unexpected. New methodologies are needed to address issues in this domain.<p> Video game development has two major phases: preproduction and production. During <i>preproduction</i>, the game designer and other members of the creative team create and capture a vision of the intended player experience in the game design document. The game design document tells the story and describes the game - it does not usually explicitly elaborate all of the details of the intended player experience, particularly with respect to how the player is intended to feel as the game progresses. Details of the intended experience tend to be communicated verbally, on an as-needed basis during iterations of the production effort.<p> During <i>production</i>, the software and media development teams attempt to realize the preproduction vision in a game artifact. However, the game design document is not traditionally intended to capture production-ready requirements, particularly for software development. As a result, there is a communications chasm between preproduction and production efforts that can lead to production issues such as excessive reliance on direct communication with the game designer, difficulty scoping project elements, and difficulty in determining reasonably accurate effort estimates.<p> We posit that defining and capturing the intended player experience in a manner that is influenced and informed by established requirements engineering principles and techniques will help cross the communications chasm between preproduction and production. The proposed experience requirements methodology is a novel contribution composed of:<p> <ol> <li>a model for the elements that compose experience requirements,</li> <li>a framework that provides guidance for expressing experience requirements, and</li> <li>an exemplary process for the elicitation, capture, and negotiation of experience requirements.</li> <ol><p> Experience requirements capture the designer' s intent for the user experience; they represent user experience goals for the artifact and constraints upon the implementation and are not expected to be formal in the mathematical sense. Experience requirements are evolutionary in intent - they incrementally enhance and extend existing practices in a relatively lightweight manner using language and representations that are intended to be mutually acceptable to preproduction and to production.

This work aims to improve our knowledge about Swedish video game journalists, a new occupation in a quickly expanding business. Through a survey sent out to over 50 Swedish video game journalists, the result indicate some unexpected facts. Men largely dominate the profession; just 10 % of the work forces are females. This relation was notable in our survey, as well as in earlier studies. Video game journalists between 26 and 35 years old seems to be the largest age group, which we think is a bit surprising considering how new this form of journalism is, and the fact the internet provides opportunities for every enthusiastic video game writer. The professionalism of video game journalists has been debated over a long time, with many people arguing that this type of work has little to do with traditional journalism. In our survey, a large part of the respondents made clear that they consider themselves to be writers rather than journalists. The majority of the Swedish video game journalists don't have a journalistic education, whether it's from high school or university. Most of them don't value education as much as writing skill, talent, passion and knowledge about video games. The minority arguing about the importance of education motivates their standpoint with arguments of classical journalistic ethics and values. One respondent told us that the whole industry would benefit from more highly educated journalists, another one pointed out that a lot of ground has been covered in the last 5-6 years, although there ́s still a great distance compared to traditional journalism.

While the demand for video streaming services has risen rapidly in recent years, supporting video streaming service to a large number of receivers still remains a challenging task. Issues of video streaming in the Internet, such as scalability, and reliability are still under extensive research. Recently proposed network contexts such as overlay networks, and mobile ad hoc networks pose even tougher challenges. This thesis focuses on supporting scalable video streaming applications under various network environments. More specifically, this thesis investigates the following problems: i) Server selection in replicated batching video on demand (VoD) systems: we find out that, to optimize the user perceived latency, it is vital to consider the server state information and channel allocation schemes when making server selection decisions. We develop and evaluate a set of server selection algorithms that use increasingly more information. ii) Scalable live video streaming with time shifting and video patching: we consider the problem of how to enable continuous live video streaming to a large group of clients in cooperative but unreliable overlay networks. We design a server-based architecture which uses a combined technique of time-shifting video server and P2P video patching. iii) A Cooperative patching architecture in overlay networks: We design a cooperative patching architecture which shifts video patching responsibility completely to the client side. An end-host retrieves lost data from other end-hosts within the same multicast group. iv) V3: a vehicle to vehicle video streaming architecture: We propose V3, an architecture to provide live video streaming service to driving vehicles through vehicle-to-vehicle (V2V) networks. V3 incorporates a novel signaling mechanism to continuously trigger video sources to send video data back to the receiver. It also adopts a store-carry-and-forward approach to transmit video data in a partitioned network environment. We also develop a multicasting framework that enables live video streaming applications from multiple sources to multiple receivers in V2V networks. A message integration scheme is used to suppress the signaling overhead, and a two-level tree-based routing approach is adopted to forward the video data.

As digital communication of television content becomes more pervasive, and as networks supporting such communication become increasingly diverse, the long-standing problem of assessing video quality by objective measurements becomes particularly important. Content owners as well as content distributors stand to benefit from rapid objective measurements that correlate well with subjective assessments, and further, do not depend on the availability of the original reference video. This thesis investigates different techniques of subjective and objective video evaluation. Our research recommends a functional quality metric called Mean Time Between Failures (MTBF) where failure refers to video artifacts deemed to be perceptually noticeable, and investigates objective measurements that correlate well with subjective evaluations of MTBF. Work has been done for determining the usefulness of some existing objective metric by noting their correlation with MTBF. The research also includes experimentation with network-induced artifacts, and a study on statistical methods for correlating candidate objective measurements with the subjective metric. The statistical significance and spread properties for the correlations are studied, and a comparison of subjective MTBF with the existing subjective measure of MOS is performed. These results suggest that MTBF has a direct and predictable relationship with MOS, and that they have similar variations across different viewers. The research is particularly concerned with the development of new no-reference objective metrics that are easy to compute in real time, as well as correlate better than current metrics with the intuitively appealing MTBF measure. The approach to obtaining greater subjective relevance has included the study of better spatial-temporal models for noise-masking and test data pooling in video perception. A new objective metric, 'Automatic Video Quality' metric (AVQ) is described and shown to be implemented in real time with a high degree of correlation with actual subjective scores, with the correlation values approaching the correlations of metrics that use full or partial reference. This is metric does not need any reference to the original video, and when used to display MPEG2 streams, calculates and indicates the video quality in terms of MTBF. Certain diagnostics like the amount of compression and network artifacts are also shown.