Esta Tese propõe duas métricas objetivas para avaliar a percepção de qualidade de vídeos sujeitos a degradações de transmissão em uma rede de pacotes. A primeira métrica usa apenas o vídeo degradado, enquanto que a segunda usa os vídeos de referência e degradado. Esta última é uma métrica de referência completa (FR - Full Reference) chamada de QCM (Quadratic Combinational Metric) e a primeira é uma métrica sem referência (NR - No Reference) chamada de VQOM (Viewing Quality Objective Metric). Em particular, o procedimento de projeto é aplicado à degradação de variação de atraso de pacotes (PDV - Packet Delay Variation). A métrica NR é descrita por uma spline cúbica composta por dois polinômios cúbicos que se encontram suavemente num ponto chamado de nó. Para o projeto de ambas métricas, colhem-se opiniões de observadores a respeito das sequências de vídeo degradadas que compõem o conjunto. A função objetiva inclui o erro quadrático total entre as opiniões e suas estimativas paramétricas, ainda consideradas como expressões algébricas. Acrescentam-se à função objetiva três condições de igualdades de derivadas tomadas no nó, cuja posição é especificada dentro de uma grade fina de pontos entre o valor mínimo e o valor máximo do fator de degradação. Essas condições são afetadas por multiplicadores de Lagrange e adicionadas à função objetiva, obtendo-se o lagrangiano, que é minimizado pela determinação dos coeficientes subótimos dos polinômios em função de cada valor do nó na grade. Finalmente escolhe-se o valor do nó que produz o erro quadrático mínimo, determinando assim os valores finais para dos coeficientes do polinômio. Por outro lado, a métrica FR é uma combinação não-linear de duas métricas populares, a PSNR (Peak Signal-to-Noise Ratio) e a SSIM (Structural Similarity Index). Um polinômio completo de segundo grau de duas variáveis é usado para realizar a combinação, porque é sensível a ambas métricas constituintes, evitando o sobreajuste em decorrência do baixo grau. Na fase de treinamento, o conjunto de valores dos coeficientes do polinômio é determinado através da minimização do erro quadrático médio para as opiniões sobre a base de dados de treino. Ambas métricas, a VQOM e a QCM, são treinadas e validadas usando uma base de dados, e testadas com outra independente. Os resultados de teste são comparados com métricas NR e FR recentes através de coeficientes de correlação, obtendo-se resultados favoráveis para as métricas propostas. / This dissertation proposes two objective metrics for estimating human perception of quality for video subject to transmission degradation over packet networks. The first metric just uses traffic data while the second one uses both the degraded and the reference video sequences. That is, the latter is a full reference (FR) metric called Quadratic Combinational Metric (QCM) and the former one is a no reference (NR) metric called Viewing Quality Objective Metric (VQOM). In particular, the design procedure is applied to packet delay variation (PDV) impairments, whose compensation or control is very important to maintain quality. The NR metric is described by a cubic spline composed of two cubic polynomials that meet smoothly at a point called a knot. As the first step in the design of either metric, the spectators score a training set of degraded video sequences. The objective function for designing the NR metric includes the total square error between the scores and their parametric estimates, still regarded as algebraic expressions. In addition, the objective function is augmented by the addition of three equality constraints for the derivatives at the knot, whose position is specified within a fine grid of points between the minimum value and the maximum value of the degradation factor. These constraints are affected by Lagrange multipliers and added to the objective function to obtain the Lagrangian, which is minimized by the suboptimal polynomial coefficients determined as a function of each knot in the grid. Finally, the knot value is selected that yields the minimum square error. By means of the selected knot value, the final values of the polynomial coefficients are determined. On the other hand, the FR metric is a nonlinear combination of two popular metrics, namely, the Peak Signal-to-Noise Ratio (PSNR) and the Structural Similarity Index (SSIM). A complete second-degree two-variable polynomial is used for the combination since it is sensitive to both constituent metrics while avoiding overfitting. In the training phase, the set of values for the coefficients of this polynomial is determined by minimizing the mean square error to the opinions over the training database. Both metrics, the VQOM and the QCM, are trained and validated using one database and tested with a different one. The test results are compared with recent NR and FR metrics by means of correlation coefficients, obtaining favorable results for the proposed metrics.

Objective Image and Video Quality Assessment (I/VQA) measures predict image/video quality as perceived by human beings - the ultimate consumers of visual data. Existing research in the area is mainly limited to benchmarking and monitoring of visual data. The use of I/VQA measures in the design and optimization of image/video processing algorithms and systems is more desirable, challenging and fruitful but has not been well explored. Among the recently proposed objective I/VQA approaches, the structural similarity (SSIM) index and its variants have emerged as promising measures that show superior performance as compared to the widely used mean squared error (MSE) and are computationally simple compared with other state-of-the-art perceptual quality measures. In addition, SSIM has a number of desirable mathematical properties for optimization tasks. The goal of this research is to break the tradition of using MSE as the optimization criterion for image and video processing algorithms. We tackle several important problems in visual communication applications by exploiting SSIM-inspired design and optimization to achieve significantly better performance. Firstly, the original SSIM is a Full-Reference IQA (FR-IQA) measure that requires access to the original reference image, making it impractical in many visual communication applications. We propose a general purpose Reduced-Reference IQA (RR-IQA) method that can estimate SSIM with high accuracy with the help of a small number of RR features extracted from the original image. Furthermore, we introduce and demonstrate the novel idea of partially repairing an image using RR features. Secondly, image processing algorithms such as image de-noising and image super-resolution are required at various stages of visual communication systems, starting from image acquisition to image display at the receiver. We incorporate SSIM into the framework of sparse signal representation and non-local means methods and demonstrate improved performance in image de-noising and super-resolution. Thirdly, we incorporate SSIM into the framework of perceptual video compression. We propose an SSIM-based rate-distortion optimization scheme and an SSIM-inspired divisive optimization method that transforms the DCT domain frame residuals to a perceptually uniform space. Both approaches demonstrate the potential to largely improve the rate-distortion performance of state-of-the-art video codecs. Finally, in real-world visual communications, it is a common experience that end-users receive video with significantly time-varying quality due to the variations in video content/complexity, codec configuration, and network conditions. How human visual quality of experience (QoE) changes with such time-varying video quality is not yet well-understood. We propose a quality adaptation model that is asymmetrically tuned to increasing and decreasing quality. The model improves upon the direct SSIM approach in predicting subjective perceptual experience of time-varying video quality.

This research investigates scalable video communications and its applications to video streaming, conferencing and 3DTV. Scalable video coding (SVC) is a layer-based encoding scheme that provides spatial, temporal and quality scalability. Heterogeneity of the Internet and clients' operating environment necessitate the adaptation of media content to ensure a satisfactory multimedia experience. SVC's layer structure allows the extraction of partial bitstreams at reduced spatial, quality and temporal resolutions that adjust the media bitrate at a fine granularity to changes in network state. The main focus of this research work is in developing such extraction algorithms in the context of SVC. Based on a combination of metadata computations and prediction mechanisms, these algorithms evaluate the quality contribution of each layer in the SVC bitstream and make extraction decisions that are aimed at maximizing video quality while operating within the available bandwidth resources. These techniques are applied in two-way interaction and one-way streaming of 2D and 3D content. Depending on the delay tolerance of these applications, rate-distortion optimized extraction algorithms are proposed. For conferencing applications, the extraction decisions are made over single frames and frame pairs due to tight end-to-end delay constraints. The proposed extraction algorithms for 3D content streaming maximize the overall perceived 3D quality based on human stereoscopic perception. When compared to current extraction methods, the new algorithms offer better video quality at a given bitrate while performing lesser number of metadata computations in the post-encoding phase. The solutions proposed for each application achieve the recurring goal of maintaining the best possible level of end-user quality of multimedia experience in spite of network impairments.

Progressive download video services, such as YouTube and podcasts, are responsible for a major part of the transmitted data volume in the Internet and it is expected, that they will also strongly affect mobile networks. Streaming video quality mainly depends on the sustainable throughput achieved during transmission. To ensure acceptable video quality in mobile networks (with limited capacity resources) the perceived quality by the customer (QoE) needs to be monitored by estimation. For that, the streaming video quality needs to be measured and monitored permanently. For TCP based progressive download we propose to extract the the video timestamps which are encoded within the payload of the TCP segments by decoding the video within the payload. The actual estimation is then done by play out buffer fill level calculations based on the TCP segment timestamp and their internal play out timestamp. The perceived quality for the user is derived from the number and duration of video stalls. Algorithms for decoding Flash Video, MP4 and WebM Video have already been implemented. After deriving the play out time it is compared to the timestamp of the respective TCP segment. The result of this comparison is an estimate of the fill level of the play out buffer in terms of play out time within the client. This estimation is done without access to the end device. The same measurement procedure can be applied for any TCP based progressive download Internet service. Video was simply taken as an example because of its current large share in traffic volume in operator networks.

Objective Image and Video Quality Assessment (I/VQA) measures predict image/video quality as perceived by human beings - the ultimate consumers of visual data. Existing research in the area is mainly limited to benchmarking and monitoring of visual data. The use of I/VQA measures in the design and optimization of image/video processing algorithms and systems is more desirable, challenging and fruitful but has not been well explored. Among the recently proposed objective I/VQA approaches, the structural similarity (SSIM) index and its variants have emerged as promising measures that show superior performance as compared to the widely used mean squared error (MSE) and are computationally simple compared with other state-of-the-art perceptual quality measures. In addition, SSIM has a number of desirable mathematical properties for optimization tasks. The goal of this research is to break the tradition of using MSE as the optimization criterion for image and video processing algorithms. We tackle several important problems in visual communication applications by exploiting SSIM-inspired design and optimization to achieve significantly better performance. Firstly, the original SSIM is a Full-Reference IQA (FR-IQA) measure that requires access to the original reference image, making it impractical in many visual communication applications. We propose a general purpose Reduced-Reference IQA (RR-IQA) method that can estimate SSIM with high accuracy with the help of a small number of RR features extracted from the original image. Furthermore, we introduce and demonstrate the novel idea of partially repairing an image using RR features. Secondly, image processing algorithms such as image de-noising and image super-resolution are required at various stages of visual communication systems, starting from image acquisition to image display at the receiver. We incorporate SSIM into the framework of sparse signal representation and non-local means methods and demonstrate improved performance in image de-noising and super-resolution. Thirdly, we incorporate SSIM into the framework of perceptual video compression. We propose an SSIM-based rate-distortion optimization scheme and an SSIM-inspired divisive optimization method that transforms the DCT domain frame residuals to a perceptually uniform space. Both approaches demonstrate the potential to largely improve the rate-distortion performance of state-of-the-art video codecs. Finally, in real-world visual communications, it is a common experience that end-users receive video with significantly time-varying quality due to the variations in video content/complexity, codec configuration, and network conditions. How human visual quality of experience (QoE) changes with such time-varying video quality is not yet well-understood. We propose a quality adaptation model that is asymmetrically tuned to increasing and decreasing quality. The model improves upon the direct SSIM approach in predicting subjective perceptual experience of time-varying video quality.

The popularity of multimedia services over Internet has increased in the recent years. These services include Video on Demand (VoD) and mobile TV which are predominantly growing, and the user expectations towards the quality of videos are gradually increasing. Different video codec’s are used for encoding and decoding. Recently Google has introduced the VP8 codec which is an open source compression format. It is introduced to compete with existing popular codec namely H.264/AVC developed by ITU-T Video Coding Expert Group (VCEG), as by 2016 there will be a license fee for H.264. In this work we compare the performance of H.264/AVC and WebM/VP8 in an emulated environment. NetEm is used as an emulator to introduce delay/delay variation and packet loss. We have evaluated the user perception of impaired videos using Mean Opinion Score (MOS) by following the International Telecommunication Union (ITU) Recommendations Absolute Category Rating (ACR) and analyzed the results using statistical methods. It was found that both video codec’s exhibit similar performance in packet loss, But in case of delay variation H.264 codec shows better results when compared to WebM/VP8. Moreover along with the MOS ratings we also studied the effect of user feelings and online video watching experience impacts on their perception. / Yeshwanth Alahari Phone : +91-9986739097 Buddhiraja Prashant Phone : +46-734897359

Diploma work „Multimedia player for iOS“ deals with the description of the video formats such as HLS, MP4, MPEG Transport Stream, and Dash. Next, it continues with protected content DRM, VAST advertisements and analytics tool called Google Analytics. During compilation of this diploma work was created a framework in programming language Swift. This framework is basically a player, that besides playing Interactive videos is able to show ads, subtitles, can change the video quality, AirPlay and download the content for offline playing. A feature to play a protected content could not be done as a developer's request for a product deployment SDK Fairplay was refused from an SDK owner. A testing app was also created that implements developed a framework and prove the functionality of the developed functions.

Master's thesis deals with the assessment of quality of Ultra HDTV video sequences applying objective metrics. Thesis theoretically describes coding of selected codecs H.265/HEVC and VP9, objective video quality metrics and also subjective methods for assessment of the video sequences quality. Next chapter deals with the implementation of the H.265/HEVC and the VP9 codecs at selected video sequences in the raw format from which arises the test sequences database. Quality of these videos is measured afterwards by objective metrics and selected subjective method. These results are compared for the purpose of finding the most consistent correlations among objective metrics and subjective assessment.

Disertační práce se zabývá metodami a prostředky pro hodnocení kvality obrazu ve videosekvencích, což je velmi aktuální téma, zažívající velký rozmach zejména v souvislosti s digitálním zpracováním videosignálů. Přestože již existuje relativně velké množství metod a metrik pro objektivní, tedy automatizované měření kvality videosekvencí, jsou tyto metody zpravidla založeny na porovnání zpracované (poškozené, například komprimací) a originální videosekvence. Metod pro hodnocení kvality videosekvení bez reference, tedy pouze na základě analýzy zpracovaného materiálu, je velmi málo. Navíc se takové metody převážně zaměřují na analýzu hodnot signálu (typicky jasu) v jednotlivých obrazových bodech dekódovaného signálu, což je jen těžko aplikovatelné pro moderní komprimační algoritmy jako je H.264/AVC, který používá sofistikovené techniky pro odstranění komprimačních artefaktů. V práci je nejprve podán stučný přehled dostupných metod pro objektivní hodnocení komprimovaných videosekvencí se zdůrazněním rozdílného principu metod využívajících referenční materiál a metod pracujících bez reference. Na základě analýzy možných přístupů pro hodnocení video sekvencí komprimovaných moderními komprimačními algoritmy je v dalším textu práce popsán návrh nové metody určené pro hodnocení kvality obrazu ve videosekvencích komprimovaných s využitím algoritmu H.264/AVC. Nová metoda je založena na sledování hodnot parametrů, které jsou obsaženy v transportním toku komprimovaného videa, a přímo souvisí s procesem kódování. Nejprve je provedena úvaha nad vlivem některých takových parametrů na kvalitu výsledného videa. Následně je navržen algoritmus, který s využitím umělé neuronové sítě určuje špičkový poměr signálu a šumu (peak signal-to-noise ratio -- PSNR) v komprimované videosekvenci -- plně referenční metrika je tedy nahrazována metrikou bez reference. Je ověřeno několik konfigurací umělých neuronových sítí od těch nejjednodušších až po třívrstvé dopředné sítě. Pro učení sítí a následnou analýzu jejich výkonnosti a věrnosti určení PSNR jsou vytvořeny dva soubory nekomprimovaných videosekvencí, které jsou následně komprimovány algoritmem H.264/AVC s proměnným nastavením kodéru. V závěrečné části práce je proveden rozbor chování nově navrženého algoritmu v případě, že se změní vlastnosti zpracovávaného videa (rozlišení, střih), případně kodéru (formát skupiny současně kódovaných snímků). Chování algoritmu je analyzováno až do plného vysokého rozlišení zdrojového signálu (full HD -1920 x 1080 obrazových bodů).

Progressive download video services, such as YouTube and podcasts, are responsible for a major part of the transmitted data volume in the Internet and it is expected, that they will also strongly affect mobile networks. Streaming video quality mainly depends on the sustainable throughput achieved during transmission. To ensure acceptable video quality in mobile networks (with limited capacity resources) the perceived quality by the customer (QoE) needs to be monitored by estimation. For that, the streaming video quality needs to be measured and monitored permanently. For TCP based progressive download we propose to extract the the video timestamps which are encoded within the payload of the TCP segments by decoding the video within the payload. The actual estimation is then done by play out buffer fill level calculations based on the TCP segment timestamp and their internal play out timestamp. The perceived quality for the user is derived from the number and duration of video stalls. Algorithms for decoding Flash Video, MP4 and WebM Video have already been implemented. After deriving the play out time it is compared to the timestamp of the respective TCP segment. The result of this comparison is an estimate of the fill level of the play out buffer in terms of play out time within the client. This estimation is done without access to the end device. The same measurement procedure can be applied for any TCP based progressive download Internet service. Video was simply taken as an example because of its current large share in traffic volume in operator networks.