This thesis focuses on video watermarking robust against geometric attacks and video compressions. In addition to the requirements for an image watermarking algorithm, a digital video watermarking algorithm has to be robust against advanced video compressions, frame loss, frame swapping, aspect ratio change, frame rate change, intra- and inter-frame filtering, etc. Video compression, especially, the most efficient compression standard, H.264, and geometric attacks, such as rotation and cropping, frame aspect ratio change, and translation, are considered the most challenging attacks for video watermarking algorithms. In this thesis, we first review typical watermarking algorithms robust against geometric attacks and video compressions, and point out their advantages and disadvantages. Then, we propose our robust video watermarking algorithms against Rotation, Scaling and Translation (RST) attacks and MPEG-2 compression based on the logpolar mapping and the phase-only filtering method. Rotation or scaling transformation in the spatial domain results in vertical or horizontal shift in the log-polar mapping (LPM) of the magnitude of the Fourier spectrum of the target frame. Translation has no effect in this domain. This method is very robust to RST attacks and MPEG-2 compression. We also demonstrate that this method can be used as a RST parameters detector to work with other watermarking algorithms to improve their robustness to RST attacks. Furthermore, we propose a new video watermarking algorithm based on the 1D DFT (one-dimensional Discrete Fourier Transform) and 1D projection. This algorithm enhances the robustness to video compression and is able to resist the most advanced video compression, H.264. The 1D DFT for a video sequence along the temporal domain generates an ideal domain, in which the spatial information is still kept and the temporal information is obtained. With detailed analysis and calculation, we choose the frames with highest temporal frequencies to embed the fence-shaped watermark pattern in the Radon transform domain of the selected frames. The performance of the proposed algorithm is evaluated by video compression standards MPEG-2 and H.264; geometric attacks such as rotation, translation, and aspect-ratio changes; and other video processing. The most important advantages of this video watermarking algorithm are its simplicity, practicality and robustness.

Since its introduction in 2004, the Structural Similarity (SSIM) index has gained widespread popularity as an image quality assessment measure. SSIM is currently recognized to be one of the most powerful methods of assessing the visual closeness of images. That being said, the Mean Squared Error (MSE), which performs very poorly from a perceptual point of view, still remains the most common optimization criterion in image processing applications because of its relative simplicity along with a number of other properties that are deemed important. In this thesis, some necessary tools to assist in the design of SSIM-optimal algorithms are developed. This work combines theoretical developments with experimental research and practical algorithms. The description of the mathematical properties of the SSIM index represents the principal theoretical achievement in this thesis. Indeed, it is demonstrated how the SSIM index can be transformed into a distance metric. Local convexity, quasi-convexity, symmetries and invariance properties are also proved. The study of the SSIM index is also generalized to a family of metrics called normalized (or M-relative) metrics. Various analytical techniques for different kinds of SSIM-based optimization are then devised. For example, the best approximation according to the SSIM is described for orthogonal and redundant basis sets. SSIM-geodesic paths with arclength parameterization are also traced between images. Finally, formulas for SSIM-optimal point estimators are obtained. On the experimental side of the research, the structural self-similarity of images is studied. This leads to the confirmation of the hypothesis that the main source of self-similarity of images lies in their regions of low variance. On the practical side, an implementation of local statistical tests on the image residual is proposed for the assessment of denoised images. Also, heuristic estimations of the SSIM index and the MSE are developed. The research performed in this thesis should lead to the development of state-of-the-art image denoising algorithms. A better comprehension of the mathematical properties of the SSIM index represents another step toward the replacement of the MSE with SSIM in image processing applications.

Digitization of medical data has become a very important part of the modern healthcare system. Data can be transmitted easily at any time to anywhere in the world using Internet to get the best diagnosis possible for a patient. This digitized medical data must be protected at all times to preserve the doctor-patient confidentiality. Watermarking can be used as an effective tool to achieve this. In this research project, image watermarking is performed both in the spatial domain and the frequency domain to embed a shared image with medical image data and the patient data which would include the patient identification number. For the proposed system, Structural Similarity (SSIM) is used as an index to measure the quality of the watermarking process instead of Peak Signal to Noise Ratio (PSNR) since SSIM takes into account the visual perception of the images compared to PSNR which uses the intensity levels to measure the quality of the watermarking process. The system response under ideal conditions as well as under the influence of noise were measured and the results were analyzed.

This thesis focuses on video watermarking robust against geometric attacks and video compressions. In addition to the requirements for an image watermarking algorithm, a digital video watermarking algorithm has to be robust against advanced video compressions, frame loss, frame swapping, aspect ratio change, frame rate change, intra- and inter-frame filtering, etc. Video compression, especially, the most efficient compression standard, H.264, and geometric attacks, such as rotation and cropping, frame aspect ratio change, and translation, are considered the most challenging attacks for video watermarking algorithms. In this thesis, we first review typical watermarking algorithms robust against geometric attacks and video compressions, and point out their advantages and disadvantages. Then, we propose our robust video watermarking algorithms against Rotation, Scaling and Translation (RST) attacks and MPEG-2 compression based on the logpolar mapping and the phase-only filtering method. Rotation or scaling transformation in the spatial domain results in vertical or horizontal shift in the log-polar mapping (LPM) of the magnitude of the Fourier spectrum of the target frame. Translation has no effect in this domain. This method is very robust to RST attacks and MPEG-2 compression. We also demonstrate that this method can be used as a RST parameters detector to work with other watermarking algorithms to improve their robustness to RST attacks. Furthermore, we propose a new video watermarking algorithm based on the 1D DFT (one-dimensional Discrete Fourier Transform) and 1D projection. This algorithm enhances the robustness to video compression and is able to resist the most advanced video compression, H.264. The 1D DFT for a video sequence along the temporal domain generates an ideal domain, in which the spatial information is still kept and the temporal information is obtained. With detailed analysis and calculation, we choose the frames with highest temporal frequencies to embed the fence-shaped watermark pattern in the Radon transform domain of the selected frames. The performance of the proposed algorithm is evaluated by video compression standards MPEG-2 and H.264; geometric attacks such as rotation, translation, and aspect-ratio changes; and other video processing. The most important advantages of this video watermarking algorithm are its simplicity, practicality and robustness.

Generative Artificial Intelligence is a field of artificial intelligence where systems can learn underlying patterns in previously seen content and generate new content. This thesis explores a generative artificial intelligence technique used for image-toimage translations called Cycle-consistent Adversarial network (CycleGAN), which can translate images from one domain into another. The CycleGAN is a stateof-the-art technique for doing unsupervised image-to-image translations. It uses the concept of cycle-consistency to learn a mapping between image distributions, where the Mean Absolute Error function is used to compare images and thereby learn an underlying mapping between the two image distributions. In this work, we propose to use the Structural Similarity Index Measure (SSIM) as an alternative to the Mean Absolute Error function. The SSIM is a metric inspired by human perception, which measures the difference in two images by comparing the difference in, contrast, luminance, and structure. We examine if using the SSIM as the cycle-consistency loss in the CycleGAN will improve the image quality of generated images as measured by the Inception Score and Fréchet Inception Distance. The inception Score and Fréchet Inception Distance are both metrics that have been proposed as methods for evaluating the quality of images generated by generative adversarial networks (GAN). We conduct a controlled experiment to collect the quantitative metrics. Our results suggest that using the SSIM in the CycleGAN as the cycle-consistency loss will, in most cases, improve the image quality of generated images as measured Inception Score and Fréchet Inception Distance.

Projektet som beskrivs i denna rapport utgör modifieringen av videokonferensprogrammet Jitsi Meet, där öppen källkod tillåter anpassning av tjänsten för videoinspelning. Den mottagna videoströmmen som utgör samtalet mellan två deltagare skall sparas till disk, för att sedan möjliggöra för utredning och analys av sessionens videokvalitet. För att uppnå behoven för denna kvalitetskontroll krävdes en inspelning av mottagen video med jämförbar klarhet och komposition till originalet. Olika tillvägagångssätt utvärderades för att fullborda videoströmmens utvinning. Av de tre ansatser som presenterade sig valdes framtagandet av en inspelningsfunktion på klientsidan, en lösning som nyttjar användarens webbläsare för att utföra uppgiften. Jitsi Meet stöds av standardiserade teknologier inom webbaserad mediahantering, vilket möjliggjorde för tillämpningen av intilliggande metoder och verktyg i inspelningsfunktionens utveckling. Den resulterande inspelningsfunktionen utgör ena parten i ett tvåsidigt system, där den modifierade webbklienten agerar mottagare för en särskild mobilapplikations videoström. Enskilda bildrutor från enheternas videoinspelningar jämfördes via dess strukturella likheter och brusanalys, där skillnader i videons kvalitet före och efter strömning uppenbarade sig genom konkreta värden. Undersökning av resulterande grafer indikerar att särskilda händelser så som förändrad bithastighet, försvunna bildrutor och andra avvikelser kan identifieras med hjälp av den framtagna mätuppställningen. Projektet har därmed uppnått målet att producera en inspelningslösning för strömmad video som kan användas i kontroll av videokvalitet. Därmed ges goda förutsättningar för fortsatt arbete med utvärdering av kvalitet för videosamtal. / The project described in this report is the modification of the video conferencing software Jitsi Meet, where its open-source code allows for customization for the purpose of video recording. The received video stream that constitutes the conversation between two participants shall be saved to disk, to then allow for the examination and analysis of the session’s video quality. To meet the needs of this quality control, a recording of the received video with comparable clarity and composition to the original was required. Various approaches were evaluated to allow for the extraction of the video stream. Of the three approaches that presented themselves, the development of a recording function on the client side was chosen, a solution that uses the user's web browser to perform the task. Jitsi Meet is based on standardized technologies in web-based media management, which enabled the use of adjacent methods and tools in the development of its recording solution. The resulting recording solution forms one part of a two-sided system, where the modified web client acts as a receiver for a particular mobile application's video stream. Individual frames from the devices' video recordings were compared via their structural similarities and noise analysis, where differences in video quality before and after streaming revealed themselves through measured values. Study of the resulting graphs indicates that special events such as changing bit rates, missing frames and other deviations can be identified with the help of this measurement setup. The project has thus achieved its goal of producing a recording solution for streamed video that can be used in video quality control. This provides good conditions for continued work with evaluation of the quality of video calls.

One of the worst wildfires in Portugal in 2017 on June17- 21 started at the central part of Pedrógão Grande and spread fast to the surrounding areas Góis, Pampilhosa da Serra and Arganil. The wildfire took 64 lives and a large smoke plume was observed. The interest in smoke plumes from wildfires is partly due to their emitting of greenhouse gases (CO2), a large source of aerosols, CO, oxides of nitrogen and other trace gases that can affect the air quality at local and regional scale. The regional scale can be affected because the smoke from wildfires can get elevated and be transported into the free troposphere and the lower stratosphere by either pyro convection or radiative driven convection and can be transported long distances, for example from Canada to Germany. This thesis investigates how the emission source height in a model affects the transport of the smoke plume and compares the simulations with observations. Observations of transport of emissions from wildfires are often done with satellites and in this thesis data from the second modern-era retrospective analysis for research and applications (MERRA2) is used as the observations. In this thesis the numerical model FLEXPART 10.02 is used to calculate the transportation of CO from the wildfire in Pedrógão Grande. The altitude of the emission source top height and bottom height in FLEXPART was changed to see how it affected the smoke plume in the simulation. The agreement between plumes from the observations and the simulation plumes were calculated with the structural similarity (SSIM) index and the change of SSIM index was investigated. The results were that the best similarity for horizontal images was with an emission source height of 100- 300 m, for vertical images at 40°N with an emission source height 0- 1500 m and for vertical images at 41°N with an emission source height 100- 1200. The overall best simulation was the simulation with emission source height 100- 1200 m (average of the three similarity calculations). Some uncertainty occurs in the results due to for example differences in resolutions between MERRA2 and FLEXPART and the weather condition may have contributed. To improve the results there is a need to compare simulations with more wildfires to see that the SSIM index behaves the same. / En av de värsta skogsbränderna i Portugal 2017 var den 17–21 juni och började i centrala Pedrógão Grande och spred sig snabbt till de omgivande områdena Góis, Pampilhosa da Serra och Arganil. Skogsbranden tog 64 liv och stora rökplymer observerades. Rökplymer från skogsbränder är intressanta därför att de bland annat är en källa av växthusgaser till exempel koldioxid (CO2), aerosoler, kolmonoxid (CO) och andra spårgaser som kan påverka luftkvaliteten på lokal och regional skala. Den regionala skalan kan påverkas eftersom rök från skogsbränder kan sprida sig upp till den fria troposfären och den nedre stratosfären med konvektion. Konvektion är när luften blir varm av olika skäl och stiger, luften kyls av när den stiger och blir då tyngre och till slut sjunker luften. Det som kan driva konvektion vid en brand är branden själv och strålning och kan transportera röken från branden långa avstånd, till exempel från Kanada till Tyskland. Den här uppsatsen undersöker hur utsläppshöjden (mellan vilka höjder röken släpps ut från) påverkar transporten av rök och jämför beräkningarna med observationer. Observationer av transport av utsläpp från bränder sker ofta med satelliter och i denna uppsats används data från en återanalys av en samling observationer från MERRA2 (the second modern-era retrospective analysis for research and applications) som observationer. En återanalys är ett dataset som räknas med en modell från i tid och rum oregelbundna observationer från historiska och nutida data. I denna uppsats används den numeriska transportmodellen FLEXPART 10.02 för att beräkna transport av kolmonoxid från skogsbranden i Pedrógão Grande. Höjden av utsläppskällans topphöjd och bottenhöjd i FLEXPART ändrades för att se hur det påverkat rökplymen i simuleringen. Likheten mellan rökplymen från observationerna och simuleringarna beräknades sedan med ett test kallat SSIM och förändringen av SSIM indexet undersöktes. Resultaten var att den bästa simuleringen för de horisontella bilderna med en utsläppshöjd på 100– 300 m, för de vertikala bilderna på 40°N en utsläppshöjd på 0– 1500 m och för de vertikala bilderna på 41°N en utsläppshöjd på 100– 1200 m. Totalt sett (medelvärde från de tre simuleringarna) hade emissions höjden 100- 1200 m den bästa likheten med observationerna. Det finns en osäkerhet i resultaten på grund av tillexempel att det är olika upplösning i MERRA2 och FLEXPART och att väderförhållanden kan ha påverkat beräkningarna. För att förbättra resultatet så kan man göra simuleringar av fler skogsbränder för att se om SSIM indexet uppför sig densamma.

A utilização de vídeos em laboratórios baseados em redes de computadores (weblabs) para pesquisa de abelhas está sendo conduzida como parte do projeto ViNCES (Virtual Network Center of Ecosystem Services), um consórcio de pesquisa composto pelo LAA (Laboratório de Automação Agrícola da Escola Politécnica da Universidade de São Paulo) e o Laboratório de Abelhas do Departamento de Ecologia do Instituto de Biociências da Universidade de São Paulo. O ViNCES é um membro do Kyatera, uma rede de computadores acadêmica de alta velocidade que tem como objetivos o desenvolvimento de aplicações e tecnologias para a Internet Avançada. Nesse contexto, o Weblab de abelhas, denominado BBBee, permite a realização de pesquisa e observação à distância da entrada de uma colméias, sendo coletadas e transmitidas informações de condições climáticas, de áudio e vídeo. O áudio e o vídeo da colméia são transmitidos em rede local e pela Internet por meio de streaming em tempo real. Os requisitos de largura de banda para transmissões de vídeo costumam ser elevados, e a qualidade obtida após o processo de compressão convencional, em geral, é baixa, podendo afetar ou dificultar a pesquisa. Além disso, está prevista a transmissão de vídeos de Alta Definição para weblab com o objetivo de melhorar a qualidade da informação, o que irá demandar maior largura de banda. Este trabalho propõe um algoritmo de codificação de vídeo para melhorar a qualidade das imagens transmitidas, ao mesmo tempo em que se reduzem os requisitos de largura de banda de transmissão e armazenagem, por meio da aplicação de técnicas de segmentação e rastreamento de objetos de vídeo. O algoritmo proposto foi implementado e testado utilizando vídeos capturados nas colméias e os resultados das análises em diversas condições são apresentados neste documento. Para avaliar a qualidade resultante do processo, foi utilizada uma métrica objetiva que permite avaliar a qualidade das imagens em relação às imagens de origem. Os resultados das análises proveram informações e direções para melhorias e futuras implementações. / The use of videos in web based laboratories over computer networks, known as weblabs, is being conducted for bee research as part of the ViNCES (Virtual Network Center of Ecosystem Services) project, a research consortium composed by: LAA (Laboratório de Automação Agrícola da Escola Politécnica) and the Laboratório de Abelhas of the Ecology Department of the Biosciences Institute. ViNCES is a member of Kyatera, a high speed academic computer network which aims at developing aplications and technologies for the Advanced Internet. The bee weblab, called BBBee, allows the research and observation of the interior of a beehive, in which are collected climate, audio and video data. Real time audio and video are broadcasted over the Internet and through a local network using streaming format. The bandwidth requirements for video transmission are usually high and the video quality after conventional compression standards is, in general, low and can interfere in the biological research. Also, High-Definition video transmission for weblab is planned to be implemented in order to provide better tools for the researcher, which will demand even higher network bandwidth. On this scenario, this work intends to provide a video codification algorithm to enhance the quality of the transmitted images, at the region of interest, and also reduce the transmission and storage requirements for some situations by using segmentation and tracking techniques for videos. A complete CODEC was implemented and tested with actual video, quality and data rates were measured and evaluated using an objective metric. The results of the analysis provided some indications of directions for future enhancements and implementations.

A utilização de vídeos em laboratórios baseados em redes de computadores (weblabs) para pesquisa de abelhas está sendo conduzida como parte do projeto ViNCES (Virtual Network Center of Ecosystem Services), um consórcio de pesquisa composto pelo LAA (Laboratório de Automação Agrícola da Escola Politécnica da Universidade de São Paulo) e o Laboratório de Abelhas do Departamento de Ecologia do Instituto de Biociências da Universidade de São Paulo. O ViNCES é um membro do Kyatera, uma rede de computadores acadêmica de alta velocidade que tem como objetivos o desenvolvimento de aplicações e tecnologias para a Internet Avançada. Nesse contexto, o Weblab de abelhas, denominado BBBee, permite a realização de pesquisa e observação à distância da entrada de uma colméias, sendo coletadas e transmitidas informações de condições climáticas, de áudio e vídeo. O áudio e o vídeo da colméia são transmitidos em rede local e pela Internet por meio de streaming em tempo real. Os requisitos de largura de banda para transmissões de vídeo costumam ser elevados, e a qualidade obtida após o processo de compressão convencional, em geral, é baixa, podendo afetar ou dificultar a pesquisa. Além disso, está prevista a transmissão de vídeos de Alta Definição para weblab com o objetivo de melhorar a qualidade da informação, o que irá demandar maior largura de banda. Este trabalho propõe um algoritmo de codificação de vídeo para melhorar a qualidade das imagens transmitidas, ao mesmo tempo em que se reduzem os requisitos de largura de banda de transmissão e armazenagem, por meio da aplicação de técnicas de segmentação e rastreamento de objetos de vídeo. O algoritmo proposto foi implementado e testado utilizando vídeos capturados nas colméias e os resultados das análises em diversas condições são apresentados neste documento. Para avaliar a qualidade resultante do processo, foi utilizada uma métrica objetiva que permite avaliar a qualidade das imagens em relação às imagens de origem. Os resultados das análises proveram informações e direções para melhorias e futuras implementações. / The use of videos in web based laboratories over computer networks, known as weblabs, is being conducted for bee research as part of the ViNCES (Virtual Network Center of Ecosystem Services) project, a research consortium composed by: LAA (Laboratório de Automação Agrícola da Escola Politécnica) and the Laboratório de Abelhas of the Ecology Department of the Biosciences Institute. ViNCES is a member of Kyatera, a high speed academic computer network which aims at developing aplications and technologies for the Advanced Internet. The bee weblab, called BBBee, allows the research and observation of the interior of a beehive, in which are collected climate, audio and video data. Real time audio and video are broadcasted over the Internet and through a local network using streaming format. The bandwidth requirements for video transmission are usually high and the video quality after conventional compression standards is, in general, low and can interfere in the biological research. Also, High-Definition video transmission for weblab is planned to be implemented in order to provide better tools for the researcher, which will demand even higher network bandwidth. On this scenario, this work intends to provide a video codification algorithm to enhance the quality of the transmitted images, at the region of interest, and also reduce the transmission and storage requirements for some situations by using segmentation and tracking techniques for videos. A complete CODEC was implemented and tested with actual video, quality and data rates were measured and evaluated using an objective metric. The results of the analysis provided some indications of directions for future enhancements and implementations.

Objective Image and Video Quality Assessment (I/VQA) measures predict image/video quality as perceived by human beings - the ultimate consumers of visual data. Existing research in the area is mainly limited to benchmarking and monitoring of visual data. The use of I/VQA measures in the design and optimization of image/video processing algorithms and systems is more desirable, challenging and fruitful but has not been well explored. Among the recently proposed objective I/VQA approaches, the structural similarity (SSIM) index and its variants have emerged as promising measures that show superior performance as compared to the widely used mean squared error (MSE) and are computationally simple compared with other state-of-the-art perceptual quality measures. In addition, SSIM has a number of desirable mathematical properties for optimization tasks. The goal of this research is to break the tradition of using MSE as the optimization criterion for image and video processing algorithms. We tackle several important problems in visual communication applications by exploiting SSIM-inspired design and optimization to achieve significantly better performance. Firstly, the original SSIM is a Full-Reference IQA (FR-IQA) measure that requires access to the original reference image, making it impractical in many visual communication applications. We propose a general purpose Reduced-Reference IQA (RR-IQA) method that can estimate SSIM with high accuracy with the help of a small number of RR features extracted from the original image. Furthermore, we introduce and demonstrate the novel idea of partially repairing an image using RR features. Secondly, image processing algorithms such as image de-noising and image super-resolution are required at various stages of visual communication systems, starting from image acquisition to image display at the receiver. We incorporate SSIM into the framework of sparse signal representation and non-local means methods and demonstrate improved performance in image de-noising and super-resolution. Thirdly, we incorporate SSIM into the framework of perceptual video compression. We propose an SSIM-based rate-distortion optimization scheme and an SSIM-inspired divisive optimization method that transforms the DCT domain frame residuals to a perceptually uniform space. Both approaches demonstrate the potential to largely improve the rate-distortion performance of state-of-the-art video codecs. Finally, in real-world visual communications, it is a common experience that end-users receive video with significantly time-varying quality due to the variations in video content/complexity, codec configuration, and network conditions. How human visual quality of experience (QoE) changes with such time-varying video quality is not yet well-understood. We propose a quality adaptation model that is asymmetrically tuned to increasing and decreasing quality. The model improves upon the direct SSIM approach in predicting subjective perceptual experience of time-varying video quality.