by Chan Ho Chow. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2001. / Includes bibliographical references (leaves 65-67). / Abstracts in English and Chinese. / Chapter Chapter 1 --- Introduction --- p.1 / Chapter Chapter 2 --- Background --- p.5 / Chapter 2.1. --- Packet Loss in the Internet --- p.5 / Chapter 2.2. --- Shared Internet --- p.7 / Chapter 2.3. --- Video Streaming --- p.10 / Chapter 2.4. --- Real-time video transmission criteria --- p.11 / Chapter 2.5. --- Existing Video Streaming Protocol --- p.13 / Chapter Chapter 3 --- Transmission Control Protocol (TCP) --- p.16 / Chapter Chapter 4 --- Variable-rate Streaming TCP (VarS-TCP) --- p.22 / Chapter 4.1. --- General Idea --- p.22 / Chapter 4.2. --- Assumptions --- p.25 / Chapter 4.3. --- VarS-TCP Algorithm --- p.26 / Chapter 4.3.1. --- Connection Initialization --- p.26 / Chapter 4.3.2. --- Normal Data Transfer --- p.27 / Chapter 4.4. --- Skipping packets in TCP --- p.32 / Chapter 4.4.1. --- Types of skipped packet --- p.32 / Chapter 4.4.2. --- Acknowledging skipped packets --- p.34 / Chapter 4.4.3. --- Maintaining Normal Data Flow --- p.35 / Chapter 4.4.4. --- Congestion Control --- p.37 / Chapter 4.4.5. --- Packets skipped by receiver --- p.41 / Chapter 4.5. --- Rebuffering --- p.44 / Chapter Chapter 5 --- Simulation Result --- p.45 / Chapter 5.1. --- Accumulating Data --- p.46 / Chapter 5.2. --- Delay Constraints --- p.48 / Chapter 5.3. --- Adapting network situation --- p.50 / Chapter 5.4. --- Sharing bandwidth with TCP --- p.52 / Chapter 5.5. --- Random Traffic --- p.58 / Chapter 5.6. --- Effect of packet skip threshold --- p.59 / Chapter 5.7. --- Effect of round-trip-time --- p.61 / Chapter Chapter 6 --- Conclusion and Future Works --- p.63 / Bibliography --- p.65

創意媒體研究中，圖像/視頻再藝術作為有表現力的用戶定制外觀的創作手段受到了很大關注。交互設計中，特別是在圖像空間只有單張圖像或視頻輸入的情況下，運用計算機輔助設計虛擬地再渲染關注物體的風格化外觀來實現紋理替換是很強大的。現行的紋理替換往往通過操作圖像空間中像素的間距來處理紋理扭曲，原始圖像中潛在的紋理扭曲總是被破壞，因為現行的方法要麼存在由於手動網格拉伸導致的不恰當扭曲，要麼就由於紋理合成而導致不可避免的紋理開裂。圖像/視頻拼貼畫是被發明用以支持在顯示畫布上並行展示多個物體和活動。隨著數字視頻俘獲裝置的快速發展，相關的議題就是快速檢閱和摘要大量的視覺媒體數據集來找出關注的資料。這會是一項繁瑣的任務來審查長且乏味的監控視頻並快速把握重要信息。以關鍵信息和縮短視頻形式為交流媒介，視頻摘要是增強視覺數據集瀏覽效率和簡易理解的手段。 / 本文首先將圖像/視頻再藝術聚焦在高效紋理替換和風格化上。我們展示了一種交互紋理替換方法，能夠在不知潛在幾何結構和光照環境的情況下保持相似的紋理扭曲。我們運用SIFT 棱角特徵來自然地發現潛在紋理扭曲，並應用梯度深度圖復原和皺褶重要性優化來完成扭曲過程。我們運用GPU-CUDA 的並行性，通過實時雙邊網格和特徵導向的扭曲優化來促成交互紋理替換。我們運用基於塊的實時高精度TV-L¹光流，通過基於關鍵幀的紋理傳遞來完成視頻紋理替換。我們進一步研究了基於GPU 的風格化方法，並運用梯度優化保持原始圖像的精細結構。我們提出了一種能夠自然建模原始圖像精細結構的圖像結構圖，並運用基於梯度的切線生成和切線導向的形態學來構建這個結構圖。我們在GPU-CUDA 上通過並行雙邊網格和結構保持促成最終風格化。實驗中，我們的方法實時連續地展現了高質量的圖像/視頻的抽象再藝術。 / 當前，視頻拼貼畫大多創作靜態的基於關鍵幀的拼貼圖片，該結果只包含動態視頻有限的信息，會很大程度影響視覺數據集的理解。爲了便於瀏覽，我們展示了一種在顯示畫布上有效並行摘要動態活動的動態視頻拼貼畫。我們提出應用活動長方體來重組織及提取事件，執行視頻防抖來生成穩定的活動長方體，實行時空域優化來優化活動長方體在三維拼貼空間的位置。我們通過在GPU 上的事件相似性和移動關係優化來完成高效的動態拼貼畫，允許多視頻輸入。擁有再序核函數CUDA 處理，我們的視頻拼貼畫爲便捷瀏覽長視頻激活了動態摘要，節省大量存儲傳輸空間。實驗和調查表明我們的動態拼貼畫快捷有效，能被廣泛應用于視頻摘要。將來，我們會擴展交互紋理替換來支持更複雜的具大運動和遮蔽場景的一般視頻，避免紋理跳動。我們會採用最新視頻技術靈感使視頻紋理替換更加穩定。我們未來關於視頻拼貼畫的工作包括審查監控業中動態拼貼畫應用，並研究含有大量相機運動和不同種視頻過度的移動相機和一般視頻。 / Image/video restyling as an expressive way for producing usercustomized appearances has received much attention in creative media researches. In interactive design, it would be powerful to re-render the stylized presentation of interested objects virtually using computer-aided design tools for retexturing, especially in the image space with a single image or video as input. The nowaday retexturing methods mostly process texture distortion by inter-pixel distance manipulation in image space, the underlying texture distortion is always destroyed due to limitations like improper distortion caused by human mesh stretching, or unavoidable texture splitting caused by texture synthesis. Image/ video collage techniques are invented to allow parallel presenting of multiple objects and events on the display canvas. With the rapid development of digital video capture devices, the related issues are to quickly review and brief such large amount of visual media datasets to find out interested video materials. It will be a tedious task to investigate long boring surveillance videos and grasp the essential information quickly. By applying key information and shortened video forms as vehicles for communication, video abstraction and summary are the means to enhance the browsing efficiency and easy understanding of visual media datasets. / In this thesis, we first focused our image/video restyling work on efficient retexturing and stylization. We present an interactive retexturing that preserves similar texture distortion without knowing the underlying geometry and lighting environment. We utilized SIFT corner features to naturally discover the underlying texture distortion. The gradient depth recovery and wrinkle stress optimization are applied to accomplish the distortion process. We facilitate the interactive retexturing via real-time bilateral grids and feature-guided distortion optimization using GPU-CUDA parallelism. Video retexturing is achieved through a keyframe-based texture transferring strategy using accurate TV-L¹ optical flow with patch motion tracking techniques in real-time. Further, we work on GPU-based abstract stylization that preserves the fine structure in the original images using gradient optimization. We propose an image structure map to naturally distill the fine structure of the original images. Gradientbased tangent generation and tangent-guided morphology are applied to build the structure map. We facilitate the final stylization via parallel bilateral grids and structure-aware stylizing in real-time on GPU-CUDA. In the experiments, our proposed methods consistently demonstrate high quality performance of image/video abstract restyling in real-time. / Currently, in video abstraction, video collages are mostly produced with static keyfame-based collage pictures, which contain limited information of dynamic videos and in uence understanding of visual media datasets greatly. We present dynamic video collage that effectively summarizes condensed dynamic activities in parallel on the canvas for easy browsing. We propose to utilize activity cuboids to reorganize and extract dynamic objects for further collaging, and video stabilization is performed to generate stabilized activity cuboids. Spatial-temporal optimization is carried out to optimize the positions of activity cuboids in the 3D collage space. We facilitate the efficient dynamic collage via event similarity and moving relationship optimization on GPU allowing multi-video inputs. Our video collage approach with kernel reordering CUDA processing enables dynamic summaries for easy browsing of long videos, while saving huge memory space for storing and transmitting them. The experiments and user study have shown the efficiency and usefulness of our dynamic video collage, which can be widely applied for video briefing and summary applications. In the future, we will further extend the interactive retexturing to more complicated general video applications with large motion and occluded scene avoiding textures icking. We will also work on new approaches to make video retexturing more stable by inspiration from latest video processing techniques. Our future work for video collage includes investigating applications of dynamic collage into the surveillance industry, and working on moving camera and general videos, which may contain large amount of camera motions and different types of video shot transitions. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Li, Ping. / Thesis (Ph.D.)--Chinese University of Hong Kong, 2013. / Includes bibliographical references (leaves 109-121). / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Abstracts also in Chinese. / Abstract --- p.i / Acknowledgements --- p.v / Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- Background --- p.1 / Chapter 1.2 --- Main Contributions --- p.5 / Chapter 1.3 --- Thesis Overview --- p.7 / Chapter 2 --- Efficient Image/video Retexturing --- p.8 / Chapter 2.1 --- Introduction --- p.8 / Chapter 2.2 --- Related Work --- p.11 / Chapter 2.3 --- Image/video Retexturing on GPU --- p.16 / Chapter 2.3.1 --- Wrinkle Stress Optimization --- p.19 / Chapter 2.3.2 --- Efficient Video Retexturing --- p.24 / Chapter 2.3.3 --- Interactive Parallel Retexturing --- p.29 / Chapter 2.4 --- Results and Discussion --- p.35 / Chapter 2.5 --- Chapter Summary --- p.41 / Chapter 3 --- Structure-Aware Image Stylization --- p.43 / Chapter 3.1 --- Introduction --- p.43 / Chapter 3.2 --- Related Work --- p.46 / Chapter 3.3 --- Structure-Aware Stylization --- p.50 / Chapter 3.3.1 --- Approach Overview --- p.50 / Chapter 3.3.2 --- Gradient-Based Tangent Generation --- p.52 / Chapter 3.3.3 --- Tangent-Guided Image Morphology --- p.54 / Chapter 3.3.4 --- Structure-Aware Optimization --- p.56 / Chapter 3.3.5 --- GPU-Accelerated Stylization --- p.58 / Chapter 3.4 --- Results and Discussion --- p.61 / Chapter 3.5 --- Chapter Summary --- p.66 / Chapter 4 --- Dynamic Video Collage --- p.67 / Chapter 4.1 --- Introduction --- p.67 / Chapter 4.2 --- Related Work --- p.70 / Chapter 4.3 --- Dynamic Video Collage on GPU --- p.74 / Chapter 4.3.1 --- Activity Cuboid Generation --- p.75 / Chapter 4.3.2 --- Spatial-Temporal Optimization --- p.80 / Chapter 4.3.3 --- GPU-Accelerated Parallel Collage --- p.86 / Chapter 4.4 --- Results and Discussion --- p.90 / Chapter 4.5 --- Chapter Summary --- p.100 / Chapter 5 --- Conclusion --- p.101 / Chapter 5.1 --- Research Summary --- p.101 / Chapter 5.2 --- Future Work --- p.104 / Chapter A --- Publication List --- p.107 / Bibliography --- p.109

"Packet loss can be detrimental to real-time interactive video over lossy networks because one lost video packet can propagate errors to many subsequent video frames due to the encoding dependency between frames. Feedback-based error control techniques use feedback information from the decoder to adjust coding parameters at the encoder or retransmit lost packets to reduce the error propagation due to data loss. Feedback-based error control techniques have been shown to be more effective than trying to conceal the error at the encoder or decoder alone since they allow the encoder and decoder to cooperate in the error control process. However, there has been no systematic exploration of the impact of video content and network conditions on the performance of feedback-based error control techniques. In particular, the impact of packet loss, round-trip delay, network capacity constraint, video motion and reference distance on the quality of videos using feedback-based error control techniques have not been systematically studied. This thesis presents analytical models for the major feedback-based error control techniques: Retransmission, Reference Picture Selection (both NACK and ACK modes) and Intra Update. These feedback-based error control techniques have been included in H.263/H.264 and MPEG4, the state of the art video in compression standards. Given a round-trip time, packet loss rate, network capacity constraint, our models can predict the quality for a streaming video with retransmission, Intra Update and RPS over a lossy network. In order to exploit our analytical models, a series of studies has been conducted to explore the effect of reference distance, capacity constraint and Intra coding on video quality. The accuracy of our analytical models in predicting the video quality under different network conditions is validated through simulations. These models are used to examine the behavior of feedback-based error control schemes under a variety of network conditions and video content through a series of analytic experiments. Analysis shows that the performance of feedback-based error control techniques is affected by a variety of factors including round-trip time, loss rate, video content and the Group of Pictures (GOP) length. In particular: 1) RPS NACK achieves the best performance when loss rate is low while RPS ACK outperforms other repair techniques when loss rate is high. However RPS ACK performs the worst when loss rate is low. Retransmission performs the worst when the loss rate is high; 2) for a given round-trip time, the loss rate where RPS NACK performs worse than RPS ACK is higher for low motion videos than it is for high motion videos; 3) Videos with RPS NACK always perform the same or better than videos without repair. However, when small GOP sizes are used, videos without repair perform better than videos with RPS ACK; 4) RPS NACK outperform Intra Update for low-motion videos. However, the performance gap between RPS NACK and Intra Update drops when the round-trip time or the intensity of video motion increases. 5) Although the above trends hold for both VQM and PSNR, when VQM is the video quality metric the performance results are much more sensitive to network loss. 6) Retransmission is effective only when the round-trip time is low. When the round-trip time is high, Partial Retransmission achieves almost the same performance as Full Retransmission. These insights derived from our models can help determine appropriate choices for feedback-based error control techniques under various network conditions and video content. "

The first part of this thesis studies advanced wavelet transform techniques for scalable still image object coding. In order to adapt to the content of a given signal and obtain more flexible adaptive representation, two advanced wavelet transform techniques, wavelet packet transform and directional wavelet transform, are developed for object-based image coding. Extensive experiments demonstrate that the new wavelet image coding systems perform comparable to or better than state-of-the-art in image compression while possessing some attractive features such as object-based coding functionality and high coding scalability. / The objective of this thesis is to develop an object-based coding framework built upon a family of wavelet coding techniques for a variety of arbitrarily shaped visual object scalable coding applications. Two kinds of arbitrarily shaped visual object scalable coding techniques are investigated in this thesis. One is object-based scalable wavelet still image coding; another is object-based scalable wavelet video coding. / The second part of this thesis investigates various components of object-based scalable wavelet video coding. A generalized 3-D object-based directional threading, which unifies the concepts of temporal motion threading and spatial directional threading, is seamlessly incorporated into 3-D shape-adaptive directional wavelet transform to exploit the spatio-temporal correlation inside the 3-D video object. To improve the computational efficiency of multi-resolution motion estimation (MRME) in shift-invariant wavelet domain, two fast MRME algorithms are proposed for wavelet-based scalable video coding. As demonstrated in the experiments, the proposed 3-D object-based wavelet video coding techniques consistently outperform MPEG-4 and other wavelet-based schemes for coding arbitrarily shaped video object, while providing full spatio-temporal-quality scalability with non-redundant 3-D subband decomposition. / Liu, Yu. / Adviser: King Ngi Ngan. / Source: Dissertation Abstracts International, Volume: 70-06, Section: B, page: 3693. / Thesis (Ph.D.)--Chinese University of Hong Kong, 2008. / Includes bibliographical references (leaves 166-173). / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Electronic reproduction. [Ann Arbor, MI] : ProQuest Information and Learning, [200-] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Abstracts in English and Chinese. / School code: 1307.

Classroom teachers are constantly trying to find the answer to those many behavior situations that arise on a daily basis in schools. In this study, we looked at the benefits of using a digital recorder in the classroom to help students become more “aware” of their behaviors. The study was done in a rural, public elementary school with three males that ranged from third grade to fifth grade. Each of these three boys has been identified as students with either specific learning disabilities or emotional disturbances. This program was run by the resource teacher and a trained paraprofessional. The methods of this study included a digital recording of classroom activities with a focus on the three students. On a daily basis, each student would watch the recording with the resource teacher and evaluate their behaviors. The students were asked to self-evaluate their behaviors before discussing them with the resource teacher. The results for two of the three subjects were a decrease in the target behavior while the third subject’s results were not consistent. With additional time and training, this subject could also benefit from this program. The benefits of this program could reach individual students to improve their classroom behaviors while benefiting the entire classroom as well. Teachers will be able to teach and students able to learn without being interrupted by behavior incidents.

Abstract not available

[Truncated abstract] Video sequence synchronization is necessary for any computer vision application that integrates data from multiple simultaneously recorded video sequences. With the increased availability of video cameras as either dedicated devices, or as components within digital cameras or mobile phones, a large volume of video data is available as input for a growing range of computer vision applications that process multiple video sequences. To ensure that the output of these applications is correct, accurate video sequence synchronization is essential. Whilst hardware synchronization methods can embed timestamps into each sequence on-the-fly, they require specialized hardware and it is necessary to set up the camera network in advance. On the other hand, computer vision-based software synchronization algorithms can be used to post-process video sequences recorded by cameras that are not networked, such as common consumer hand-held video cameras or cameras embedded in mobile phones, or to synchronize historical videos for which hardware synchronization was not possible. The current state-of-the-art software algorithms vary in their input and output requirements and camera configuration assumptions. ... Next, I describe an approach that synchronizes two video sequences where an object exhibits ballistic motions. Given the epipolar geometry relating the two cameras and the imaged ballistic trajectory of an object, the algorithm uses a novel iterative approach that exploits object motion to rapidly determine pairs of temporally corresponding frames. This algorithm accurately synchronizes videos recorded at different frame rates and takes few iterations to converge to sub-frame accuracy. Whereas the method presented by the first algorithm integrates tracking data from all frames to synchronize the sequences as a whole, this algorithm recovers the synchronization by locating pairs of temporally corresponding frames in each sequence. Finally, I introduce an algorithm for synchronizing two video sequences recorded by stationary cameras with unknown epipolar geometry. This approach is unique in that it recovers both the frame rate ratio and the frame offset of the two sequences by finding matching space-time interest points that represent events in each sequence; the algorithm does not require object tracking. RANSAC-based approaches that take a set of putatively matching interest points and recover either a homography or a fundamental matrix relating a pair of still images are well known. This algorithm extends these techniques using space-time interest points in place of spatial features, and uses nested instances of RANSAC to also recover the frame rate ratio and frame offset of a pair of video sequences. In this thesis, it is demonstrated that each of the above algorithms can accurately recover the frame rate ratio and frame offset of a range of real video sequences. Each algorithm makes a contribution to the body of video sequence synchronization literature, and it is shown that the synchronization problem can be solved using a range of approaches.

<p>I detta examensarbete har olika sätt att implementera färgrymdskonverterare i multipel konstant multiplikationsteknik beskrivits med VHDL, syntetiserats och jämförts med avseende på effektförbrukning.</p>

<p>In order to efficiently manage the large amounts of video data stored in a digital video archive, computerized management systems must be developed for storing and making the video available to users. In this thesis, we study tertiary storage technologies and storage architectures for storing and retrieving digital video in video archives.</p><p>We evaluate serpentine tape as a storage medium for digital video. In order to increase the performance of storage systems using serpentine tape, we present and evaluate a detailed access-time model for serpentine tape and a novel scheduling algorithm for optimizing concurrent accesses to the tape. The scheduling algorithm is used for evaluating serpentine tape for storing images and video sequences. The main conclusion is that by using the access-time model and the proposed scheduling algorithm, it is possible to achieve significant improvements in initial latency, average access time, and the number of requests that can be served by a single tape drive.</p><p>Tertiary storage technologies including magnetic tape and DVD are evaluated for use in digital video archives. The evaluation is performed using a simulator of the storage system of a video archive. The simulation model is based on the architecture of the Elvira~II video archive server. Different configurations for the storage system are evaluated with regards to performance and cost. In the evaluation different allocation strategies, access distributions, and user loads are studied. The effect of using a cache based on magnetic disks is investigated.</p><p>The main conclusion is that the choice of architecture and storage technology for a video archive depends on the user generated load, the size of the requested video sequences, and the access distribution for the stored videos. It also depends on whether throughput, response time, storage cost, or cost per retrieved video is the main evaluation criterion. Furthermore, we show that a video archive based on DVD as the main storage technology outperforms a video archive using magnetic tape, and that including a relatively small disk cache in most cases improves the performance and reduces the total cost of the archive. </p><p>The ideas and results presented in this thesis are also useful outside the video archive context. The strategies and results are beneficial for applications that require hierarchical storage management systems for managing large data volumes.</p>

<p>To encode, query and present the semantic content of digital video precisely and flexibly is very useful for many kinds of knowledge work: system analysis and evaluation, documentation and education, to name a few. However, that kind of video management is not a trivial matter. The traditional stratified annotation model has quite poor facilities for specifying the meaning – the structure and relationships – of the strata. Because of this, it may also be troublesome to present the annotations to the users in a clear and flexible manner.</p><p>This thesis presents <i>OntoLog</i>, a system for managing the semantic content of video. It extends the stratified annotation model by defining the <i>strata as objects and classes in ontologies</i>, thereby making their semantic meaning more explicit and relating them to each other in a semantic network. The same ontologies are also used to define properties and objects for describing both the strata, individual video intervals and entire videos. This constitutes a very customisable, expressive and precise description model, without sacrificing simplicity and conceptual integrity.</p><p>Arranging the annotation strata in a near-hierarchical network with specified semantics (classes, subclasses and instances) also enables reasoning about the annotations during query and browsing. In particular, it enables <i>visual aggregation of traditional timeline-based strata graphics</i>. Using this to create compact content visualisations, the OntoLog system is able to present tens of videos on screen at the same time, thus providing<i> inter-video browsing</i>. By judiciously disaggregating selected parts of the strata hierarchy, users can focus on relevant strata at their preferred level of detail – <i>overview-and-zoom</i> functionality for semantic annotations, in other words.</p><p>The OntoLog system has been implemented in the form of six Java applications and web services – together covering annotation editing, browsing, analysis, search, query and presentation with various approaches – built on top of an RDF database founded on SQL. The system has been tested under realistic conditions in several real-world projects, with good results. A novel information gathering interface for OntoLog data, Savanta, has been created. This is based on an iterative interaction paradigm featuring inter-video browsing, filtering, navigation and context-sensitive temporal analysis of the annotations. In a comparative usability evaluation, <i>Savanta</i> is shown to outperform more traditional user interfaces for video search/browsing with regard to expressive power, straightforwardness and user satisfaction.</p>