<p>Recent advancements in deep learning (DL) and high-communication bandwidth access networks such as 5G enable applications that require intelligence and faster computational power at the edge with low power consumption. In this thesis, we study how to improve the Quality-of-Experience (QoE) of these emerging 5G applications, e.g., virtual reality (VR) and video analytics on edge devices. These 5G applications either require high-quality visual effects with a stringent latency requirement (for VR) or high analytics accuracy (for video analytics) while maintaining frame rate requirements under dynamic conditions. </p> <p>In part 1, we study how to support high-quality untethered immersive multiplayer VR on commodity mobile devices. Simply replicating the prior-art for a single-user VR will result in a linear increase in network bandwidth requirement that exceeds the bandwidth of WiFi (802.11ac). We propose a novel technique, <em>Coterie, </em>that splits the rendering of background environment (BE) frames between the mobile device and the edge server that drastically enhances the similarity of the BE frames and reduces the network load via frame caching and reuse. Our proposed VR framework, Coterie, reduces per-player network requirement by over 10x and easily supports 4 players on Pixel 2 over 802.11ac while maintaining the QoE constraints of 4K VR.</p> <p>In part 2, we study how to achieve high accuracy of analytics in video analytics pipelines (VAP). We observe that the frames captured by the surveillance video cameras powering a variety of 24X7 analytics applications are not always pristine -- they can be distorted due to environmental condition changes, lighting issues, sensor noise, compression, etc. Such distortions not only deteriorate the accuracy of deep learning applications but also negatively impact the utilization of the edge server resources used to run these computationally expensive DL models. First, we study how to dynamically filter out low-quality frames captured. We propose a lightweight DL-based quality estimator, <em>AQuA</em>, that can be used to filter out low-quality frames that can lead to high-confidence errors (false-positives) if fed into the analytic units (AU) in the VAP. AQuA-filter reduces false positives by 17% and the compute and network usage by up to 27% when used in a face-recognition VAP. Second, we study how to reduce such poor-quality frame captures by the camera. We propose <em>CamTuner, </em>a system that automatically and dynamically adapts the complex camera settings to changing environmental conditions based on analytical quality estimation to enhance the accuracy of video analytics. In a real customer deployment, <em>CamTuner</em> enhances VAP accuracy by detecting 15.9% additional persons and 2.6%–4.2% additional cars (without any false positives) than the default camera setting. While <em>CamTuner</em> focuses on improving the accuracy of single-AU running on a camera stream, next we present <em>Elixir</em>, a system that enhances the video stream quality for multiple analytics on a video stream by jointly optimizing different AUs’ objectives. In a real-world deployment, <em>Elixir</em> correctly detects 7.1% (22,068) and 5.0% (15,731) more cars, 94% (551) and 72% (478) more faces, and 670.4% (4975) and 158.6% (3507) more persons than the default-camera-setting and time-sharing approaches, respectively.</p>

News videos play an important rule in shaping our everyday communication. At the same time, news videos use narrative patterns to keep people entertained. Understanding how these patterns work and are being applied in news videos is crucial for understanding how they may affect a videos ideological message, which is an important dimension in times of fake news and disinformation campaigns. We present Zoetrope, a web-based tool that supports the discovery of narrative patterns in news videos by means of a visual exploration approach. Zoetrope integrates a number of multimodal information extraction frameworks into an interactive visualization, to allow for an efficient exploratory access to large collections of news videos

<p dir="ltr">Agricultural applications often require human experts with domain knowledge to ensure compliance and improve productivity, which can be costly and inefficient. To tackle this problem, automated video systems can be implemented for agricultural tasks thanks to the ubiquity of cameras. In this thesis, we focus on designing and implementing video analytics systems for real applications in agriculture by combining both traditional image processing and recent advancements in computer vision. Existing research and available methods have been heavily focused on obtaining the best performance on large-scale benchmarking datasets, while neglecting the applications to real-world problems. Our goal is to bridge the gap between state-of-art methods and real agricultural applications. More specifically, we design video systems for the two tasks of monitoring turkey behavior for turkey welfare and handwashing action recognition for improved food safety. For monitoring turkeys, we implement a turkey detector, a turkey tracker, and a turkey head tracker by combining object detection and multi-object tracking. Furthermore, we detect turkey activities by incorporating motion information. For recognizing handwashing activities, we combine a hand extraction method for focusing on the hand regions with a neural network to build a hand image classifier. In addition, we apply a two-stream network with RGB and hand streams to further improve performance and robustness.</p><p dir="ltr">Besides designing a robust hand classifier, we explore how dataset attributes and distribution shifts can impact system performance. In particular, distribution shifts caused by changes in hand poses and shadow can cause a classifier’s performance to degrade sharply or breakdown beyond a certain point. To better explore the impact of hand poses and shadow and to mitigate the induced breakdown points, we generate synthetic data with desired variations to introduce controlled distribution shift. Experimental results show that the breakdown points are heavily impacted by pose and shadow conditions. In addition, we demonstrate mitigation strategies to significant performance degradation by using selective additional training data and adding synthetic shadow to images. By incorporating domain knowledge and understanding the applications, we can effectively design video analytics systems and apply advanced techniques in agricultural scenarios.</p>

No description available.

No description available.