Human-computer interaction (HCI) is being revolutionized by computational design and artificial intelligence. As the diversity of user interfaces shifts from personal desktops to mobile and wearable devices, yesterday’s tools and interfaces are insufficient to meet the demands of tomorrow’s devices. This dissertation describes my research on leveraging different physical channels (e.g., vibration, light, capacitance) to enable novel interaction opportunities. We first introduce FontCode, an information embedding technique for text documents. Given a text document with specific fonts, our method can embed user-specified information (e.g., URLs, meta data, etc) in the text by perturbing the glyphs of text characters while preserving the text content. The embedded information can later be retrieved using a smartphone in real time. Then, we present Vidgets, a family of mechanical widgets, specifically push buttons and rotary knobs that augment mobile devices with tangible user interfaces. When these widgets are attached to a mobile device and a user interacts with them, the nonlinear mechanical response of the widgets shifts the device slightly and quickly. Subsequently, this subtle motion can be detected by the Inertial Measurement Units (IMUs), which is commonly installed on mobile devices. Next, we propose BackTrack, a trackpad placed on the back of a smartphone to track finegrained finger motions. Our system has a small form factor, with all the circuits encapsulated in a thin layer attached to a phone case. It can be used with any off-the-shelf smartphone, requiring no power supply or modification of the operating systems. BackTrack simply extends the finger tracking area of the front screen, without interrupting the use of the front screen. Lastly, we demonstrate MoiréBoard, a new camera tracking method that leverages a seemingly irrelevant visual phenomenon, the moiré effect. Based on a systematic analysis of the moiré effect under camera projection, MoiréBoard requires no power nor camera calibration. It can easily be made at a low cost (e.g., through 3D printing) and ready to use with any stock mobile device with a camera. Its tracking algorithm is computationally efficient and can run at a high frame rate. It is not only simple to implement, but also tracks devices at a high accuracy, comparable to the state-of-the-art commercial VR tracking systems.

Blackmagic Design destaca entre la competencia en cámaras de cine por ofrecer equipos con calidad cinematográficas a un precio menor a marcas como ARRI y RED. Los diferentes modelos han sido la opción preferida entre cineastas independientes y estudiantes audiovisuales como primer equipo con capacidad de crear contenido en calidad cine estándar. Con el dominio de las cámaras DSLR por su flexibilidad en imagen y video, la investigación analiza las diferencias y los beneficios que ambos equipos ofrecen en calidad de imagen y formato recopilando la opinión tanto de profesionales audiovisuales en dirección de fotografía y edición como de estudiantes audiovisuales. Toda la investigación se desarrollará desde la industria peruana haciendo recopilación de material audiovisual realizado con ambos tipos de equipos apoyado por la opinión de profesionales que hayan hecho uso de cámaras Blackmagic Design, sustentado desde el hecho que son igualmente accesibles tanto en adquisición como alquiler. La investigación ayudará a que estudiantes audiovisuales puedan contar con información acerca de las cámaras de cine y la experiencia de uso para hacer un mejor juicio de qué equipo es ideal segun el proyecto que anhele realizar. / Blackmagic Design is known among other cinema camera companies for offering cinema quality equipment for a lower price than ARRI and RED. The different models have been the favorite choice among independent filmmakers and audiovisual students as a first camera capable of creating content in standard cinema quality. With the dominance of DSLR cameras for their flexibility in image and video, the research analyzes the differences and benefits that both offer in terms of image quality and format collecting the opinion of both audiovisual professionals such direction of photography and video editing as much as students. The whole research is made from the peruvian industry collecting audiovisual material made by both camera types and experiences of professionals that tested Blackmagic Design cameras by the fact that they are equally accessible for buying or rent. The research will help students to have information about cinema cameras and experiences to make better choices of which equipment is ideal for the project they want to shoot. / Trabajo de investigación

Several questionable officer involved shootings and perceived abuses of authority disproportionately involving minority citizens have resulted in public outcry, protests, and nationwide scrutiny of police in recent years. The resulting police legitimacy crisis has prompted agencies to rapidly equip officers with body-worn video cameras (BWCs). BWC advocates lauded the findings of an early study that attributed significant reductions in use of force incidents and citizen complaints to the devices and it is this and a handful of other short-term studies upon which the claims of these benefits are predicated. However, subsequent research has produced mixed findings and the sustainability of any reductions remains questionable. The limited knowledge concerning the impact of BWCs on the aforementioned outcomes is problematic considering the potential negative impact of unrealistic expectations and the expense of BWC program maintenance. The objective of this dissertation is to address gaps in the extant research by exploring the impact of an incremental deployment of the devices on the frequency and severity of use of force incidents and the frequency and outcome of citizen complaints while controlling for staffing, volume of officer-initiated enforcement contacts, and the Ferguson incident. Utilizing 86-months of secondary data collected from the Newport News, Virginia Police Department (NNPD) a vector autoregressive multivariate time series analysis indicates that BWCs were a significant factor in a substantial sustained reduction in use of force and a substantial sustained increase in exonerated complaint dispositions at the NNPD.

The wastewater- and sewage distribution network is a critical aspect for maintaining a socialandenvironmental well-being. However, in the latest time, many failures (such as leakages) haveoccurred due to poor conditions in the wastewater-infrastructure, causing several consequencessuch as damage to the environment or pose health hazards to the public. This study is dividedinto three parts. Part one compares five different sewage inspection technologies: Closed-CircuitTelevision (CCTV), Zoom Cameras, Digital Scanning, SONAR and Electro Scanning with thepurpose of identifying a new system that Stockholm Vatten & Avfall (SVOA) should considerimplementing. Part two goes through the inspection procedure that SVOA applies, andrecommendations are concluded as to how the process can be optimized. Part three coversessential aspects that should be taken into account before investing in a new technology. Theresearch methodology involves collecting data from literature, internal reports from SVOA, andconducting interviews with supervisors and field workers. The main finding is that SVOA shouldcontinue with the current approach that the utility is using because the current data found in thecurrent literature is too insufficient and not too strong which makes it not convincing to investinto a new technology. On the other hand, the collected data about the technologies can beconsidered whenever the approach is moving towards a proactive approach.

In this dissertation, we present vision based scene interpretation methods for monitoring of people and vehicles, in real-time, within a busy environment using a forest of co-operative electro-optical (EO) sensors. We have developed novel video understanding algorithms with learning capability, to detect and categorize people and vehicles, track them with in a camera and hand-off this information across multiple networked cameras for multi-camera tracking. The ability to learn prevents the need for extensive manual intervention, site models and camera calibration, and provides adaptability to changing environmental conditions. For object detection and categorization in the video stream, a two step detection procedure is used. First, regions of interest are determined using a novel hierarchical background subtraction algorithm that uses color and gradient information for interest region detection. Second, objects are located and classified from within these regions using a weakly supervised learning mechanism based on co-training that employs motion and appearance features. The main contribution of this approach is that it is an online procedure in which separate views (features) of the data are used for co-training, while the combined view (all features) is used to make classification decisions in a single boosted framework. The advantage of this approach is that it requires only a few initial training samples and can automatically adjust its parameters online to improve the detection and classification performance. Once objects are detected and classified they are tracked in individual cameras. Single camera tracking is performed using a voting based approach that utilizes color and shape cues to establish correspondence in individual cameras. The tracker has the capability to handle multiple occluded objects. Next, the objects are tracked across a forest of cameras with non-overlapping views. This is a hard problem because of two reasons. First, the observations of an object are often widely separated in time and space when viewed from non-overlapping cameras. Secondly, the appearance of an object in one camera view might be very different from its appearance in another camera view due to the differences in illumination, pose and camera properties. To deal with the first problem, the system learns the inter-camera relationships to constrain track correspondences. These relationships are learned in the form of multivariate probability density of space-time variables (object entry and exit locations, velocities, and inter-camera transition times) using Parzen windows. To handle the appearance change of an object as it moves from one camera to another, we show that all color transfer functions from a given camera to another camera lie in a low dimensional subspace. The tracking algorithm learns this subspace by using probabilistic principal component analysis and uses it for appearance matching. The proposed system learns the camera topology and subspace of inter-camera color transfer functions during a training phase. Once the training is complete, correspondences are assigned using the maximum a posteriori (MAP) estimation framework using both the location and appearance cues. Extensive experiments and deployment of this system in realistic scenarios has demonstrated the robustness of the proposed methods. The proposed system was able to detect and classify targets, and seamlessly tracked them across multiple cameras. It also generated a summary in terms of key frames and textual description of trajectories to a monitoring officer for final analysis and response decision. This level of interpretation was the goal of our research effort, and we believe that it is a significant step forward in the development of intelligent systems that can deal with the complexities of real world scenarios.

Roads pose two central problems for wildlife: wildlife-vehicle collisions (WVCs) and habitat fragmentation. Wildlife exclusion fencing can reduce WVCs but can exacerbate fragmentation. In Chapter 1, I summarize the relevant studies addressing these two problems, with a focus on large mammals in North America. Chapters 2 and 3 summarize field assessments of technologies to reduce WVCs and maintain connectivity, specifically jumpout ramps and underpasses, along Highway 101 near San Luis Obispo, CA. In a fenced highway, some animals inevitably breach the fence and become trapped, which increases the risk of a wildlife-vehicle collision. Earthen escape ramps, or “jumpouts”, can allow the trapped animal to escape the highway corridor. Few studies have quantified wildlife use of jumpouts, and none for >2 years. We used wildlife cameras to quantify wildlife use of 4 jumpouts from 2012-2017. Mule deer were 88% percent of our detections and jumped out 20% of the time. After accounting for pseudoreplication, 33% of the events were independent events, and 2 groups of deer accounted for 41% of all detections at the top of the jumpout. Female deer were 86% of the detections and were much more likely than males to return to the jumpout multiple times. This is the first study to document use of jumpouts for more than 3 years, the first to account for pseudoreplication, and the first to quantify differences in jumpout use between male and female mule deer. We recommend a jumpout height between 1.75m-2m for mule deer to increase the jumpout success rate. Chapter 3 addresses factors that may affect the use of undercrossings by mule deer and other wildlife. Wildlife crossings combined with wildlife exclusion fencing have been shown to be the most effective method to reduce wildlife-vehicle collisions while maintaining ecological connectivity. Although several studies have quantified wildlife use of undercrossings, very few have exceeded 24 months, and the factors affecting carnivores use of the undercrossings remain unclear. We quantified mule deer, black bear, mountain lion, and bobcat use of 11 undercrossings along Highway 101 near San Luis Obispo, California from 2012-2017. We constructed zero-inflated Poisson general linear models on the monthly activity of our focal species using underpass dimensionality, distance to cover, substrate, human activity, and location relative to the wildlife exclusion fence as predictor variables. We accounted for temporal variation, as well as spatial variation by quantifying the landscape resistance near each undercrossing. We found that deer almost exclusively used the larger underpasses whereas the carnivores were considerably less selective. Bears used undercrossings more that were within the wildlife exclusion fence, whereas mountain lion activity was higher outside the wildlife exclusion fence. Bobcat activity was highest and most widespread, and was negatively associated with distance to cover. Regional connectivity is most important for bear and mountain lion, and the surrounding habitat may be the most important predictor for their use of undercrossings. We recommend placing GPS collars on our focal species to more clearly document fine-scale habitat selection near the highway.

Bolometers are the most sensitive detectors for measuring millimeter and submillimeter wavelength astrophysical signals. Cameras comprised of arrays of bolometers have already made significant contributions to the field of astronomy. A challenge for bolometer cameras is obtaining observations at multiple wavelengths. Traditionally, observing in multiple bands requires a partial disassembly of the instrument to replace bandpass filters, a task which prevents immediate spectral interrogation of a source. More complex cameras have been constructed to observe in several bands using beam splitters and dichroic filters, but the added complexity leads to physically larger instruments with reduced efficiencies. The SPEctral Energy Distribution camera (SPEED) is a new type of bolometer camera designed to efficiently observe in multiple wavebands without the need for excess bandpass filters and beam splitters. SPEED is a ground-based millimeter-wave bolometer camera designed to observe at 2.1, 1.3, 1.1, and 0.85 mm simultaneously. SPEED makes use of a new type of bolometer, the frequency selective bolometer (FSB), to observe all of the wavebands within each of the camera's four pixels. FSBs incorporate frequency selective dipole surfaces as absorbing elements allowing each detector to absorb a single, narrow band of radiation and pass all other radiation with low loss. Each FSB also contains a superconducting transition-edge sensor (TES) that acts as a sensitive thermistor for measuring the temperature of the FSB. This thesis describes the development of the SPEED camera and FSB detectors. The design of the detectors used in the instrument is described as well as the the general optical performance of frequency selective dipole surfaces. Laboratory results of both the optical and thermal properties of millimeter-wave FSBs are also presented. The SPEED instrument and its components are highlighted and the optical design of the optics which couple SPEED to the Heinrich Hertz Telescope is given. This thesis concludes with an introduction to the jiggle mapping data analysis of bolometer instruments like SPEED.

High frame rate video (HFV) is an important investigational tool in sciences, engineering and military. In ultrahigh speed imaging, the obtainable temporal, spatial and spectral resolutions are limited by the sustainable throughput of in-camera mass memory, the lower bound of exposure time, and illumination conditions. In order to break these bottlenecks, we propose a new coded video acquisition framework that employs K>1 cameras, each of which makes random measurements of the video signal in both temporal and spatial domains. For each of the K cameras, this multi-camera strategy greatly relaxes the stringent requirements in memory speed, shutter speed, and illumination strength. The recovery of HFV from these random measurements is posed and solved as a large scale l1 minimization problem by exploiting joint temporal and spatial sparsities of the 3D signal. Three coded video acquisition techniques of varied trade o s between performance and hardware complexity are developed: frame-wise coded acquisition, pixel-wise coded acquisition, and column-row-wise coded acquisition. The performances of these techniques are analyzed in relation to the sparsity of the underlying video signal. To make ultra high speed cameras of coded exposure more practical and a fordable, we develop a coded exposure video/image acquisition system by an innovative assembling of multiple rolling shutter cameras. Each of the constituent rolling shutter cameras adopts a random pixel read-out mechanism by simply changing the read out order of pixel rows from sequential to random. Simulations of these new image/video coded acquisition techniques are carried out and experimental results are reported. / Dissertation / Doctor of Philosophy (PhD)

This work focuses on the high frequency restoration of video sequences captured by a hybrid camera, using key-frames as high frequency samples. The proposed method outlines a hierarchy to the super-resolution process, and is aimed at maximizing both speed and performance. Additionally, an advanced image processing simulator (EngineX) was developed to fine tune the algorithm. / Super-resolution algorithms are designed to enhance the detail level of a particular image or video sequence. However, it is very difficult to achieve in practice due to the problem being ill-posed, and often requires regularization based on assumptions about texture or edges. The process can be aided using high-resolution key-frames such as those generated from a hybrid camera. A hybrid camera is capable of capturing footage in multiple spatial and temporal resolutions. The typical output consists of a high resolution stream captured at low frame rate, and a low resolution stream captured at a high frame rate. Key-frame based super-resolution algorithms exploit the spatial and temporal correlation between the high resolution and low resolution streams to reconstruct a high resolution and high frame rate output stream. The proposed algorithm outlines a hierarchy to the super-resolution process, combining several different classical and novel methods. A residue formulation decides which pixels are required to be further reconstructed if a particular hierarchy stage fails to provide the expected results when compared to the low resolution prior. The hierarchy includes the optical flow based estimation which warps high frequency information from adjacent key-frames to the current frame. Specialized candidate pixel selection reduces the total number of pixels considered in the NLM stage. Occlusion is handled by a final fallback stage in the hierarchy. Additionally, the running time for a CIF sequence of 30 frames has been significantly reduced to within 3 minutes by identifying which pixels require reconstruction with a particular method. A custom simulation environment implements the proposed method as well as many common image processing algorithms. EngineX provides a graphical interface where video sequences and image processing methods can be manipulated and combined. The framework allows for advanced features such as multithreading, parameter sweeping, and block level abstraction which aided the development of the proposed super-resolution algorithm. Both speed and performance were fine tuned using the simulator which is the key to its improved quality over other traditional super-resolution schemes. / Thesis / Master of Applied Science (MASc)

Colonoscopy is essential for colorectal cancer screening and disease surveillance. It can remove pre-cancerous colon polyps to reduce a patient’s cancer risk. This thesis aims to improve colonoscopy’s localization using motion tracking and colonoscopy’s coverage using surface mapping. Chapter 4 describes an endoscope motion tracker that records the scope’s insertion length, rotation, and speed during a colonoscopy. The endoscope tracker’s motion record can be combined with the endoscope’s video to localize colon polyps or cancers. In the future, the device could record highly skilled manoeuvres performed by endoscopists to help train medical residents. It is difficult to image the colon’s mucosa because the colonoscope’s camera has a limited field of view. Chapter 3 uses a 180° fisheye camera to unwrap high resolution panoramas of a colon phantom. The panoramas are then combined into a mosaic map of the colon phantom’s surface. The colon’s surface is approximated as a cylinder. Follow up experiments could test our mapping algorithm using imagery from a wide-angle, high-definition colonoscope. Chapter 2 describes another technique to localize locations where polyps have been removed—blood vessel landmarks. Colonic blood vessels from a pig were imaged to determine if they could be used to fingerprint locations on the colon’s wall. Blood vessels are also useful image features for surface mapping. The proof-of-concept experiments successfully imaged large arteries but further work is needed to image the small capillaries in the colonic mucosa and to image the veins. In summary, we have visualized colonic blood vessels to test if they could be useful landmarks, tested using an extended field of view camera to create an unwrapped map of the colon wall, and designed an endoscope tracker to help localize abnormal tissue. Combining the endoscope tracker with the other two techniques should make is possible to accurately map the colon. / Thesis / Doctor of Philosophy (PhD) / Colonoscopy is a powerful tool for colon cancer screening. A colonoscopy can decrease the chance of developing advanced cancers by removing pre-cancerous polyps before they grow. This research works to improve colonoscopy’s localization using motion tracking and its coverage using surface mapping. We have developed an endoscope motion tracker that records the scope’s insertion length, rotation, and speed during a colonoscopy. It is In described in Chapter 4. The recorded motion can be combined with the endoscope’s video to improve colon cancer localization. Next, it is difficult to image the colon’s mucosa because the colonoscope’s camera has a limited field of view. Chapter 3 uses a 180° fisheye camera to unwrap high resolution panoramas of a colon phantom. The panoramas are then combined into a cylindrical surface map. Finally, Chapter 2 images the colon’s blood vessels to determine if they can fingerprint locations on the colon’s wall.