The importance of digital information collections is growing. Collections are typically represented with text-only, in a linear list format, which turns out to be a weak representation for cognition. We learned this from empirical research in cognitive psychology, and by conducting a study to develop an understanding of current practices and resulting breakdowns in human experiences of building and utilizing collections. Because of limited human attention and memory, participants had trouble finding specific elements in their collections, resulting in low levels of collection utilization. To address these issues, this research develops new collection representations for visual cognition. First, we present the image+text surrogate, a concise representation for a document, or portion thereof, which is easy to understand and think about. An information extraction algorithm is developed to automatically transform a document into a small set of image+text surrogates. After refinement, the average accuracy performance of the algorithm was 90%. Then, we introduce the composition space to represent collections, which helps people connect elements visually in a spatial format. To ensure diverse information from multiple sources to be presented evenly in the composition space, we developed a new control structure, the ResultDis- tributor. A user study has demonstrated that the participants were able to browse more diverse information using the ResultDistributor-enhanced composition space. Participants also found it easier and more entertaining to browse information in this representation. This research is applicable to represent the information resources in contexts such as search engines or digital libraries. The better representation will enhance the cognitive efficacy and enjoyment of people’s everyday tasks of information searching, browsing, collecting, and discovering.

No description available.

Music analysis and source separation have become important and allied areas of research over the last decade. Towards this, analyzing a music signal for important events such as onsets, offsets and transients are important problems. These tasks help in music source separation and transcription. Approaches in source separation too have been making great strides, but most of these techniques are aimed at Western music and fail to perform well for Indian music. The fluid style of instrumentation in Indian music requires a slightly modified approach to analysis and source separation. We propose an onset detection algorithm that is motivated by the human auditory system. This algorithm has the advantage of having a unified framework for the detection of both onsets and offsets in music signals. This onset detection algorithm is further extended to detect percussive transients. Percussive transients have sharp onsets followed closely by sharp offsets. This characteristic is exploited in the percussive transients detection algorithm. This detection does not lend itself well to the extraction of transients and hence we propose an iterative algorithm to extract all types of transients from a polyphonic music signal. The proposed iterative algorithm is both fast and accurate to extract transients of various strengths. This problem of transient extraction can be extended to the problem of harmonic/percussion sound separation(HPSS), where a music signal is separated into two streams consisting of components mainly from percussion and harmonic instruments. Many algorithms that have been proposed till date deal with HPSS for Western music. But with Indian classical/film music, a different style of instrumentation or singing is seen, including high degree of vibratos or glissando content. This requires new approaches to HPSS. We propose extensions to two existing HPSS techniques, adapting them for Indian music. In both the extensions, we retain the original framework of the algorithm, showing that it is easy to incorporate the changes needed to handle Indian music. We also propose a new HPSS algorithm that is inspired by our transient extraction technique. This algorithm can be considered a generalized extension to our transient extraction algorithm and showcases our view that HPSS can be considered as an extension to transient analysis. Even the best HPSS techniques have leakages of harmonic components into percussion and this can lead to poor performances in tasks like rhythm analysis. In order to reduce this leakage, we propose a post processing technique on the percussion stream of the HPSS algorithm. The proposed method utilizes signal stitching by exploiting a commonly used model for percussive envelopes. We also developed a vocals extraction algorithm from the harmonic stream of the HPSS algorithm. The vocals extraction follows the popular paradigm of extracting the predominant pitch followed by generation of the vocals signal corresponding to the pitch. We show that HPSS as a pre-processing technique gives an advantage in reducing the interference from percussive sources in the extraction stage. It is also shown that the performance of vocal extraction algorithms improve with the knowledge about locations of the vocal segments. This is shown with the help of an oracle to locate the vocal segments. The use of the oracle greatly reduces the interferences from other dominating sources in the extracted vocals signal.

<div>With more and more videos taken from dash cams on thousands of cars, retrieving these videos and searching for important information is a daunting task. The purpose of this work is to mine some key road and vehicle motion attributes in a large-scale driving video data set for traffic analysis, sensing algorithm development and autonomous driving test benchmarks. Current sensing and control of autonomous cars based on full-view identification makes it difficult to maintain a high-frequency with a fast-moving vehicle, since computation is increasingly used to cope with driving environment changes.</div><div><br></div><div>A big challenge in video data mining is how to deal with huge amounts of data. We use a compact representation called the road profile system to visualize the road environment in long 2D images. It reduces the data from each frame of image to one line, thereby compressing the video clip to the image. This data dimensionality reduction method has several advantages: First, the data size is greatly compressed. The data is compressed from a video to an image, and each frame in the video is compressed into a line. The data size is compressed hundreds of times. While the size and dimensionality of the data has been compressed greatly, the useful information in the driving video is still completely preserved, and motion information is even better represented more intuitively. Because of the data and dimensionality reduction, the identification algorithm computational efficiency is higher than the full-view identification method, and it makes the real-time identification on road is possible. Second, the data is easier to be visualized, because the data is reduced in dimensionality, and the three-dimensional video data is compressed into two-dimensional data, the reduction is more conducive to the visualization and mutual comparison of the data. Third, continuously changing attributes are easier to show and be captured. Due to the more convenient visualization of two-dimensional data, the position, color and size of the same object within a few frames will be easier to compare and capture. At the same time, in many cases, the trouble caused by tracking and matching can be eliminated. Based on the road profile system, there are three tasks in autonomous driving are achieved using the road profile images.</div><div><br></div><div>The first application is road edge detection under different weather and appearance for road following in autonomous driving to capture the road profile image and linearity profile image in the road profile system. This work uses naturalistic driving video data mining to study the appearance of roads, which covers large-scale road data and changes. This work excavated a large number of naturalistic driving video sets to sample the light-sensitive area for color feature distribution. The effective road contour image is extracted from the long-time driving video, thereby greatly reducing the amount of video data. Then, the weather and lighting type can be identified. For each weather and lighting condition obvious features are I identified at the edge of the road to distinguish the road edge. </div><div><br></div><div>The second application is detecting vehicle interactions in driving videos via motion profile images to capture the motion profile image in the road profile system. This work uses visual actions recorded in driving videos taken by a dashboard camera to identify this interaction. The motion profile images of the video are filtered at key locations, thereby reducing the complexity of object detection, depth sensing, target tracking and motion estimation. The purpose of this reduction is for decision making of vehicle actions such as lane changing, vehicle following, and cut-in handling.</div><div><br></div><div>The third application is motion planning based on vehicle interactions and driving video. Taking note of the fact that a car travels in a straight line, we simply identify a few sample lines in the view to constantly scan the road, vehicles, and environment, generating a portion of the entire video data. Without using redundant data processing, we performed semantic segmentation to streaming road profile images. We plan the vehicle's path/motion using the smallest data set possible that contains all necessary information for driving.</div><div><br></div><div>The results are obtained efficiently, and the accuracy is acceptable. The results can be used for driving video mining, traffic analysis, driver behavior understanding, etc.</div>