Chopin’s Etudes are difficult musical compositions for advanced piano students. Helmut Brauss, a professional pianist and educator, has created a number of videos to teach students motion patterns that will help them perfect the Etudes. The subtleties of motion shown in the videos are not apparently obvious to students, and in our research, we have developed four markerless based approaches to visually augment the videos: Predictive Optical Flow, Historical Optical Flow, Predictive Hand Tracking and Historical Hand Tracking. A survey of students learning the Etudes was conducted, and it was determined that the participants found the Historical techniques to be the most useful. No difference could be found between the usefulness of the Optical Flow and Hand Tracking augmentations. / Graduate

Some insects have the capability to detect and track small moving objects, often against cluttered moving backgrounds. Determining how this task is performed is an intriguing challenge, both from a physiological and computational perspective. Previous research has characterized higher-order neurons within the fly brain known as 'small target motion detectors‘ (STMD) that respond selectively to targets, even within complex moving surrounds. Interestingly, these cells still respond robustly when the velocity of the target is matched to the velocity of the background (i.e. with no relative motion cues). We performed intracellular recordings from intermediate-order neurons in the fly visual system (the medulla). These full-wave rectifying, transient cells (RTC) reveal independent adaptation to luminance changes of opposite signs (suggesting separate 'on‘ and 'off‘ channels) and fast adaptive temporal mechanisms (as seen in some previously described cell types). We show, via electrophysiological experiments, that the RTC is temporally responsive to rapidly changing stimuli and is well suited to serving an important function in a proposed target-detecting pathway. To model this target discrimination, we use high dynamic range (HDR) natural images to represent 'real-world‘ luminance values that serve as inputs to a biomimetic representation of photoreceptor processing. Adaptive spatiotemporal high-pass filtering (1st-order interneurons) shapes the transient 'edge-like‘ responses, useful for feature discrimination. Following this, a model for the RTC implements a nonlinear facilitation between the rapidly adapting, and independent polarity contrast channels, each with centre-surround antagonism. The recombination of the channels results in increased discrimination of small targets, of approximately the size of a single pixel, without the need for relative motion cues. This method of feature discrimination contrasts with traditional target and background motion-field computations. We show that our RTC-based target detection model is well matched to properties described for the higher-order STMD neurons, such as contrast sensitivity, height tuning and velocity tuning. The model output shows that the spatiotemporal profile of small targets is sufficiently rare within natural scene imagery to allow our highly nonlinear 'matched filter‘ to successfully detect many targets from the background. The model produces robust target discrimination across a biologically plausible range of target sizes and a range of velocities. We show that the model for small target motion detection is highly correlated to the velocity of the stimulus but not other background statistics, such as local brightness or local contrast, which normally influence target detection tasks. From an engineering perspective, we examine model elaborations for improved target discrimination via inhibitory interactions from correlation-type motion detectors, using a form of antagonism between our feature correlator and the more typical motion correlator. We also observe that a changing optimal threshold is highly correlated to the value of observer ego-motion. We present an elaborated target detection model that allows for implementation of a static optimal threshold, by scaling the target discrimination mechanism with a model-derived velocity estimation of ego-motion. Finally, we investigate the physiological relevance of this target discrimination model. We show that via very subtle image manipulation of the visual stimulus, our model accurately predicts dramatic changes in observed electrophysiological responses from STMD neurons. / http://proxy.library.adelaide.edu.au/login?url= http://library.adelaide.edu.au/cgi-bin/Pwebrecon.cgi?BBID=1368818 / Thesis (Ph.D.) - University of Adelaide, School of Molecular and Biomedical Science, 2009

In many surveillance scenarios, one concern that arises is how to construct an imager that is capable of capturing the scene with high fidelity. This could be problematic for two reasons: first, the optics and electronics in the camera may have difficulty in dealing with so much information; secondly, bandwidth constraints, may pose difficulty in transmitting information from the imager to the user efficiently for reconstruction or realization. In this thesis, we will discuss a mathematical framework that is capable of skirting the two aforementioned issues. This framework is rooted in a technique commonly referred to as compressive sensing. We will explore two of the seminal works in compressive sensing and will present the key theorems and definitions from these two papers. We will then survey three different surveillance scenarios and their respective compressive sensing solutions. The original contribution of this thesis is the development of a distributed compressive sensing model.

This thesis attempts to develop features identification and tracking system for an autonomous ground vehicle by focusing on four fundamental tasks: Motion detection, object tracking, scene recognition, and object detection and recognition. For motion detection, we combined the background subtraction method using the mixture of Gaussian models and the optical flow to highlight any moving objects or new entering objects which stayed still. To increase robustness for object tracking result, we used the Kalman filter to combine the tracking method based on the color histogram and the method based on invariant features. For scene recognition, we applied the algorithm Census Transform Histogram (CENTRIST), which is based on Census Transform images of the training data and the Support Vector Machine classifier, to recognize a total of 8 scene categories. Because detecting the horizon is also an important task for many navigation applications, we also performed horizon detection in this thesis. Finally, the deformable parts-based models algorithm was implemented to detect some common objects, such as humans and vehicles. Furthermore, objects were only detected in the area under the horizon to reduce the detecting time and false matching rate. / Master of Science

Birds are a key environmental asset and this is recognised through comprehensive legislation and policy ensuring their protection and conservation. Many species are active at night and surveys are required to understand the implications of proposed developments such as towers and reduce possible conflicts with these structures. Night vision devices are commonly used in nocturnal surveys, either to scope an area for bird numbers and activity, or in remotely sensing an area to determine potential risk. This thesis explores some practical and theoretical approaches that can improve the accuracy, confidence and efficiency of nocturnal bird surveillance. As image intensifiers and thermal imagers have operational differences, each device has associated strengths and limitations. Empirical work established that image intensifiers are best used for species identification of birds against the ground or vegetation. Thermal imagers perform best in detection tasks and monitoring bird airspace usage. The typically used approach of viewing bird survey video from remote sensing in its entirety is a slow, inaccurate and inefficient approach. Accuracy can be significantly improved by viewing the survey video at half the playback speed. Motion detection efficiency and accuracy can be greatly improved through the use of adaptive background subtraction and cumulative image differencing. An experienced ornithologist uses bird flight style and wing oscillations to identify bird species. Changes in wing oscillations can be represented in a single inter-frame similarity matrix through area-based differencing. Bird species classification can then be automated using singular value decomposition to reduce the matrices to one-dimensional vectors for training a feed-forward neural network.

It is proven that locations of internal body joints are sufficient visual cues to characterize human motion. In this dissertation I propose that locations of human extremities including heads, hands and feet provide powerful approximation to internal body motion. I propose detection of precise extremities from contours obtained from image segmentation or contour tracking. Junctions of medial axis of contours are selected as stars. Contour points with a local maximum distance to various stars are chosen as candidate extremities. All the candidates are filtered by cues including proximity to other candidates, visibility to stars and robustness to noise smoothing parameters. I present my applications of using precise extremities for fast human action detection and recognition. Environment specific features are built from precise extremities and feed into a block based Hidden Markov Model to decode the fence climbing action from continuous videos. Precise extremities are grouped into stable contacts if the same extremity does not move for a certain duration. Such stable contacts are utilized to decompose a long continuous video into shorter pieces. Each piece is associated with certain motion features to form primitive motion units. In this way the sequence is abstracted into more meaningful segments and a searching strategy is used to detect the fence climbing action. Moreover, I propose the histogram of extremities as a general posture descriptor. It is tested in a Hidden Markov Model based framework for action recognition. I further propose detection of probable extremities from raw images without any segmentation. Modeling the extremity as an image patch instead of a single point on the contour helps overcome the segmentation difficulty and increase the detection robustness. I represent the extremity patches with Histograms of Oriented Gradients. The detection is achieved by window based image scanning. In order to reduce computation load, I adopt the integral histograms technique without sacrificing accuracy. The result is a probability map where each pixel denotes probability of the patch forming the specific class of extremities. With a probable extremity map, I propose the histogram of probable extremities as another general posture descriptor. It is tested on several data sets and the results are compared with that of precise extremities to show the superiority of probable extremities. / text

Streaming video applications on the Internet generally have very high bandwidth requirements and yet are often unresponsive to network congestion. In order to avoid congestion collapse and improve video quality, these applications need to respond to congestion in the network by deploying mechanisms to reduce their bandwidth requirements under conditions of heavy load. In reducing bandwidth, video with high motion will look better if all the frames are kept but the frames have low quality, while video with low motion will look better if some frames are dropped but the remaining frames have high quality. Unfortunately current video applications scale to fit the available bandwidth without regard to the video content. In this thesis, we present an adaptive content-aware scaling mechanism that reduces the bandwidth occupied by an application by either dropping frames (temporal scaling) or by reducing the quality of the frames transmitted (quality scaling). We have designed a streaming video client and server with the server capable of quantifying the amount of motion in an MPEG stream and scaling each scene either temporally or by quality as appropriate, maximizing the appearance of each video stream. We have evaluated the impact of content-aware scaling by conducting a user study wherein the subjects rated the quality of video clips that were first scaled temporally and then by quality in order to establish the optimal mechanism for scaling a particular stream. We find that content-aware scaling can improve video quality by as much as 50%. We have also evaluated the practical impact of adaptively scaling the video stream by conducting a user study for longer video clips with varying amounts of motion and available bandwidth. We find that for such clips also the improvement in perceptual quality on account of adaptive content-aware scaling is as high as 30%

Algoritmos de detecção de movimento em seqüências de imagens devem satisfazer os requisitos de precisão, robustez e velocidade de processamento. A forma de combinar estes três itens depende do desenvolvimento do algoritmo e da aplicação a que se destina, sem que os itens de robustez e precisão sejam comprometidos. Neste trabalho investigamos técnicas para detecção do movimento que satisfazem tais requisitos. A técnica escolhida para detecção de movimento foi a do fluxo Ótico (FO) devido as suas características de precisão nos resultados. Como esta técnica exige elevado esforço computacional, propõe-se nesta tese uma metodologia que aplica as equações de fluxo ótico em reduzidas áreas da imagem processada. Estas áreas são selecionadas utilizando algumas técnicas de pré-processamento que identificam regiões da imagem com maior probabilidade de movimento presente. Posteriormente a esta identificação são aplicadas as equações de FO nas regiões de interesse. Para avaliação e validação do método proposto, comparam-se os diagramas de agulhas resultantes das áreas reduzidas aos diagramas resultantes de toda a imagem mediante critérios estatísticos e de tempo de processamento envolvido. Os algoritmos são testados utilizando imagens sintéticas e imagens reais. / Algorithms for motion detection in image sequences must satisfy the following requirements: accuracy, robustness and speed. The way that accuracy, robustness and speed are combined depends on the algorithm development and on the application. In this work, it has investigated motion detection techniques that satisfy the mentioned requirements. The Optical Flow technique was chosen for the motion detection due to its good performance in the results. As the Optical Flow requires intensive computational load, we propose in this thesis a methodology where Optical Flow Equations are applied in specific areas of the processed image. These areas were selected using pre-processing techniques that identify regions of image with larger motion probability. After the motion areas identification, Optical Flow Equations are applied to the regions of interest. To assess and validate the proposed method, the needle diagrams obtained in the reduced areas are compared with the ones obtained from the whole image according to statistical criteria and processing time. The proposed algorithms have been tested in synthetic and real images.

FundaÃÃo Cearense de Apoio ao Desenvolvimento Cientifico e TecnolÃgico / CoordenaÃÃo de AperfeiÃoamento de Pessoal de NÃvel Superior / A diminuiÃÃo dos custos de hardware associado ao aumento da capacidade de processamento dos dispositivos portÃteis tem causado um crescimento muito acelerado do mercado consumidor no Brasil e no mundo, de maneira que estes dispositivos tornaram-se uma parte indispensÃvel no nosso cotidiano. Contudo, o crescente fator de miniaturizaÃÃo de componentes gerou problemas em relaÃÃo à interaÃÃo eficiente com perifÃricos de entrada e saÃda tradicionais, como os cursores e o teclado. Assim, para evitar custos adicionais, uma alternativa à utilizar componentes jà existentes no dispositivo, como sua cÃmera integrada. Neste contexto, o objetivo geral desta tese à o desenvolvimento de um novo algoritmo para a interaÃÃo humano-dispositivo portÃtil multiplataforma baseado em fluxo Ãptico. Embora o fluxo Ãptico de Horn (1981) ser conhecido por ser um algoritmo de extrema complexidade computacional, nesta tese à proposto um conjunto de adaptaÃÃes neste algoritmo para tornar possÃvel seu processamento em tempo real em dispositivos portÃteis. O algoritmo proposto à desenvolvido em C ANSI e embarcado em dispositivos com sistemas operacionais Qualcomm REX, Symbian e Android. O algoritmo implementado à comparado com mÃtodos presentes na literatura especializada para detecÃÃo de movimentos atravÃs de simulaÃÃes computacionais e testes de usabilidade. Os resultados mostram que o algoritmo proposto possui um baixo esforÃo computacional associado a uma efetiva detecÃÃo de movimentos, mostrando que à possÃvel o uso do mÃtodo de Fluxo Ãptico como sensor para interaÃÃo em sistemas embarcados. AlÃm disto, atravÃs da metodologia de implementaÃÃo proposta nesta tese, à possÃvel utilizar as funcionalidades desenvolvidas em diversos tipos de sistemas operacionais. / The decrease in hardware costs associated with improvement of processing power of embedded devices has caused a rapid growth of the consumer market, which make these devices an indispensable part of our daily life. However, the miniaturization of these components leads to problems in the usability of mobile devices, especially with traditional input interfaces, such as the cursors and keyboard. One solution to avoid this kind of problem without increase production cost of these devices is use resources available, like the embedded camera. Following this idea, the objective of this thesis is the development of a new multi-platform human-handheld interaction algorithm based in optical flow. Although the traditional optical flow algorithms have high computational effort, in this thesis, adaptations and optimizations of this algorithm are proposed to overcome hardware limitations of embedded systems. The proposed algorithm is implemented in ANSI C and is embedded at devices with Android, Rex and Symbian OS. The implemented algorithm is compared with traditional motion detection algorithm through computational simulations and usability tests. The results shown that the proposed algorithm associates a low computational effort associated with effective motion detection. So it is possible to use the optical flow as sensor to interact with handheld devices. Furthermore, through the implementation methodology of this thesis, is possible use the developed functionalities in various operational systems.

Successful and high precision localization is an important feature for autonomous vehicles in an urban environment. GPS solutions are not good on their own and laser, sonar and radar are often used as complementary sensors. Localization with these sensors requires the use of techniques grouped under the acronym SLAM (Simultaneous Localization And Mapping). These techniques work by comparing the current sensor inputs to either an incrementally built or known map, also adding the information to the map.Most of the SLAM techniques assume the environment to be static, which means that dynamics and clutter in the environment might cause SLAM to fail. To ob-tain a more robust algorithm, the dynamics need to be dealt with. This study seeks a solution where measurements from different points in time can be used in pairwise comparisons to detect non-static content in the mapped area. Parked cars could for example be detected at a parking lot by using measurements from several different days.The method successfully detects most non-static objects in the different test datasets from the sensor. The algorithm can be used in conjunction with Pose-SLAM to get a better localization estimate and a map for later use. This map is good for localization with SLAM or other techniques since only static objects are left in it.