Detection of events and actions in video entails substantial processing of very large, even open-ended, video streams. Video data presents a unique challenge for the information retrieval community because properly representing video events is challenging. We propose a novel approach to analyze temporal aspects of video data. We consider video data as a sequence of images that form a 3-dimensional spatiotemporal structure, and perform multiview orthographic projection to transform the video data into 2-dimensional representations. The projected views allow a unique way to rep- resent video events and capture the temporal aspect of video data. We extract local salient points from 2D projection views and perform detection-via-similarity approach on a wide range of events against real-world surveillance data. We demonstrate our example-based detection framework is competitive and robust. We also investigate the synthetic example driven retrieval as a basis for query-by-example.

The use of social media is expanding significantly and can serve a variety of purposes. Over the last few years, users of social media have played an increasing role in the dissemination of emergency and disaster information. It is becoming more common for affected populations and other stakeholders to turn to Twitter to gather information about a crisis when decisions need to be made, and action is taken. However, social media platforms, especially on Twitter, presents some drawbacks when it comes to gathering information during disasters. These drawbacks include information overload, messages are written in an informal format, the presence of noise and irrelevant information. These factors make gathering accurate information online very challenging and confusing, which in turn may affect public, communities, and organizations to prepare for, respond to, and recover from disasters. To address these challenges, we present an integrated three parts (clustering-classification-ranking) framework, which helps users choose through the masses of Twitter data to find useful information. In the first part, we build standard machine learning models to automatically extract and identify topics present in a text and to derive hidden patterns exhibited by a dataset. Next part, we developed a binary and multi-class classification model of Twitter data to categorize each tweet as relevant or irrelevant and to further classify relevant tweets into four types of community engagement: reporting information, expressing negative engagement, expressing positive engagement, and asking for information. In the third part, we propose a binary classification model to categorize the collected tweets into high or low priority tweets. We present an evaluation of the effectiveness of detecting events using a variety of features derived from Twitter posts, namely: textual content, term frequency-inverse document frequency, Linguistic, sentiment, psychometric, temporal, and spatial. Our framework also provides insights for researchers and developers to build more robust socio-technical disasters for identifying types of online community engagement and ranking high-priority tweets in disaster situations.

The remarkable increase in the use of sensors in our daily lives has provided an increasing number of opportunities for decision-making and automation of events. Opportunities for decision-making have further risen with the advent of smart technology and the omnipresence of sensors. Various methods have been devised to detect different events in a driving environment using smart-phones as they provide two main advantages: they remove the need to have dedicated hardware in vehicles and they are widely accessible. Rewarding safe driving has always been an important issue for insurance companies. With this intention, they are now moving toward implementing plans that consider current driving usage (Usage-based-drive plans) in contrast with traditional history-based-plans. The detection of driving events is important in insurance telematics for this purpose. Events such as acceleration and turning are good examples of important information. The sensors are capable of detecting whether a car is accelerating or braking, while through fusing services we can detect other events like speeding or the occurrence of a severe weather phenomenon that can affect driving. This thesis aims to look at the telematics from a new angle that employs smart-phones as the sensing platform. We proposed a new hybrid classification algorithm that detects acceleration-based events with an F1-score of 0.9304 and turn events with an F1-score of 0.9038. We further performed a case study on measuring the performance of driving utilizing various measures. This index can be used by a wide range of benefactors such as the insurance and transportation industries.

Online social networks have become a staging ground for many modern movements, with the Arab Spring being the most prominent example. In an effort to understand and predict those movements, social media can be regarded as a valuable social sensor for disclosing underlying behaviors and patterns. To fully understand mass movement information propagation patterns in social networks, several problems need to be considered and addressed. Specifically, modeling mass movements that incorporate multiple spaces, a dynamic network structure, and misinformation propagation, can be exceptionally useful in understanding information propagation in social media. This dissertation explores four research problems underlying efforts to identify and track the adoption of mass movements in social media. First, how do mass movements become mobilized on Twitter, especially in a specific geographic area? Second, can we detect protest activity in social networks by observing group anomalies in graph? Third, how can we distinguish real movements from rumors or misinformation campaigns? and fourth, how can we infer the indicators of a specific type of protest, say climate related protest? A fundamental objective of this research has been to conduct a comprehensive study of how mass movement adoption functions in social networks. For example, it may cross multiple spaces, evolve with dynamic network structures, or consist of swift outbreaks or long term slowly evolving transmissions. In many cases, it may also be mixed with misinformation campaigns, either deliberate or in the form of rumors. Each of those issues requires the development of new mathematical models and algorithmic approaches such as those explored here. This work aims to facilitate advances in information propagation, group anomaly detection and misinformation distinction and, ultimately, help improve our understanding of mass movements and their adoption in social networks. / Ph. D.

We investigate sequential event detection problems arising in Wireless Sensor Networks (WSNs). A number of battery–powered sensor nodes of the same sensing modality are deployed in a region of interest(ROI). By an event we mean a random time(and, for spatial events, a random location) after which the random process being observed by the sensor field experiences a change in its probability law. The sensors make measurements at periodic time instants, perform some computations, and then communicate the results of their computations to the fusion centre. The decision making algorithm in the fusion centre employs a procedure that makes a decision on whether the event has occurred or not based on the information it has received until the current decision instant. We seek event detection algorithms in various scenarios, that are optimal in the sense that the mean detection delay (delay between the event occurrence time and the alarm time) is minimum under certain detection error constraints. In the first part of the thesis, we study event detection problems in a small extent network where the sensing coverage of any sensor includes the ROI. In particular, we are interested in the following problems: 1) quickest event detection with optimal control of the number of sensors that make observations(while the others sleep),2) quickest event detection on wireless ad hoc networks, and3) optimal transient change detection. In the second part of the thesis, we study the problem of quickest detection and isolation of an event in a large extent sensor network where the sensing coverage of any sensor is only a small portion of the ROI. One of the major applications envisioned for WSNs is detecting any abnormal activity or intrusions in the ROI. An intrusion is typically a rare event, and hence, much of the energy of sensors gets drained away in the pre–intrusion period. Hence, keeping all the sensors in the awake state is wasteful of resources and reduces the lifetime of the WSN. This motivates us to consider the problem of sleep–wake scheduling of sensors along with quickest event detection. We formulate the Bayesian quickest event detection problem with the objective of minimising the expected total cost due to i)the detection delay and ii) the usage of sensors, subject to the constraint that the probability of false alarm is upper bounded by .We obtain optimal event detection procedures, along with optimal closed loop and open loop control for the sleep–wake scheduling of sensors. In the classical change detection problem, at each sampling instant, a batch of samples(where is the number of sensors deployed in the ROI) is generated at the sensors and reaches the fusion centre instantaneously. However, in practice, the communication between the sensors and the fusion centre is facilitated by a wireless ad hoc network based on a random access mechanism such as in IEEE802.11 or IEEE802.15.4. Because of the medium access control(MAC)protocol of the wireless network employed, different samples of the same batch reach the fusion centre after random delays. The problem is to detect the occurrence of an event as early as possible subject to a false alarm constraint. In this more realistic situation, we consider a design in which the fusion centre comprises a sequencer followed by a decision maker. In earlier work from our research group, a Network Oblivious Decision Making (NODM) was considered. In NODM, the decision maker in the fusion centre is presented with complete batches of observations as if the network was not present and makes a decision only at instants at which these batches are presented. In this thesis, we consider the design in which the decision maker makes a decision at all time instants based on the samples of all the complete batches received thus far, and the samples, if any, that it has received from the next (partial) batch. We show that for optimal decision making the network–state is required by the decision maker. Hence, we call this setting Network Aware Decision Making (NADM). Also, we obtain a mean delay optimal NADM procedure, and show that it is a network–state dependent threshold rule on the a posteriori probability of change. In the classical change detection problem, the change is persistent, i.e., after the change–point, the state of nature remains in the in–change state for ever. However, in applications like intrusion detection, the event which causes the change disappears after a finite time, and the system goes to an out–of–change state. The distribution of observations in the out–of–change state is the same as that in the pre–change state. We call this short–lived change a transient change. We are interested in detecting whether a change has occurred, even after the change has disappeared at the time of detection. We model the transient change and formulate the problem of quickest transient change detection under the constraint that the probability of false alarm is bounded by . We also formulate a change detection problem which maximizes the probability of detection (i.e., probability of stopping in the in–change state) subject to the probability of false alarm being bounded by . We obtain optimal detection rules and show that they are threshold d rules on the a posteriori probability of pre–change, where the threshold depends on the a posteriori probabilities of pre–change, in–change, and out–of–change states. Finally, we consider the problem of detecting an event in a large extent WSN, where the event influences the observations of sensors only in the vicinity of where it occurs. Thus, in addition to the problem of event detection, we are faced with the problem of locating the event, also called the isolation problem. Since the distance of the sensor from the event affects the mean signal level that the sensor node senses, we consider a realistic signal propagation model in which the signal strength decays with distance. Thus, the post–change mean of the distribution of observations across sensors is different, and is unknown as the location of the event is unknown, making the problem highly challenging. Also, for a large extent WSN, a distributed solution is desirable. Thus, we are interested in obtaining distributed detection/isolation procedures which are detection delay optimal subject to false alarm and false isolation constraints. For this problem, we propose the following local decision rules, MAX, HALL, and ALL, which are based on the CUSUM statistic, at each of the sensor nodes. We identify corroborating sets of sensor nodes for event location, and propose a global rule for detection/isolation based on the local decisions of sensors in the corroborating sets. Also, we show the minimax detection delay optimality of the procedures HALL and ALL.

An audio event detection and classification framework (AQUA) is developed for the North Pacific underwater acoustic research community. AQUA has been developed, tested, and verified on Ocean Networks Canada (ONC) hydrophone data. Ocean Networks Canada is an non-governmental organization collecting underwater passive acoustic data. AQUA enables the processing of a large acoustic database that grows at a rate of 5 GB per day. Novel algorithms to overcome challenges such as activity detection in broadband non-Gaussian type noise have achieved accurate and high classification rates. The main AQUA modules are blind activity detector, denoiser and classifier. The AQUA algorithms yield promising classification results with accurate time stamps. / Graduate

Oral feeding competency is a primary requirement for preterm infant hospital release. Currently there is no widely accepted method to objectively measure oral feeding. Feeding consists primarily of the integration of three individual feeding events: sucking, breathing, and swallowing, and the objective of feeding coordination is to minimize aspiration. The purpose of this work was to quantify the infant feeding process from signals obtained during bottle feeding and ultimately develop a measure of feeding coordination. Sucking was measured using a pressure transducer embedded within a modified silicone bottle block. Breathing was measured using a thermistor embedded within nasal cannula, and swallowing was measured through the use of several different piezoelectric sensors. In addition to feeding signals, electrocardiogram (ECG) signals were obtained as an indicator of overall infant behavioral state during feeding. Event detection algorithms for the individual feeding signals were developed and validated, then used for the development of a measurement of feeding coordination. The final suck event detection algorithm was the result of an iterative process that depended on the validity of the signal model. As the model adapted to better represent the data, the accuracy and specificity of the algorithm improved. For the breath signal, however, the primary barrier to effective event detection was significant baseline drift. The frequency components of the baseline drift overlapped significantly with the breath event frequency components, so a time domain solution was developed. Several methods were tested, and it was found that the acceleration vector of the signal provided the most robust representation of the underlying breath signal while minimizing baseline drift. Swallow signal event detection was not possible due to a lack of available data resulting from problems with the consistency of the obtained signal. A robust method was developed for the batch processing of heart rate variability analysis. Finally a method of coordination analysis was developed based on the event detection algorithm outputs. Coordination was measured by determining the percentage of feeding time that consisted of overlapping suck and breath activity.

In this thesis, we consider the analysis of music and environmental audio recordings with neural networks. Recently, neural networks have been shown to be an effective family of models for speech recognition, computer vision, natural language processing and a number of other statistical modelling problems. The composite layer-wise structure of neural networks allows for flexible model design, where prior knowledge about the domain of application can be used to inform the design and architecture of the neural network models. Additionally, it has been shown that when trained on sufficient quantities of data, neural networks can be directly applied to low-level features to learn mappings to high level concepts like phonemes in speech and object classes in computer vision. In this thesis we investigate whether neural network models can be usefully applied to processing music and environmental audio. With regards to music signal analysis, we investigate 2 different problems. The fi rst problem, automatic music transcription, aims to identify the score or the sequence of musical notes that comprise an audio recording. We also consider the problem of automatic chord transcription, where the aim is to identify the sequence of chords in a given audio recording. For both problems, we design neural network acoustic models which are applied to low-level time-frequency features in order to detect the presence of notes or chords. Our results demonstrate that the neural network acoustic models perform similarly to state-of-the-art acoustic models, without the need for any feature engineering. The networks are able to learn complex transformations from time-frequency features to the desired outputs, given sufficient amounts of training data. Additionally, we use recurrent neural networks to model the temporal structure of sequences of notes or chords, similar to language modelling in speech. Our results demonstrate that the combination of the acoustic and language model predictions yields improved performance over the acoustic models alone. We also observe that convolutional neural networks yield better performance compared to other neural network architectures for acoustic modelling. For the analysis of environmental audio recordings, we consider the problem of acoustic event detection. Acoustic event detection has a similar structure to automatic music and chord transcription, where the system is required to output the correct sequence of semantic labels along with onset and offset times. We compare the performance of neural network architectures against Gaussian mixture models and support vector machines. In order to account for the fact that such systems are typically deployed on embedded devices, we compare performance as a function of the computational cost of each model. We evaluate the models on 2 large datasets of real-world recordings of baby cries and smoke alarms. Our results demonstrate that the neural networks clearly outperform the other models and they are able to do so without incurring a heavy computation cost.

(non disponible, voir en anglais)

Recent advances in computer and network technologies have contributed significantly to global connectivity and stimulated the amount of online textual document to grow extremely rapidly. The rapid accumulation of textual documents on the Web or within an organization requires effective document management techniques, covering from information retrieval, information filtering and text mining. The word mismatch problem represents a challenging issue to be addressed by the document management research. Word mismatch has been extensively investigated in information retrieval (IR) research by the use of term expansion (or specifically query expansion). However, a review of text mining literature suggests that the word mismatch problem has seldom been addressed by text mining techniques. Thus, this thesis aims at investigating the use of term expansion on some text mining techniques, specifically including text categorization, document clustering and event detection. Accordingly, we developed term expansion extensions to these three text mining techniques. The empirical evaluation results showed that term expansion increased the categorization effectiveness when the correlation coefficient feature selection was employed. With respect to document clustering, techniques extended with term expansion achieved comparable clustering effectiveness to existing techniques and showed its superiority in improving clustering specificity measure. Finally, the use of term expansion for supporting event detection has degraded the detection effectiveness as compared to the traditional event detection technique.