Um dos grandes desafios em mineração de dados é a integração de dados temporais ao seu processo. Existe um grande número de aplicações emergentes que envolvem dados temporais, incluindo a identificação de transações fraudulentas em cartões de crédito e ligações telefônicas, a detecção de intrusão em sistemas computacionais, a predição de estruturas secundárias de proteínas, a análise de dados provenientes de sensores, entre muitas outras. Neste trabalho, tem-se interesse na classificação de séries temporais que representam sinais de áudio. Como aplicação principal, tem-se interesse em classificar sinais de insetos coletados por um sensor óptico, que deve ser capaz de contar e classificar os insetos de maneira automática. Apesar de serem coletados opticamente, os sinais capturados se assemelham a sinais de áudio. O objetivo desta pesquisa é comparar métodos de classificação por similaridade e por extração de atributos que possam ser utilizados no contexto da classificação de insetos. Para isso, foram empregados os principais métodos de classificação de sinais de áudio, que têm sido propostos para problemas como reconhecimento de instrumentos musicais, fala e espécies animais. Neste trabalho, é mostrado que, de modo geral, a abordagem por extração de atributos é mais eficaz do que a classificação por similaridade. Mais especificamente, os melhores resultados são obtidos com a utilização de coeficientes mel-cepstrais. Este trabalho apresenta contribuições significativas em outras aplicações, também relacionadas à análise de séries temporais e sinais de áudio, por similaridade e por extração de atributos / One of the major challenges in data mining is the integration of temporal data to its process. There are a number of emerging applications that involve temporal data, including fraud detection in credit card transactions and phone calls, intrusion detection in computer systems, the prediction of secondary structures of proteins, the analysis of data from sensors, and many others. In this work, our main interest is the classification of time series that represent audio signals. Our main interest is an application for classifying signals of insects collected from an optical sensor, which should count and classify insects automatically. Although these signals are optically collected, they resemble audio signals. The objective of this research is to compare classification methods based on similarity and feature extraction in the context of insects classification. For this purpose, we used the main classification methods for audio signals, which have been proposed for problems such as musical instrument, speech and animal species recognition. This work shows that, in general, the approach based on feature extraction is more accurate than the classification by similarity. More specifically, the best results are obtained with mel-frequency cepstrum coefficients. This work also presents significant contributions in other applications, also related to the analysis of time series and audio signals by similarity and feature extraction

Um dos grandes desafios em mineração de dados é a integração de dados temporais ao seu processo. Existe um grande número de aplicações emergentes que envolvem dados temporais, incluindo a identificação de transações fraudulentas em cartões de crédito e ligações telefônicas, a detecção de intrusão em sistemas computacionais, a predição de estruturas secundárias de proteínas, a análise de dados provenientes de sensores, entre muitas outras. Neste trabalho, tem-se interesse na classificação de séries temporais que representam sinais de áudio. Como aplicação principal, tem-se interesse em classificar sinais de insetos coletados por um sensor óptico, que deve ser capaz de contar e classificar os insetos de maneira automática. Apesar de serem coletados opticamente, os sinais capturados se assemelham a sinais de áudio. O objetivo desta pesquisa é comparar métodos de classificação por similaridade e por extração de atributos que possam ser utilizados no contexto da classificação de insetos. Para isso, foram empregados os principais métodos de classificação de sinais de áudio, que têm sido propostos para problemas como reconhecimento de instrumentos musicais, fala e espécies animais. Neste trabalho, é mostrado que, de modo geral, a abordagem por extração de atributos é mais eficaz do que a classificação por similaridade. Mais especificamente, os melhores resultados são obtidos com a utilização de coeficientes mel-cepstrais. Este trabalho apresenta contribuições significativas em outras aplicações, também relacionadas à análise de séries temporais e sinais de áudio, por similaridade e por extração de atributos / One of the major challenges in data mining is the integration of temporal data to its process. There are a number of emerging applications that involve temporal data, including fraud detection in credit card transactions and phone calls, intrusion detection in computer systems, the prediction of secondary structures of proteins, the analysis of data from sensors, and many others. In this work, our main interest is the classification of time series that represent audio signals. Our main interest is an application for classifying signals of insects collected from an optical sensor, which should count and classify insects automatically. Although these signals are optically collected, they resemble audio signals. The objective of this research is to compare classification methods based on similarity and feature extraction in the context of insects classification. For this purpose, we used the main classification methods for audio signals, which have been proposed for problems such as musical instrument, speech and animal species recognition. This work shows that, in general, the approach based on feature extraction is more accurate than the classification by similarity. More specifically, the best results are obtained with mel-frequency cepstrum coefficients. This work also presents significant contributions in other applications, also related to the analysis of time series and audio signals by similarity and feature extraction

It has been noticed that one of the main challenges for e-commerce sites is providing the users with an efficient search function. It has also been noticed that the search function is one of the things the user is valuing the most when evaluating an e-commerce. The information technology enables the possibility to consume almost anything one could wish for. The challenge is in how to find this specific item. It is hence of interest to examine how to improve the search tool and what effect the updated search tool results in. The objective of this research is twofold. The objective motivated by economic factors is to examine how a user’s ability to find relevant items is affected by being able to refine a search result by selecting relevant attribute values. The other objective has a more technical character and is to examine how the rule based method performs in terms of extracting attribute values for enable the creation of the filtering function. The examinations for this research is conducted at a Swedish online auction company that due to the structure of its e-catalogue provides a suitable setup for the examinations. Because of the examinations being conducted in the environment of the auction company’s system this limits the result to only being representative for data with the same characteristics as the auction company’s. For answering the questions stated in the objective two methods are applied. One for examining the economic part and one for examining the technical part. The economic question is answered after analysing the result of an A/B-test conducted at the auction company. For answering the technical examination an information extraction technique is evaluated. The result of the economical examination is that a significant increase in conversion rate can be concluded for the system version with filtering enabled. The result of the technical examination shows that the rule based method can be used for information extraction in some cases. However, the obtained accuracy will be affected by the characteristics of the data the information extraction is performed on.

<p>The availability of abundant multimodal data, including textual, visual, and sensor-based information, holds the potential to improve decision-making in diverse domains. Extracting data-driven decision-making information from heterogeneous and changing datasets in real-world data-centric applications requires achieving complementary functionalities of multimodal data integration, knowledge extraction and mining, situationally-aware data recommendation to different users, and uncertainty management in the open-world setting. To achieve a system that encompasses all of these functionalities, several challenges need to be effectively addressed: (1) How to represent and analyze heterogeneous source contents and application context for multimodal data recommendation? (2) How to predict and fulfill current and future needs as new information streams in without user intervention? (3) How to integrate disconnected data sources and learn relevant information to specific mission needs? (4) How to scale from processing petabytes of data to exabytes? (5) How to deal with uncertainties in open-world that stem from changes in data sources and user requirements?</p> <p><br></p> <p>This dissertation tackles these challenges by proposing novel frameworks, learning-based data integration and retrieval models, and algorithms to empower decision-makers to extract valuable insights from diverse multimodal data sources. The contributions of this dissertation can be summarized as follows: (1) We developed SKOD, a novel multimodal knowledge querying framework that overcomes the data representation, scalability, and data completeness issues while utilizing streaming brokers and RDBMS capabilities with entity-centric semantic features as an effective representation of content and context. Additionally, as part of the framework, a novel text attribute recognition model called HART was developed, which leveraged language models and syntactic properties of large unstructured texts. (2) In the SKOD framework, we incrementally proposed three different approaches for data integration of the disconnected sources from their semantic features to build a common knowledge base with the user information need: (i) EARS: A mediator approach using schema mapping of the semantic features and SQL joins was proposed to address scalability challenges in data integration; (ii) FemmIR: A data integration approach for more susceptible and flexible applications, that utilizes neural network-based graph matching techniques to learn coordinated graph representations of the data. It introduces a novel graph creation approach from the features and a novel similarity metric among data sources; (iii) WeSJem: This approach allows zero-shot similarity matching and data discovery by using contrastive learning<br> to embed data samples and query examples in a high-dimensional space using features as a novel source of supervision instead of relevance labels. (3) Finally, to manage uncertainties in multimodal data management for open-world environments, we characterized novelties in multimodal information retrieval based on data drift. Moreover, we proposed a novelty detection and adaptation technique as an augmentation to WeSJem.<br> </p> <p>The effectiveness of the proposed frameworks, models, and algorithms was demonstrated<br> through real-world system prototypes that solved open problems requiring large-scale human<br> endeavors and computational resources. Specifically, these prototypes assisted law enforcement officers in automating investigations and finding missing persons.<br> </p>