Among the various natural calamities, flood is considered one of the most catastrophic natural hazards, which has a significant impact on the socio-economic lifeline of a country. The Assessment of flood risks facilitates taking appropriate measures to reduce the consequences of flooding. The flood risk assessment requires Big data which are coming from different sources, such as sensors, social media, and organizations. However, these data sources contain various types of uncertainties because of the presence of incomplete and inaccurate information. This paper presents a Belief rule-based expert system (BRBES) which is developed in Big data platform to assess flood risk in real time. The system processes extremely large dataset by integrating BRBES with Apache Spark while a web-based interface has developed allowing the visualization of flood risk in real time. Since the integrated BRBES employs knowledge driven learning mechanism, it has been compared with other data-driven learning mechanisms to determine the reliability in assessing flood risk. Integrated BRBES produces reliable results comparing from the other data-driven approaches. Data for the expert system has been collected targeting different case study areas from Bangladesh to validate the integrated system.

Ces dernières années, la biologie a subi une profonde mutation, impulsée notamment par les technologies à haut débit et la montée de la génomique personnalisée. L’augmentation massive et constante de l’information biologique qui en résulte offre de nouvelles opportunités pour comprendre la fonction et l’évolution des gènes et génomes à différentes échelles et leurs rôles dans les maladies humaines. Ma thèse s’est articulée autour de la relation entre chercheurs et information biologique, et j’ai contribué à (OrthoInspector) ou créé (Parsec, MyGeneFriends) des systèmes permettant aux chercheurs d’accéder, analyser, visualiser, filtrer et annoter en temps réel l’énorme quantité de données disponibles à l’ère post génomique. MyGeneFriends est un premier pas dans une direction passionnante, faire en sorte que ce ne soient plus les chercheurs qui aillent vers l’information, mais que l’information pertinente aille vers les chercheurs sous une forme adaptée, permettant l’accès personnalisé et efficace aux grandes quantités d’informations, la visualisation deces informations et leur interconnexion en réseaux. / In recent years, biology has undergone a profound evolution, mainly due to high through put technologies and the rise of personal genomics. The resulting constant and massive increase of biological data offers unprecedented opportunities to decipher the function and evolution of genes and genomes at different scales and their roles in human diseases. My thesis addressed the relationship between researchers and biological information, and I contributed to (OrthoInspector) or created (Parsec, MyGeneFriends) systems allowing researchers to access, analyze, visualize, filter and annotate in real time the enormous quantity of data available in the post genomic era. MyGeneFriends is a first step in an exciting new direction: where researchers no longer search forinformation, but instead pertinent information is brought to researchers in a suitable form, allowing personalized and efficient access to large amounts of information, visualization of this information,and their integration in networks.

The study of networks has seen a tremendous breed of researches due to the explosive spectrum of practical problems that involve networks as the access point. Those problems widely range from detecting functionally correlated proteins in biology to finding people to give discounts and gain maximum popularity of a product in economics. Thus, understanding and further being able to manipulate/control the development and evolution of the networks become critical tasks for network scientists. Despite the vast research effort putting towards these studies, the present state-of-the-arts largely either lack of high quality solutions or require excessive amount of time in real-world `Big Data' requirement. This research aims at affirmatively boosting the modern algorithmic efficiency to approach practical requirements. That is developing a ground-breaking class of algorithms that provide simultaneously both provably good solution qualities and low time and space complexities. Specifically, I target the important yet challenging problems in the three main areas: Information Diffusion: Analyzing and maximizing the influence in networks and extending results for different variations of the problems. Community Detection: Finding communities from multiple sources of information. Security and Privacy: Assessing organization vulnerability under targeted-cyber attacks via social networks.

Stochastic and data-distributed optimization algorithms have received lots of attention from the machine learning community due to the tremendous demand from the large-scale learning and the big-data related optimization. A lot of stochastic and deterministic learning algorithms are proposed recently under various application scenarios. Nevertheless, many of these algorithms are based on heuristics and their optimality in terms of the generalization error is not sufficiently justified. In this talk, I will explain the concept of an optimal learning algorithm, and show that given a time budget and proper hypothesis space, only those achieving the lower bounds of the estimation error and the optimization error are optimal. Guided by this concept, we investigated the stochastic minimization of nonsmooth convex loss functions, a central problem in machine learning. We proposed a novel algorithm named Accelerated Nonsmooth Stochastic Gradient Descent, which exploits the structure of common nonsmooth loss functions to achieve optimal convergence rates for a class of problems including SVMs. It is the first stochastic algorithm that can achieve the optimal O(1/t) rate for minimizing nonsmooth loss functions. The fast rates are confirmed by empirical comparisons with state-of-the-art algorithms including the averaged SGD. The Alternating Direction Method of Multipliers (ADMM) is another flexible method to explore function structures. In the second part we proposed stochastic ADMM that can be applied to a general class of convex and nonsmooth functions, beyond the smooth and separable least squares loss used in lasso. We also demonstrate the rates of convergence for our algorithm under various structural assumptions of the stochastic function: O(1/sqrt{t}) for convex functions and O(log t/t) for strongly convex functions. A novel application named Graph-Guided SVM is proposed to demonstrate the usefulness of our algorithm. We also extend the scalability of stochastic algorithms to nonlinear kernel machines, where the problem is formulated as a constrained dual quadratic optimization. The simplex constraint can be handled by the classic Frank-Wolfe method. The proposed stochastic Frank-Wolfe methods achieve comparable or even better accuracies than state-of-the-art batch and online kernel SVM solvers, and are significantly faster. The last part investigates the problem of data-distributed learning. We formulate it as a consensus-constrained optimization problem and solve it with ADMM. It turns out that the underlying communication topology is a key factor in achieving a balance between a fast learning rate and computation resource consumption. We analyze the linear convergence behavior of consensus ADMM so as to characterize the interplay between the communication topology and the penalty parameters used in ADMM. We observe that given optimal parameters, the complete bipartite and the master-slave graphs exhibit the fastest convergence, followed by bi-regular graphs.

No description available.

Submitted by Cleide Dantas (cleidedantas@ufpa.br) on 2014-07-31T13:38:32Z No. of bitstreams: 2 license_rdf: 23898 bytes, checksum: e363e809996cf46ada20da1accfcd9c7 (MD5) Dissertacao_OtimizacaoProcessoAprendizagem.pdf: 1776244 bytes, checksum: 70399c027bdcfb2e5676cb7cc2b4d049 (MD5) / Approved for entry into archive by Ana Rosa Silva (arosa@ufpa.br) on 2014-09-05T12:32:05Z (GMT) No. of bitstreams: 2 license_rdf: 23898 bytes, checksum: e363e809996cf46ada20da1accfcd9c7 (MD5) Dissertacao_OtimizacaoProcessoAprendizagem.pdf: 1776244 bytes, checksum: 70399c027bdcfb2e5676cb7cc2b4d049 (MD5) / Made available in DSpace on 2014-09-05T12:32:05Z (GMT). No. of bitstreams: 2 license_rdf: 23898 bytes, checksum: e363e809996cf46ada20da1accfcd9c7 (MD5) Dissertacao_OtimizacaoProcessoAprendizagem.pdf: 1776244 bytes, checksum: 70399c027bdcfb2e5676cb7cc2b4d049 (MD5) Previous issue date: 2014 / CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / A automação na gestão e análise de dados tem sido um fator crucial para as empresas que necessitam de soluções eficientes em um mundo corporativo cada vez mais competitivo. A explosão do volume de informações, que vem se mantendo crescente nos últimos anos, tem exigido cada vez mais empenho em buscar estratégias para gerenciar e, principalmente, extrair informações estratégicas valiosas a partir do uso de algoritmos de Mineração de Dados, que comumente necessitam realizar buscas exaustivas na base de dados a fim de obter estatísticas que solucionem ou otimizem os parâmetros do modelo de extração do conhecimento utilizado; processo que requer computação intensiva para a execução de cálculos e acesso frequente à base de dados. Dada a eficiência no tratamento de incerteza, Redes Bayesianas têm sido amplamente utilizadas neste processo, entretanto, à medida que o volume de dados (registros e/ou atributos) aumenta, torna-se ainda mais custoso e demorado extrair informações relevantes em uma base de conhecimento. O foco deste trabalho é propor uma nova abordagem para otimização do aprendizado da estrutura da Rede Bayesiana no contexto de BigData, por meio do uso do processo de MapReduce, com vista na melhora do tempo de processamento. Para tanto, foi gerada uma nova metodologia que inclui a criação de uma Base de Dados Intermediária contendo todas as probabilidades necessárias para a realização dos cálculos da estrutura da rede. Por meio das análises apresentadas neste estudo, mostra-se que a combinação da metodologia proposta com o processo de MapReduce é uma boa alternativa para resolver o problema de escalabilidade nas etapas de busca em frequência do algoritmo K2 e, consequentemente, reduzir o tempo de resposta na geração da rede. / Automation at data management and analysis has been a crucial factor for companies which need efficient solutions in an each more competitive corporate world. The explosion of the volume information, which has remained increasing in recent years, has demanded more and more commitment to seek strategies to manage and, especially, to extract valuable strategic informations from the use of data mining algorithms, which commonly need to perform exhausting queries at the database in order to obtain statistics that solve or optimize the parameters of the model of knowledge discovery selected; process which requires intensive computing to perform calculations and frequent access to the database. Given the effectiveness of uncertainty treatment, Bayesian networks have been widely used for this process, however, as the amount of data (records and/or attributes) increases, it becomes even more costly and time consuming to extract relevant information in a knowledge base. The goal of this work is to propose a new approach to optimization of the Bayesian Network structure learning in the context of BigData, by using the MapReduce process, in order to improve the processing time. To that end, it was generated a new methodology that includes the creation of an Intermediary Database, containing all the necessary probabilities to the calculations of the network structure. Through the analyzes presented at this work, it is shown that the combination of the proposed methodology with the MapReduce process is a good alternative to solve the scalability problem of the search frequency steps of K2 algorithm and, as a result, to reduce the response time generation of the network.

The goal of this diploma thesis is to design and develop a system for collecting, processing and storing data from large IoT networks. The developed system introduces a complex solution able to process data from various IoT networks using Apache Hadoop ecosystem. The data are real-time processed and stored in a NoSQL database, but the data are also stored  in the file system for a potential later processing. The system is optimized and tested using data from IQRF network. The data stored in the NoSQL database are visualized and the system periodically generates derived predictions. Users are connected to this system via an information system, which is able to automatically generate notifications when monitored values are out of range.

Submitted by camilla martins (camillasmmartins@gmail.com) on 2017-01-27T16:34:20Z No. of bitstreams: 2 license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) Dissertacao_StormsomClusterizacaoTempo-Real.pdf: 1081222 bytes, checksum: 30261425224872c11433d064abb4a2d8 (MD5) / Approved for entry into archive by Edisangela Bastos (edisangela@ufpa.br) on 2017-01-30T13:30:32Z (GMT) No. of bitstreams: 2 license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) Dissertacao_StormsomClusterizacaoTempo-Real.pdf: 1081222 bytes, checksum: 30261425224872c11433d064abb4a2d8 (MD5) / Made available in DSpace on 2017-01-30T13:30:32Z (GMT). No. of bitstreams: 2 license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) Dissertacao_StormsomClusterizacaoTempo-Real.pdf: 1081222 bytes, checksum: 30261425224872c11433d064abb4a2d8 (MD5) Previous issue date: 2015-08-28 / Cresce cada vez mais a quantidade de cenários e aplicações que algoritmo necessitam de processamento e respostas em tempo real e que se utilizam de modelos estatísticos e de mineração de dados a fim de garantir um melhor suporte à tomada de decisão. As ferramentas disponíveis no mercado carecem de processos computacionais mais refinados que sejam capazes de extrair padrões de forma mais eficiente a partir de grandes volumes de dados. Além disso, há a grande necessidade, em diversos cenários, que o os resultados sejam providos em tempo real, tão logo inicie o processo, uma resposta imediata já deve estar sendo produzida. A partir dessas necessidades identificadas, neste trabalho propomos um processo autoral, chamado StormSOM, que consiste em um modelo de processamento, baseado em topologia distribuída, para a clusterização de grandes volumes de fluxos, contínuos e ilimitados, de dados, através do uso de redes neurais artificiais conhecidas como mapas auto-organizáveis, produzindo resultados em tempo real. Os experimentos foram realizados em um ambiente de computação em nuvem e os resultados comprovam a eficiência da proposta ao garantir que o modelo neural utilizado possa gerar respostas em tempo real para o processamento de Big Data.

The door manufacturing industry still heavily relies on manual technology for its production and quality assurance systems, which poses certain challenges. However, in recent years, the industry has witnessed a growing demand for personalized products, leading to a need for more adaptable production methods and shorter product life cycles. Unfortunately, this reliance on manual technologies has resulted in increased errors and inaccuracies. Moreover, manual technology requires significant time and effort investment, which reduces production efficiency and product quality. To address these issues, the purpose of this thesis is to address the issue of inefficiency at a door manufacturing company by examining its existing production systems and quality assurance system. The thesis aims to provide recommendations for improvement by exploring the integration of automated tracing and tracking technology. Furthermore, it would result in recommendations for feasible methods that may be used in Swedish production systems, as well as further study fields.

The perspective of procurement and supply chain management is changing dramatically; traditionally, it was seen as a support function; however, the procurement function is receiving increased attention and investment as an essential contributor to the strategic success and a business enabler. While an end-to-end digital supply chain is an opportunity as it unleashes the next level of strategic growth and involves minimal investment in infrastructure, it is still a challenge to optimize and transform. Furthermore, the recent pandemics and geopolitical disruptions of Covid-19, the Ukraine-Russian war, Brexit and the US-China trade war; have structurally changed the global economy and revealed a new risk assessment that will result in the re-introduction of buffers, boundaries across industries and a partial return to regionalization with sort of de-globalization in which existing just-in-time getting replaced by just-in-case strategy.