近年來，隨著大數據時代的來臨，即時資料運算面臨許多挑戰。例如在期貨交易預測方面，為了精準的預測市場狀態，我們需要在海量資料中建立預測模型，且耗時在數十毫秒之內。 在本研究中，我們將介紹一套即時巨量資料運算架構，這套架構將解決在實務上需要解決的三大需求：高速處理需求、巨量資料處理以及儲存需求。同時，在整個平行運算系統之下，我們也實作了數種人工智慧演算法，例如SVM (Support Vector Machine)和LR (Logistic Regression)等，做為策略模擬的子系統。本架構包含下列三種主要的雲端運算技術： 1. 使用Apache YARN以整合整體系統資源，使叢集資源運用更具效率。 2. 為滿足高速處理需求，本架構使用Apache Storm以便處理海量且即時之資料流。同時，借助該框架，可在數十毫秒之內，運算上千種市場狀態數值供模型建模之用。 3. 運用Apache Spark，本研究建立了一套分散式運算架構用於模型建模。藉由使用Spark RDD(Resilient Distributed Datasets)，本架構可將SVM和LR之模型建模時間縮短至數百毫秒之內。 為解決上述需求，本研究設計了一套n層分散式架構且整合上列數種技術。另外，在該架構中，我們使用Apache Kafka作為整體系統之訊息中介層，並支持系統內各子系統間之非同步訊息溝通。 / With the coming of the era of big data, the immediacy and the amount of data computation are facing with many challenges. For example, for Futures market forecasting, we need to accurately forecast the market state with the model built from large data (hundreds of GB to tens of TB) within tens of milliseconds. In this research, we will introduce a real-time big data computing architecture to resolve requests of high speed processing, the immense volume of data and the request of large data processing. In the meantime, several algorithms, such as SVM (Support Vector Machine, SVM) and LR (Logistic Regression, LR), are implemented as a subproject under the parallel distributed computing system. This architecture involves three main cloud computing techniques: 1. Use Apache YARN as a system of integrated resource management in order to apply cluster resources more efficiently. 2. To satisfy the requests of high speed processing, we apply Apache Storm in order to process large real-time data stream and compute thousands of numerical value within tens of milliseconds for following model building. 3. With Apache Spark, we establish a distributed computing architecture for model building. By using Spark RDD (Resilient Distributed Datasets, RDD), this architecture can shorten the execution time to within hundreds of milliseconds for SVM and LR model building. To resolve the requirements of the distributed system, we design an n-tier distributed architecture to integrate the foregoing several techniques. In this architecture, we use the Apache Kafka as the messaging middleware to support asynchronous message-based communication.

Este trabalho apresenta um estudo, implementação e avaliação em ambiente real de um protótipo de servidor Web com diferenciação de serviços (SWDS) para provisão de QoS relativa em servidores Web. Para tanto foram considerados algoritmos de reserva de recursos e escalonamento baseado em prioridades para prover diferenciação entre as classes de serviço, além de mecanismos de controle de admissão a fim de controlar a carga no sistema. A meta é oferecer melhor tratamento para requisições de maior prioridade, sem prejudicar em excesso as requisições de prioridade menor. Foi observado que os algoritmos de reserva de recursos (RSV e RSVadap) são eficientes para prover diferenciação entre as classes consideradas, no entanto seus desempenhos não foram satisfatórios em alguns casos, tanto devido a arquitetura em que foram implementados, como por motivos inerentes à própria natureza do algoritmo. O algoritmo de escalonamento baseado em prioridades considerado (PriProcess), mostrou-se eficiente tanto no sentido de prover diferenciação de serviços entre as classes, como na questão de desempenho, com a classe de maior prioridade sempre sendo melhor atendida em relação à classe de menor prioridade. Também foi alvo do estudo a criação de um mecanismo de controle de admissão com diferenciação de serviços. Os resultados alcançados sinalizam uma melhora em termos de tempos de respostas e número de requisições completadas para a classe de maior prioridade / This work presents a study, implementation and evaluation in real environment of a Web server architecture with service differentiation (SWDS) for provision of relative QoS in Web servers. Algorithms of resources reservation and scheduling based on priorities were considered to provide service differentiation among the request classes, besides implementation of a mechanism for admission control in order to control the load in the system. The goal is to offer better treatment for higher priority requests without harming in excess lower priority ones. It was observed that the algorithms for resource reservation (RSV and RSVapap) are efficient to provide differentiation among the considered classes, however their performance was not satisfactory in some cases, due to both, the archtectures through which they were implemented and the specific features of the algorithm nature. The considered scheduling algorithm based on priorities (PriProcess) was efficient both in the sense of providing service differentiation among the classes and with regards to performance, with the higher priority class always being better served than the lower priority one. It was also studied the creation of a mechanism for admission control with service differentiation. The achieved results show an improvement in terms of response times and the number of completed requests for the higher priority class

Este trabalho apresenta um estudo, implementação e avaliação em ambiente real de um protótipo de servidor Web com diferenciação de serviços (SWDS) para provisão de QoS relativa em servidores Web. Para tanto foram considerados algoritmos de reserva de recursos e escalonamento baseado em prioridades para prover diferenciação entre as classes de serviço, além de mecanismos de controle de admissão a fim de controlar a carga no sistema. A meta é oferecer melhor tratamento para requisições de maior prioridade, sem prejudicar em excesso as requisições de prioridade menor. Foi observado que os algoritmos de reserva de recursos (RSV e RSVadap) são eficientes para prover diferenciação entre as classes consideradas, no entanto seus desempenhos não foram satisfatórios em alguns casos, tanto devido a arquitetura em que foram implementados, como por motivos inerentes à própria natureza do algoritmo. O algoritmo de escalonamento baseado em prioridades considerado (PriProcess), mostrou-se eficiente tanto no sentido de prover diferenciação de serviços entre as classes, como na questão de desempenho, com a classe de maior prioridade sempre sendo melhor atendida em relação à classe de menor prioridade. Também foi alvo do estudo a criação de um mecanismo de controle de admissão com diferenciação de serviços. Os resultados alcançados sinalizam uma melhora em termos de tempos de respostas e número de requisições completadas para a classe de maior prioridade / This work presents a study, implementation and evaluation in real environment of a Web server architecture with service differentiation (SWDS) for provision of relative QoS in Web servers. Algorithms of resources reservation and scheduling based on priorities were considered to provide service differentiation among the request classes, besides implementation of a mechanism for admission control in order to control the load in the system. The goal is to offer better treatment for higher priority requests without harming in excess lower priority ones. It was observed that the algorithms for resource reservation (RSV and RSVapap) are efficient to provide differentiation among the considered classes, however their performance was not satisfactory in some cases, due to both, the archtectures through which they were implemented and the specific features of the algorithm nature. The considered scheduling algorithm based on priorities (PriProcess) was efficient both in the sense of providing service differentiation among the classes and with regards to performance, with the higher priority class always being better served than the lower priority one. It was also studied the creation of a mechanism for admission control with service differentiation. The achieved results show an improvement in terms of response times and the number of completed requests for the higher priority class

No description available.

The Western Apache Scouts of the 1870s who assisted the United States Army in tracking down the Chiricahua Apaches that had escaped from the federal reservations in the Arizona Territory laid the foundation for what became seventy-six years of military service in the U.S. Army. Consolidated and reassigned to Ft. Huachuca, Arizona in 1922, these scouts continued to serve with distinction long after the Army needed their skills as trackers. In 1947, the final four scouts retired from United States military service, each having served for over twenty-five years. This thesis explores how these men used their military service in order to survive, serving with honor while maintaining their cultural traditions within a changing world.

Stuttgart, Univ., Diplomarbeit, 2007.

Thesis (M.A. in History)--University of California, Berkeley, Dec. 1918. / Includes bibliographical references (leaves 220-229).

No description available.

This thesis aims to investigate the prospect of using Apache Kafka to manage data streams based on secrecy/classification level and separate these data streams in order to meet the requirement set by the secrecy/classification levels. Basalt AB has the responsibility of managing classified data for private and state actors, including the Swedish Armed Forces and other organizations. There is interest in a data-streaming solution that can securely stream large amounts of data while coordinating different data classifications and managing user access. This thesis work examines the viability of logically and physically separating producer data streams into categories based on the classification level of the data in an Apache Kafka cluster. Additionally, the thesis examines the viability of managing access control through the use of Access Control Lists. To protect against embedded attackers this thesis examines the viability of using Shamir Secret Sharing (SSS) algoritm to segment messages to on top of that use multi-factor authentication to ensure that messages cannot be read by a lone individual. The work seeks to contribute to the existing body of knowledge by improving the security, and ensuring the integrity of data through the application of detailed or granular user management of event-logs in an Apache Kafka cluster. This is of interest to organizations that require protection from both external and internal potential attackers. Our results indicate that Apache Kafka is an appropriate tool for data streaming secret data, we used a secret sharing algorithm to segment data and used Simple Authentication and Security Layer to build a multi-factor authentication system.

No description available.