It is well known that the transport layer protocol TCP has low throughput and is unfair in wireless mesh networks. Transport layer solutions for mesh networks have been primarily validated using simulations with simplified assumptions about the wireless network. The WiFu Transport framework complements simulator results by allowing developers to easily create and experiment with transport layer protocols on live networks. We provide a user-space solution that is flexible and promotes code reuse while maintaining high performance and scalability. To validate WiFu Transport we use it to build WiFu TCP, a decomposed Tahoe solution that preserves TCP semantics. Furthermore, we share other WiFu developers' experiences building several TCP variants as well as a hybrid protocol to demonstrate flexibility and code reuse. We demonstrate that WiFu Transport performs as well as the Linux kernel on 10 and 100 Mbps Ethernet connections and over a one-hop wireless connection. We also show that our WiFu TCP implementation is fair and that the framework also scales to support multiple threads.

The ever-growing demand for more computing power forces hardware vendors to put an increasing number of multiprocessors into a single server system, which usually exhibits a non-uniform memory access (NUMA). In-memory database systems running on NUMA platforms face several issues such as the increased latency and the decreased bandwidth when accessing remote main memory. To cope with these NUMA-related issues, a DBMS has to allow flexible data partitioning and data placement at runtime. In this demonstration, we present ERIS, our NUMA-aware in-memory storage engine. ERIS uses an adaptive partitioning approach that exploits the topology of the underlying NUMA platform and significantly reduces NUMA-related issues. We demonstrate throughput numbers and hardware performance counter evaluations of ERIS and a NUMA-unaware index for different workloads and configurations. All experiments are conducted on a standard server system as well as on a system consisting of 64 multiprocessors, 512 cores, and 8 TBs main memory.

Visma Labs AB or Visma wanted to conduct experiments to see if file metadata could be indexed for searching files on a public cloud storage. Given that storing files in a public cloud storage is cheaper than the current storage solution, the implementation could save Visma money otherwise spent on expensive storage costs. The thesis is therefore to find and evaluate an approach chosen for indexing file metadata and searching files on a public cloud storage with the chosen distributed search engine Elasticsearch. The architecture of the proposed solution is similar to a file service and was implemented using several containerized services for it to function. The results show that the file service solution is indeed feasible but would need further tuning and more resources to function according to the demands of Visma. / Visma Labs AB eller Visma ville genomföra experiment för att se om filmetadata skulle kunna indexeras för att söka efter filer på ett publikt moln. Med tanke på att lagring av filer på ett publikt moln är billigare än den nuvarande lagringslösningen, kan implementeringen spara Visma pengar som spenderas på dyra lagringskostnader. Denna studie är därför till för att hitta och utvärdera ett tillvägagångssätt valt för att indexera filmetadata och söka filer på ett offentligt molnlagring med den utvalda distribuerade sökmotorn Elasticsearch. Arkitekturen för den föreslagna lösningen har likenelser av en filtjänst och implementerades med flera containeriserade tjänster för att den ska fungera. Resultaten visar att filservicelösningen verkligen är möjlig men skulle behöva ytterligare modifikationer och fler resurser att fungera enligt Vismas krav.

Wind energy is on an upswing due to climate concerns and increasing energy demands on conventional sources. Wind energy is attractive and has the potential to dramatically reduce the dependency on non-renewable energy resources. With the increase in wind farms there is a need to improve the efficiency in power allocation and power generation among wind turbines. Wake interferences among wind turbines can lower the overall efficiency considerably, while offshore conditions pose increased loading on wind turbines. In wind farms, wind turbines* wake affects each other depending on their positions and operation modes. Therefore it becomes essential to optimize the wind farm power production as a whole than to just focus on individual wind turbines. The work presented here develops a hierarchical power optimization algorithm for wind farms. The algorithm includes a cooperative level (or higher level) and an individual level (or lower level) for power coordination and planning in a wind farm. The higher level scheme formulates and solves a quadratic constrained programming problem to allocate power to wind turbines in the farm while considering the aerodynamic effect of the wake interaction among the turbines and the power generation capabilities of the wind turbines. In the lower level, optimization algorithm is based on a leader-follower structure driven by the local pursuit strategy. The local pursuit strategy connects the cooperative level power allocation and the individual level power generation in a leader-follower arrangement. The leader, could be a virtual entity and dictates the overall objective, while the followers are real wind turbines considering realistic constraints, such as tower deflection limits. A nonlinear wind turbine dynamics model is adopted for the low level study with loading and other constraints considered in the optimization. The stability of the algorithm in the low level is analyzed for the wind turbine angular velocity. Simulations are used to show the advantages of the method such as the ability to handle non-square input matrix, non-homogenous dynamics, and scalability in computational cost with rise in the number of wind turbines in the wind farm.

The growing demand on the variety of internet applications requires management of large scale networks by efficient Quality of Service (QoS) routing, which considerably contributes to the QoS architecture. The biggest contemporary drawback in the maintenance and distribution of the global state is the increase in communication overheads. Unbalancing in the network, due to the frequent use of the links assigned to the shortest path retaining most of the network loads is regarded as a major problem for best effort service. Localised QoS routing, where the source nodes use statistics collected locally, is already described in contemporary sources as more advantageous. Scalability, however, is still one of the main concerns of existing localised QoS routing algorithms. The main aim of this thesis is to present and validate new localised algorithms in order to develop the scalability of QoS routing. Existing localised routing, Credit Based Routing (CBR) and Proportional Sticky Routing (PSR), use the blocking probability as a factor in selecting the routing paths and work with either credit or flow proportion respectively, which makes impossible having up-to-date information. Therefore our proposed Highest Minimum Bandwidth (HMB) and Highest Average Bottleneck Bandwidth History (HABBH) algorithms utilise bandwidth as the direct QoS criterion to select routing paths. We introduce an Integrated Delay Based Routing and Admission Control mechanism. Using this technique Minimum Total Delay (MTD), Low Fraction Failure (LFF) and Low Path Failure (LPF) were compared against the global QoS routing scheme, Dijkstra, and localised High Path Credit (HPC) scheme and showed superior performance. The simulation with the non-uniformly distributed traffic reduced blocking probability of the proposed algorithms. Therefore, we advocate the algorithms presented in the thesis, as a scalable approach to control large networks. We strongly suggest that bandwidth and mean delay are feasible QoS constraints to select optimal paths by locally collected information. We have demonstrated that a few good candidate paths can be selected to balance the load in the network and minimise communication overhead by applying the disjoint paths method, recalculation of candidate paths set and dynamic paths selection method. Thus, localised QoS routing can be used as a load balancing tool in order to improve the network resource utilization. A delay and bandwidth combination is one of the future prospects of our work, and the positive results presented in the thesis suggest that further development of a distributed approach in candidate paths selection may enhance the proposed localised algorithms. / Umm AlQura University in Mecca

Managing and efficiently analysing the vast amounts of data produced by a huge variety of data sources is one of the big challenges in computer science. The development and implementation of algorithms and applications that can extract information diamonds from these ultra-large, and often distributed, databases is a key challenge for the design of future data management infrastructures. Today’s data-intensive applications often suffer from performance problems and an inability to scale to high numbers of distributed data sources. Therefore, distributed and parallel databases have a key part to play in overcoming resource bottlenecks, achieving guaranteed quality of service and providing system scalability. The increased availability of distributed architectures, clusters, Grids and P2P systems, supported by high performance networks and intelligent middleware provides parallel and distributed databases and digital repositories with a great opportunity to cost-effectively support key everyday applications. Further, there is the prospect of data mining and knowledge discovery tools adding value to these vast new data resources by automatically extracting useful information from them.

Anomaly detection consists in automatically detecting the most unusual elements in a data set. Anomaly detection applications emerge in domains such as computer security, system monitoring, fault detection, and wireless sensor networks. The strategic importance of detecting anomalies in these domains makes anomaly detection a critical data analysis task. Moreover, the contextual nature of anomalies, among other issues, makes anomaly detection a particularly challenging problem. Anomaly detection has received significant research attention in the last two decades. Much effort has been invested in the development of novel algorithms for anomaly detection. However, several open challenges still exist in the field.This thesis presents our contributions toward solving these challenges. These contributions include: a methodological survey of the recent literature, a novel benchmarking framework for anomaly detection algorithms, an approach for scaling anomaly detection techniques to massive data sets, and a novel anomaly detection algorithm inspired by the law of universal gravitation. Our methodological survey highlights open challenges in the field, and it provides some motivation for our other contributions. Our benchmarking framework, named BAD, tackles the problem of reliably assess the accuracy of unsupervised anomaly detection algorithms. BAD leverages parallel and distributed computing to enable massive comparison studies and hyperparameter tuning tasks. The challenge of scaling unsupervised anomaly detection techniques to massive data sets is well-known in the literature. In this context, our contributions are twofold: we investigate the trade-offs between a single-threaded implementation and a distributed approach considering price-performance metrics, and we propose a scalable approach for anomaly detection algorithms to arbitrary data volumes. Our results show that, when high scalability is required, our approach can handle arbitrarily large data sets without significantly compromising detection accuracy. We conclude our contributions by proposing a novel algorithm for anomaly detection, named Gravity. Gravity identifies anomalies by considering the attraction forces among massive data elements. Our evaluation shows that Gravity is competitive with other popular anomaly detection techniques on several benchmark data sets. Additionally, the properties of Gravity makes it preferable in cases where hyperparameter tuning is challenging or unfeasible.

Today’s web applications make extensive use of APIs between server and client, or server to server in order to provide new information in the form of events. The question was whether the different methods of procuring events are different in how they scale. This study aims to compare performance between webhooks and polling, the two most commonly used pull and push based methods for event notification when scaling up traffic. The purpose is to create a basis for developers when choosing the method for event notification. The comparison has been developed through measurements of typical indicators of good performance for web applications: CPU usage, memory usage and response time. The tests gave indications that webhooks perform better in most circumstances, but further testing is needed in a more well-defined environment to draw a confident conclusion. / Dagens webbapplikationer använder sig i stor utsträckning av API:er mellan server och klient, eller server till server för att inhämta ny information i form av events (händelser). Frågan är om de olika metoder som finns för att inhämta events skalar olika bra. Förevarande studie ämnar att jämföra prestanda mellan ”webhooks” och ”polling”, de två mest använda pull- och pushbaserade metoderna för eventnotifikation vid uppskalning av trafik. Syftet är att skapa ett underlag för utvecklare vid valet av metod för eventnotifikation. Jämförelsen har tagits fram genom mätningar av typiska indikatorer för god prestanda hos en webbapplikation: CPU-användning, minnesanvändning och svarstid. Testerna gav indikationer om att webhooks är bättre men det krävs vidare testning i en mer väldefinierad miljö för att dra en säkrare slutsats.

Data-intensive scientific applications often process an enormous amount of data. The scalability of such applications depends critically on how to manage the locality of data. Our study explores two common types of applications that are vastly different in terms of memory access pattern and workload variation. One includes those with multi-stride accesses in regular nested parallel loops. The other is for processing large-scale irregular social network graphs. In the former case, the memory location or the data item accessed in a loop is predictable and the load on processing a unit work (an array element) is relatively uniform with no significant variation. On the other hand, in the latter case, the data access per unit work (a vertex) is highly irregular in terms of the number of accesses and the locations being accessed. This property is further tied to the load and presents significant challenges in the scalability of the application performance. Designing platforms to support extreme performance scaling requires understanding of how application specific information can be used to control the locality and improve the performance. Such insights are necessary to determine which control and which abstraction to provide for interfacing an underlying system and an application as well as for designing a new system. Our goal is to expose common requirements of data-intensive scientific applications for scalability. For the former type of applications, those with regular accesses and uniform workload, we contribute new methods to improve the temporal locality of software-managed local memories, and optimize the critical path of scheduling data transfers for multi-dimensional arrays in nested loops. In particular, we provide a runtime framework allowing transparent optimization by source-to-source compilers or automatic fine tuning by programmers. Finally, we demonstrate the effectiveness of the approach by comparing against a state-of-the-art language-based framework. For the latter type, those with irregular accesses and non-uniform workload, we analyze how the heavy-tailed property of input graphs limits the scalability of the application. Then, we introduce an application-specific workload model as well as a decomposition method that allows us to optimize locality with the custom load balancing constraints of the application. Finally, we demonstrate unprecedented strong scaling of a contagion simulation on two state-of-the-art high performance computing platforms. / Ph. D.

No description available.