In this paper, we discuss the consensus problem for synchronous distributed systems with orderly crash failures. For a synchronous distributed system of n processes with up to t crash failures and f failures actually occur, first, we present a bivalency argument proof to solve the open problem of proving the lower bound, min (t + 1, f + 2) rounds, for early-stopping synchronous consensus with orderly crash failures, where t < n - 1. Then, we extend the system model with orderly crash failures to a new model in which a process is allowed to send multiple messages to the same destination process in a round and the failing processes still respect the order specified by the protocol in sending messages. For this new model, we present a uniform consensus protocol, in which all non-faulty processes always decide and stop immediately by the end of f + 1 rounds. We prove that the lower bound of early stopping protocols for both consensus and uniform consensus are f + 1 rounds under the new model, and our proposed protocol is optimal. / Singapore-MIT Alliance (SMA)

The use of service-oriented (SO) distributed systems is increasing. Within service orientation web services (WS) are the de facto standard for implementing service-oriented systems. The consumers of WS want to get uninterrupted and reliable service from the service providers. But WS providers cannot always provide services in the expected level due to faults and failures in the system. As a result the fault management of these systems is becoming crucial. This work presents a distributed event-driven architecture for fault management of Web Services. According to the architecture the managed WS report different events to the event databases. From event databases these events are sent to the event processors. The event processors are distributed over the network. They process the events, detect fault scenarios in the event stream and manage faults in the WS.

We are addressing two aspects of vision based system development that are not fully exploited in current frameworks: abstraction over low-level details and high-level module reusability. Through an evaluation of existing frameworks, we relate these shortcomings to the lack of systematic classification of sub-tasks in vision based system development. Our approach for addressing these two issues is to classify vision into decoupled sub-tasks, hence defining a clear scope for a vision based system development framework and its sub-components. Firstly, we decompose the task of vision system development into data management and processing. We then proceed to further decompose data management into three components: data access, conversion and transportation. To verify our approach for vision system development we present two frameworks: the Vision Utility (VU) framework for providing abstraction over the data management component; and the Hive framework for providing the data transportation and high-level code reuse. VU provides the data management functionality for developers while hiding the low-level system details through a simple yet flexible Application Programming Interface (API). VU mediates the communication between the developer's application, vision processing modules, and data sources by utilizing different frameworks for data access, conversion and transportation (Hive). We demonstrate VU's ability for providing abstraction over low-level system details through the examination of a vision system developed using the framework. Hive is a standalone event based framework for developing distributed vision based systems. Hive provides simple high-level methods for managing communication, control and configuration of reusable components. We verify the requirements of Hive (reusability and abstraction over inter-module data transportation) by presenting a number of different systems developed on the framework using a set of reusable modules. Through this work we aim to demonstrate that this novel approach for vision system development could fundamentally change vision based system development by addressing the necessary abstraction, and promoting high-level code reuse.

A <i>web crawler</i> is a program that "walks" the Web to gather web resources. In order to scale to the ever-increasing Web, multiple crawling agents may be deployed in a distributed fashion to retrieve web data co-operatively. A common approach is to divide the Web into many partitions with an agent assigned to crawl within each one. If an agent obtains a web resource that is not from its partition, the resource will be transferred to the rightful owner. This thesis proposes a novel approach to distributed web data gathering by partitioning the Web into topics. The proposed approach employs multiple focused crawlers to retrieve pages from various topics. When a crawler retrieves a page of another topic, it transfers the page to the appropriate crawler. This approach is known as <i>topic-oriented collaborative web crawling</i>. An implementation of the system was built and experimentally evaluated. In order to identify the topic of a web page, a topic classifier was incorporated into the crawling system. As the classifier categorizes only English pages, a language identifier was also introduced to distinguish English pages from non-English ones. From the experimental results, we found that redundance retrieval was low and that a resource, retrieved by an agent, is six times more likely to be retained than a system that uses conventional hashing approach. These numbers were viewed as strong indications that <i>topic-oriented collaborative web crawling system</i> is a viable approach to web data gathering.

A <i>web crawler</i> is a program that "walks" the Web to gather web resources. In order to scale to the ever-increasing Web, multiple crawling agents may be deployed in a distributed fashion to retrieve web data co-operatively. A common approach is to divide the Web into many partitions with an agent assigned to crawl within each one. If an agent obtains a web resource that is not from its partition, the resource will be transferred to the rightful owner. This thesis proposes a novel approach to distributed web data gathering by partitioning the Web into topics. The proposed approach employs multiple focused crawlers to retrieve pages from various topics. When a crawler retrieves a page of another topic, it transfers the page to the appropriate crawler. This approach is known as <i>topic-oriented collaborative web crawling</i>. An implementation of the system was built and experimentally evaluated. In order to identify the topic of a web page, a topic classifier was incorporated into the crawling system. As the classifier categorizes only English pages, a language identifier was also introduced to distinguish English pages from non-English ones. From the experimental results, we found that redundance retrieval was low and that a resource, retrieved by an agent, is six times more likely to be retained than a system that uses conventional hashing approach. These numbers were viewed as strong indications that <i>topic-oriented collaborative web crawling system</i> is a viable approach to web data gathering.

The use of service-oriented (SO) distributed systems is increasing. Within service orientation web services (WS) are the de facto standard for implementing service-oriented systems. The consumers of WS want to get uninterrupted and reliable service from the service providers. But WS providers cannot always provide services in the expected level due to faults and failures in the system. As a result the fault management of these systems is becoming crucial. This work presents a distributed event-driven architecture for fault management of Web Services. According to the architecture the managed WS report different events to the event databases. From event databases these events are sent to the event processors. The event processors are distributed over the network. They process the events, detect fault scenarios in the event stream and manage faults in the WS.

<p>As cloud computing is gaining popularity, many application owners are migrating their</p><p>applications into the cloud. However, because of the diversity of the cloud environments</p><p>and the complexity of the modern applications, it is very challenging to find out which</p><p>cloud environment is best fitted for one's application.</p><p>In this dissertation, we design and build systems to help application owners select the</p><p>most suitable cloud environments for their applications. The first part of this thesis focuses</p><p>on how to compare the general fitness of the cloud environments. We present CloudCmp,</p><p>a novel comparator of public cloud providers. CloudCmp measures the elastic computing,</p><p>persistent storage, and networking services offered by a cloud along metrics that directly</p><p>reflect their impact on the performance of customer applications. CloudCmp strives to</p><p>ensure fairness, representativeness, and compliance of these measurements while limiting</p><p>measurement cost. Applying CloudCmp to four cloud providers that together account</p><p>for most of the cloud customers today, we find that their offered services vary widely in</p><p>performance and costs, underscoring the need for thoughtful cloud environment selection.</p><p>From case studies on three representative cloud applications, we show that CloudCmp can</p><p>guide customers in selecting the best-performing provider for their applications.</p><p>The second part focuses on how to let customers compare cloud environments in the</p><p>context of their own applications. We describe CloudProphet, a novel system that can</p><p>accurately estimate an application's performance inside a candidate cloud environment</p><p>without the need of migration. CloudProphet generates highly portable shadow programs</p><p>to mimic the behavior of a real application, and deploys them inside the cloud to estimate</p><p>the application's performance. We use the trace-and-replay technique to automatically</p><p>generate high-fidelity shadows, and leverage the popular dispatcher-worker pattern</p><p>to accurately extract and enforce the inter-component dependencies. Our evaluation in</p><p>three popular cloud platforms shows that CloudProphet can help customers pick the bestperforming</p><p>cloud environment, and can also accurately estimate the performance of a</p><p>variety of applications.</p> / Dissertation

The multi-core systems are usually implemented on homogeneous or heterogeneous cores, in order to design the better NOC (network on chip), it must consider the performance, scalability, simplifies hardware design and arbitration strategy at the on chip network. The routers are designed with circuit-switched network, circuit switching is asynchronous circuits and routers have no queuing (buffering), therefore, it is simple and efficient in implementation. Synchronous circuit is network with a clock source, but the distributing global clock has many problems such as power consumption, increasing the area and Clock skew. Ring topology with multi-transaction bus architecture. It could make multiple packets to access the bus at the same time, so that the multi-transaction bus architecture is better to get more throughputs. When the number of cores increase, the central arbiter circuit is more complexity, this thesis presents an SAP (self-adjusting priority) schedule that can fairly adjust priorities of each component by appropriately exchanging weighting at distributed arbiter. When numerous requests encounter contention on a network, a winner owning the highest priority will exchange its priority with the lowest priority of these requests. This principle guarantees that winners will decreased the opportunity of incurring network at the next time. In opposition, these losers can obtain the higher priority than that of the original. Therefore, the proposed scheme not only offers fair strategy, but also simplifies hardware design.

Over the past few years, a growing attention has been paid to the publish/subscribe (pub/sub) communication paradigm which has become the best model for disseminating information (also called events) through distributed systems on wide-area networks. There are many ways to implement the publish/subscribe system architecture. A common way is publishers and subscribers interact through one or more agents called broker. Broker has to store and management subscriptions, match messages, and efficiently delivery messages to subscribers. It also provides reliability and fault tolerant controlling. Obviously, broker is the most important part of the publish/subscribe system. It is usually required to use a lot of resources such as CPU and memory. In this paper, our publish/subscribe service which implemented by ZeroMQ API could service 10,000 subscribers only cost 3% CPU and 5% memory usage. It is a high performance and low costs publish/subscribe service.

With the progress of the network, peoples can communicate with each other easily. The popularization of the Broadband network makes many things that could be hard to achieved becoming possible. For example, video conference, distance education and on-line games have become a part of our daily life. However, most network services are client-server architecture. This kind of service need a expensive and powerful server. The restriction will makes it difficult to improve the quality of service. P2P (peer-to-peer) architecture embodies one of the key technical concepts of the internet and many internet users was attracted by it. A peer-to-peer computer network is a network that relies on computing power at the edges (ends) of a connection rather than in the network itself. It has been used in two kinds of applications which are file sharing and distributed computing. Instead of using P2P model, client-server architecture is used in muti-user conference. A software using P2P model still needs one of the users to become a server to deal with all the messages in the conference. When the number of the users increasing, the overhead of the server will increase linearly. This makes many communication softwares to limit the number of users in a conference. Therefore, most of the companies use an expensive and powerful server offering this kind of service. In this paper, we will implement a voice-conferencing system using P2P model and the technique of distributed computing. It makes the overhead of the server in logarithmic increase. In addition, we still can configure some low-end peers with a constant overhead. No mater how the number of users increase, these peers will have a constant overhead. With the good design, we can gurantee the quality of service easily.