Resource Description Framework (RDF) est devenu un modèle de données pertinentafin de décrire et de modéliser les informations qui sont partagées sur le Web.Cependant, fournir une solution permettant de stocker et de récupérer cesdonnées de manière efficace tout en passant à l’échelle reste un défi majeur.Dans le contexte de cette thèse nous proposons un intergiciel dévoué austockage, à la récupération synchrone mais aussi à la dissémination sélectiveet asynchrone en quasi temps réel d'informations RDF dans un environnementcomplètement distribué. L’objectif est de pouvoir tirer parti des informationsdu passé comme de celles filtrées en quasi temps réel. Contrairement à unegrande majorité de solutions existantes, nous avons avons fait le choixd’éviter le hachage pour indexer les données ce qui nous permet de traiter lesrequêtes à intervalles de manière efficace. Le filtrage des informations enquasi temps réel est permis par l’expression des intérêts à l’aide desouscriptions basées sur le contenu des évènements futurs. Nous avons proposédeux algorithmes qui permettent de vérifier la concordance des évènements RDFavec les souscriptions enregistrées. Les deux algorithmes ont été testésexpérimentalement. En sus de la récupération synchrone et de la diffusionasynchrone d’évènements, nous nous sommes intéressés à améliorer la répartitiondes données RDF qui souffrent de dissymétrie. Finalement, nous avons consacréun effort non négligeable à rendre notre intergiciel modulaire. / RDF has become a relevant data model for describing and modeling information on the Web but providing scalable solutions to store and retrieve RDF data in a responsive manner is still challenging. Within the context of this thesis we propose a middleware devoted to storing, retrieving synchronously but also disseminating selectively and asynchronously RDF data in a fully distributed environment. Its purposes is to allow to leverage historical information and filter data near real-time. To this aims we have built our system atop a slightly modified version of a 4-dimensional Content Addressable Network (CAN) overlay network reflecting the structure of an RDF tuple. Unlike many existing solutions we made the choice to avoid hashing for indexing data, thus allowing efficient range queries resolution. Near realtime filtering is enabled by expressing information preferences in advance through content-based subscriptions handled by a publish/subscribe layer designed atop the CAN architecture. We have proposed two algorithms to check RDF data or events satisfaction with subscriptions but also to forward solutions to interested parties. Both algorithms have been experimentally tested for throughput and scalability. Although one performs better than the other, they remain complementary to ensure correctness. Along with information retrieval and dissemination, we have proposed a solution to enhance RDF data distribution on our revised CAN network since RDF information suffers from skewness. Finally, to improve maintainability and reusability some efforts were also dedicated to provide a modular middleware reducing the coupling between its underlying software artifacts.

While developing modern applications, it is necessary to ensure an efficient and performant communication between different applications. In current environments, a middleware software is used, which supports the publish/subscribe communication pattern. Using this communication pattern, a publisher sends information encapsulated in messages to the middleware. A subscriber registers its interests at the middleware. The monograph describes three different steps to determine the performance of such a system. In a first step, the message throughput performance of a publish/subscribe in different scenarios is measured using a Java Message Service (JMS) based implementation. In the second step the maximum achievable message throughput is described by adapted models depending on the filter complexity and the replication grade. Using the model, the performance characteristics of a specific system in a given scenario can be determined. These numbers are used for the queuing model described in the third part of the thesis, which supports the dimensioning of a system in realistic scenarios. Additionally, we introduce a method to approximate an M/G/1 system numerically in an efficient way, which can be used for real time analysis to predict the expected performance in a certain scenario. Finally, the analytical model is used to investigate different possibilities to ensure the scalability of the maximum achievable message throughput of the overall system. / Bei der Entwicklung moderner Applikationen ist es notwendig eine effiziente und performante Kommunikation zwischen den einzelnen Anwendungen sicherzustellen. In der Praxis kommt dabei eine Middleware Software zum Einsatz, die das Publish/Subscribe Kommunikationsmuster unterstützt. Dabei senden Publisher Informationen in Form von Nachrichten an die Middleware. Die Subscriber hingegen zeigen durch die Nutzung von Filtern der Middleware an, welche Informationen zugestellt werden sollen. Die Arbeit beschreibt ein dreistufiges Verfahren zur Leistungsbestimmung eines solchen Systems. Zunächst wird durch Messung die Leistung von Publish/Subscribe Systemen in verschiedenen Szenarien untersucht am Beispiel von Java Message Service (JMS) basierten Implementierungen. Danach wird der maximale Nachrichtendurchsatz in Abhängigkeit der Filterkomplexität und des Nachrichtenreplikationsgrades durch einfache Modelle beschrieben. Damit können die Leistungskennwerte für ein System und vorgegebenen Randbedingungen beschrieben werden. Im dritten Teil wird mittels Leistungsbewertung und durch Anwendung eines Warteschlangenmodells die Leistung in praxisnahen Umfeld beschrieben, so dass eine Dimensionierung möglich wird. Zusätzlich wird ein mathematisch, approximatives Verfahren vorgestellt, um ein M/G/1 System numerisch effizient berechnen zu können, was bei der Echtzeitbewertung eines Systems zur Leistungsvorhersage benutzt werden kann. Des Weiteren werden mittels des Modells Möglichkeiten untersucht die Skalierbarkeit des Gesamtsystems in Bezug auf den Nachrichtendurchsatz sicherzustellen.

The increasing amount of electronically available information in society today is undeniable. Examples include the numbers of general web pages, scientific publications, and items in online auctions. From a user's perspective, this trend will lead to information overflow. Moreover, information publishers are compromised by this situation, as users have greater difficulty in identifying useful information. Publish-subscribe systems can be applied to cope with the reality of information overflow. In these systems, users specify their information interests as subscriptions and, subsequently, only matching information (event messages) is delivered; uninteresting information is filtered out before reaching users. In this dissertation, we consider content-based publish-subscribe systems, a sophisticated example of these systems. They perform the information-filtering task based on the content of provided information. In order to deal with high numbers of subscriptions and frequencies of event messages, publish-subscribe systems are realized as distributed systems. Advertisements---publisher specifications of potential future event messages---are optionally applied in these systems to reduce the internal distribution of subscriptions. Existing work on content-based publish-subscribe concepts mainly focuses on subscriptions and advertisements as pure conjunctive expressions. Therefore, subscriptions or advertisements using operators other than conjunction need to be canonically converted to disjunctive normal form by these systems. Each conjunctive component is then treated as individual subscription or advertisement. Unfortunately, the size of converted expressions is exponential in the worst case. In this dissertation, we show that the direct support of general Boolean subscriptions and advertisements improves the time and space efficiency of general-purpose content-based publish-subscribe systems. For this purpose, we develop suitable approaches for the filtering and routing of general Boolean expressions in these systems. Our approaches represent solutions to exactly those components of content-based publish-subscribe systems that currently restrict subscriptions and advertisements to conjunctive expressions. On the subscription side, we present an effective generic filtering algorithm, and a novel approach to optimize event routing tables, which we call subscription pruning. To support advertisements, we show how to calculate the overlap between subscriptions and advertisements, and introduce the first designated subscription routing optimization, which we refer to as advertisement pruning. We integrate these approaches into our prototype BoP (BOolean Publish-subscribe) which allows for the full support of general Boolean expressions in its filtering and routing components. In the evaluation part of this dissertation, we empirically analyze our prototypical implementation BoP and compare its algorithms to existing conjunctive solutions. We firstly show that our general-purpose Boolean filtering algorithm is more space- and time-efficient than a general-purpose conjunctive filtering algorithm. Secondly, we illustrate the effectiveness of the subscription pruning routing optimization and compare it to the existing covering optimization approach. Finally, we demonstrate the optimization effect of advertisement pruning while maintaining the existing overlapping relationships in the system.

<p>With the advent of Web 2.0 and the Digital Age, we are witnessing an unprecedented increase in the amount of information collected, and in the number of users interested in different types of information. This growth means that traditional techniques, where users poll data sources for information of interest, are no longer sufficient. Polling too frequently does not scale, while polling less often may result in users missing important updates. The alternative push technology has long been the goal of publish/subscribe systems, which proactively push updates (events) to users with matching interests (expressed as subscriptions). The push model is better suited for ensuring scalability and timely delivery of updates, important in many application domains: personal (e.g., RSS feeds, online auctions), financial (e.g., portfolio monitoring), security (e.g., reporting network anomalies), etc.</p><p>Early publish/subscribe systems were based on predefined subjects (channels), and were too coarse-grained to meet the specific interests of different subscribers. The second generation of content-based publish/subscribe systems offer greater flexibility by supporting subscriptions defined as predicates over message contents. However, subscriptions are still stateless filters over individual messages, so they cannot express queries across different messages or over the event history. The few systems that support more powerful database-style subscriptions do not address the problem of efficiently delivering updates to a large number of subscribers over a wide-area network. Thus, there is a need to develop next-generation publish/subscribe systems that unify the support for richer database-style subscription queries and flexible wide-area notification. This support needs to be complemented with robust processing and dissemination techniques that scale to high event rates and large databases, as well as to a large number of subscribers over the Internet.</p><p>The main contribution of our work is a collection of techniques to support efficient and scalable event processing and notification dissemination for an Internet-scale publish/subscribe system with a rich subscription model. We investigate the interface between event processing by a database server and notification delivery by a dissemination network. Previous research in publish/subscribe has largely been compartmentalized; database-centric and network-centric approaches each have their own limitations, and simply putting them together does not lead to an efficient solution. A closer examination of database/network interfaces yields a spectrum of new and interesting possibilities. In particular, we propose message and subscription reformulation as general techniques to support stateful subscriptions over existing content-driven networks, by converting them into equivalent but stateless forms. We show how reformulation can successfully be applied to various stateful subscriptions including range-aggregation, select-joins, and subscriptions with value-based notification conditions. These techniques often provide orders-of-magnitude improvement over simpler techniques adopted by current systems, and are shown to scale to millions of subscriptions. Further, the use of a standard off-the-shelf content-driven dissemination interface allows these techniques to be easily deployed, managed, and maintained in a large-scale system.</p><p>Based on our findings, we have built a high-performance publish/subscribe system named ProSem (to signify the inseparability of database processing and network dissemination). ProSem uses our novel techniques for group-processing many types of complex and expressive subscriptions, with a per-event optimization framework that chooses the best processing and dissemination strategy at runtime based on online statistics and system objectives.</p> / Dissertation

Over the past few years, a growing attention has been paid to the publish/subscribe (pub/sub) communication paradigm which has become the best model for disseminating information (also called events) through distributed systems on wide-area networks. There are many ways to implement the publish/subscribe system architecture. A common way is publishers and subscribers interact through one or more agents called broker. Broker has to store and management subscriptions, match messages, and efficiently delivery messages to subscribers. It also provides reliability and fault tolerant controlling. Obviously, broker is the most important part of the publish/subscribe system. It is usually required to use a lot of resources such as CPU and memory. In this paper, our publish/subscribe service which implemented by ZeroMQ API could service 10,000 subscribers only cost 3% CPU and 5% memory usage. It is a high performance and low costs publish/subscribe service.

The growing popularity of mobile platforms is changing the Internet user’s computing experience. Current studies suggest that the traditional ubiquitous computing landscape is shifting towards more enhanced and broader mobile computing platform consists of large number of heterogeneous devices. Smartphones and tablets begin to replace the desktop as the primary means of interacting with IT resources. While mobile devices facilitate in consuming web resources in the form of web services, the growing demand for consuming services on mobile device is introducing a complex ecosystem in the mobile environment. This research addresses the communication challenges involved in mobile distributed networks and proposes an event-driven communication approach for information dissemination. This research investigates different communication techniques such as synchronous and asynchronous polling and long-polling, server-side push as mechanisms between client-server interactions and the latest web technologies namely HTML5 standard WebSocket as communication protocol within a publish/subscribe paradigm. Finally, this research introduces and evaluates a framework that is hybrid of REST and event-based publish/subscribe for operating in the mobile environment.

Publish/Subscribe (in short pub/sub) allows clients that share common interest communicate in an asynchronous and loosely-coupled fashion. This paradigm is adopted by many distributed event-driven applications such as social networking services, distributed business processes and cyber-physical systems. These applications cannot afford to have the underlying pub/sub substrate perform unreliably, permanently fail or behave arbitrarily as it will cause significant disturbance to stably serving many end-users. Therefore, a research effort on making pub/sub systems resilient against various failures to sustain high quality of service to the clients is imperative. In this thesis, we focus on the overlay of pub/sub brokers that are widely adopted as a popular architecture for large-scale pub/sub systems. Broker overlays can suffer from various issues such as degradation of topology quality, brokers causing transient or permanent benign failures and Byzantine brokers behaving arbitrarily. We aim to make novel research contributions by exploring fundamental techniques that can help the broker overlays maintain functional and non-functional requirements even under the presence of the aforementioned failures and necessary administrative updates. We first build a set of overlay adaptation primitives that re-configure topologies such as shifting links and replicating brokers. These primitives are designed to involve a small local group of brokers in the pub/sub overlays so that the disruption during the execution of large-scale and dynamic changes can be controlled in a fined-grained manner. For the problem of degrading topology quality, automated planning systems are developed to find a sequence of adaptations that would cause minimal disruption to running services. Also, our primitives can be executed on demand to quickly fail-over a crashed broker or off-load congested brokers. In addition, these on-demand primitives can be used to form a group of dynamically replicated brokers that enforce a novel safety measure to prevent Byzantine brokers from sabotaging the pub/sub overlays. Our contributions are evaluated with systematic consideration of various trade-offs between functional and non-functional properties.

Publish/Subscribe (in short pub/sub) allows clients that share common interest communicate in an asynchronous and loosely-coupled fashion. This paradigm is adopted by many distributed event-driven applications such as social networking services, distributed business processes and cyber-physical systems. These applications cannot afford to have the underlying pub/sub substrate perform unreliably, permanently fail or behave arbitrarily as it will cause significant disturbance to stably serving many end-users. Therefore, a research effort on making pub/sub systems resilient against various failures to sustain high quality of service to the clients is imperative. In this thesis, we focus on the overlay of pub/sub brokers that are widely adopted as a popular architecture for large-scale pub/sub systems. Broker overlays can suffer from various issues such as degradation of topology quality, brokers causing transient or permanent benign failures and Byzantine brokers behaving arbitrarily. We aim to make novel research contributions by exploring fundamental techniques that can help the broker overlays maintain functional and non-functional requirements even under the presence of the aforementioned failures and necessary administrative updates. We first build a set of overlay adaptation primitives that re-configure topologies such as shifting links and replicating brokers. These primitives are designed to involve a small local group of brokers in the pub/sub overlays so that the disruption during the execution of large-scale and dynamic changes can be controlled in a fined-grained manner. For the problem of degrading topology quality, automated planning systems are developed to find a sequence of adaptations that would cause minimal disruption to running services. Also, our primitives can be executed on demand to quickly fail-over a crashed broker or off-load congested brokers. In addition, these on-demand primitives can be used to form a group of dynamically replicated brokers that enforce a novel safety measure to prevent Byzantine brokers from sabotaging the pub/sub overlays. Our contributions are evaluated with systematic consideration of various trade-offs between functional and non-functional properties.

Techn. University, Diss., 2005--Berlin.

This thesis deals with data distribution service and is divided into three parts. The first part of this thesis focuses on theoretical knowledge of data distribution service. This section gradually introduces the model used in data distribution service, quality of service and the RTPS protocol, which was specially developed for the needs of data distribution service. The most comprehensive part of the theoretical part deals with security in data distribution service. At the end of the first part of this thesis, selected impelementations of data distribution service are introduced. The second part of this thesis deals with practical use of data distribution service, while OpenDDS implementation is chosen for the practical part. This part initially deals with the design of basic communication scenarios, followed by the design of complex communication scenarios. In the next chapters, within the practical part, the thesis is focused on verifyng security of the data distribution service selected implementation, design of security incidents and their subsequent simulation. Content of the final part of this thesis is the introduction of a tool designed for this thesis, which is used to implement selected types of attacks.