Event Stream Processing (ESP) is a well-established approach for low-latency data processing enabling users to quickly react to relevant situations in soft real-time. In order to cope with the sheer amount of data being generated each day and to cope with fluctuating workloads originating from data sources such as Twitter and Facebook, such systems must be highly scalable and elastic. Hence, ESP systems are typically long running applications deployed on several hundreds of nodes in either dedicated data-centers or cloud environments such as Amazon EC2. In such environments, nodes are likely to fail due to software aging, process or hardware errors whereas the unbounded stream of data asks for continuous processing. In order to cope with node failures, several fault tolerance approaches have been proposed in literature. Active replication and rollback recovery-based on checkpointing and in-memory logging (upstream backup) are two commonly used approaches in order to cope with such failures in the context of ESP systems. However, these approaches suffer either from a high resource footprint, low throughput or unresponsiveness due to long recovery times. Moreover, in order to recover applications in a precise manner using exactly once semantics, the use of deterministic execution is required which adds another layer of complexity and overhead. The goal of this thesis is to lower the overhead for fault tolerance in ESP systems. We first present StreamMine3G, our ESP system we built entirely from scratch in order to study and evaluate novel approaches for fault tolerance and elasticity. We then present an approach to reduce the overhead of deterministic execution by using a weak, epoch-based rather than strict ordering scheme for commutative and tumbling windowed operators that allows applications to recover precisely using active or passive replication. Since most applications are running in cloud environments nowadays, we furthermore propose an approach to increase the system availability by efficiently utilizing spare but paid resources for fault tolerance. Finally, in order to free users from the burden of choosing the correct fault tolerance scheme for their applications that guarantees the desired recovery time while still saving resources, we present a controller-based approach that adapts fault tolerance at runtime. We furthermore showcase the applicability of our StreamMine3G approach using real world applications and examples.

Durant la dernière décennie, les architectures orientées services (SOA) d'une part et la gestion des processus business (BPM) d'autre part ont beaucoup évolué et semblent maintenant en train de converger vers un but commun qui est de permettre à des organisations complètement hétérogènes de partager de manière flexible leurs ressources dans le but d'atteindre des objectifs communs, et ce, à travers des schémas de collaboration avancée. Ces derniers permettent de spécifier l'interconnexion des processus métier de différentes organisations. La nature dynamique et la complexité de ces processus posent des défis majeurs quant à leur bonne exécution. Certes, les langages de description de chorégraphie aident à réduire cette complexité en fournissant des moyens pour décrire des systèmes complexes à un niveau abstrait. Toutefois, rien ne garantit que des situations erronées ne se produisent pas suite, par exemple, à des interactions "mal" spécifiées ou encore des comportements malhonnêtes d'un des partenaires. Dans ce manuscrit, nous proposons une approche décentralisée qui permet la supervision de chorégraphies au moment de leur exécution et la détection instantanée de violations de séquences d'interaction. Nous définissons un modèle de propagation hiérarchique pour l'échange de notifications externes entre les partenaires. Notre approche permet une génération optimisée de requêtes de supervision dans un environnement événementiel, et ce, d'une façon automatique et à partir de tout modèle de chorégraphie / Cross-organizational service-based processes are increasingly adopted by different companies when they can not achieve goals on their own. The dynamic nature of these processes poses various challenges to their successful execution. In order to guarantee that all involved partners are informed about errors that may happen in the collaboration, it is necessary to monitor the execution process by continuously observing and checking message exchanges during runtime. This allows a global process tracking and evaluation of process metrics. Complex event processing can address this concern by analyzing and evaluating message exchange events, to the aim of checking if the actual behavior of the interacting entities effectively adheres to the modeled business constraints. In this thesis, we present an approach for decentralized monitoring of cross-organizational choreographies. We define a hierarchical propagation model for exchanging external notifications between the collaborating parties. We also propose a runtime event-based approach to deal with the problem of monitoring conformance of interaction sequences. Our approach allows for an automatic and optimized generation of rules. After parsing the choreography graph into a hierarchy of canonical blocks, tagging each event by its block ascendancy, an optimized set of monitoring queries is generated. We evaluate the concepts based on a scenario showing how much the number of queries can be significantly reduced