Spelling suggestions: "subject:"horizontal partitioning"" "subject:"orizontal partitioning""
1 |
Conception physique statique et dynamique des entrepôts de données / Static and Dynamic Data Warehouses DesignBouchakri, Rima 17 September 2015 (has links)
Les entrepôts de données permettent le stockage et la consolidation, en une seule localité, d'une quantité gigantesque d'information pour être interrogée par des requêtes décisionnelles complexes dites requêtes de jointures en étoiles. Afin d'optimiser ses requêtes, plusieurs travaux emploient des techniques d'optimisations comme les index de jointure binaires et la fragmentation horizontale durant la phase de conception physique d'un entrepôt de données. Cependant, ces travaux proposent des algorithmes statiques qui sélectionnent ces techniques de manière isolée et s'intéressent à l'optimisation d'un seul objectif à savoir les performances des requêtes. Notre principale contribution dans cette thèse est de proposer une nouvelle vision de sélection des techniques d'optimisation. Notre première contribution est une sélection incrémentale qui permet de mettre à jour de manière continuelle le schéma d'optimisation implémenté sur l'ED, ce qui assure l'optimisation continuelle des requêtes décisionnelles. Notre seconde contribution est une sélection incrémentale jointe qui combine deux techniques d'optimisation pour couvrir l'optimisation d'un maximum de requêtes et respecter au mieux les contraintes d'optimisation liées à chacune de ces techniques. A l'issu de ces propositions, nous avons constaté que la sélection incrémentale engendre un coût de maintenance de l'ED. Ainsi, notre troisième proposition est une formulation et r!:solution du problème multi-objectif de sélection des techniques d'optimisation où il faut optimiser deux objectifs: la performance des requêtes et le coût de maintenance de l'ED. / Data Warehouses store into a single location a huge amount of data. They are interrogated by complex decisional queries called star join queries. To optimize such queries, several works propose algorithms for selecting optimization techniques such as Binary Join Indexes and Horizontal Partitioning during the DW physical design. However, these works propose static algorithms, select optimization techniques in and isolated way and focus on optimizing a single objective which is the query performance. Our main contribution in this thesis is to propose a new vision of optimization techniques selection. Our first contribution is an incremental selection that updates continuously the optimization scheme implemented on the DW, to ensure the continual optimization of queries. To deal with queries complexity increase, our second contribution is a join incremental selection of two optimization techniques which covers the optimization of a maximum number or queries and respects the optimization constraints. Finally, we note that the incremental selection generates a maintenance cost to update the optimization schemes. Thus, our third prop05ilion is to formulate and resolve a multi-objective selection problem or optimization techniques where we have two objectives to optimize : queries performance and maintenance cost of the DW.
|
2 |
Multi-flow Optimization via Horizontal Message Queue PartitioningBoehm, Matthias, Habich, Dirk, Lehner, Wolfgang 19 January 2023 (has links)
Integration flows are increasingly used to specify and execute data-intensive integration tasks between heterogeneous systems and applications. There are many different application areas such as near real-time ETL and data synchronization between operational systems. For the reasons of an increasing amount of data, highly distributed IT infrastructures, as well as high requirements for up-to-dateness of analytical query results and data consistency, many instances of integration flows are executed over time. Due to this high load, the performance of the central integration platform is crucial for an IT infrastructure. With the aim of throughput maximization, we propose the concept of multi-flow optimization (MFO). In this approach, messages are collected during a waiting time and executed in batches to optimize sequences of plan instances of a single integration flow. We introduce a horizontal (value-based) partitioning approach for message batch creation and show how to compute the optimal waiting time. This approach significantly reduces the total execution time of a message sequence and hence, it maximizes the throughput, while accepting moderate latency time.
|
Page generated in 0.1358 seconds