Integration flows are increasingly used to specify and execute data-intensive integration tasks between heterogeneous systems and applications. There are many different application areas such as near real-time ETL and data synchronization between operational systems. For the reasons of an increasing amount of data, highly distributed IT infrastructures, as well as high requirements for up-to-dateness of analytical query results and data consistency, many instances of integration flows are executed over time. Due to this high load, the performance of the central integration platform is crucial for an IT infrastructure. With the aim of throughput maximization, we propose the concept of multi-flow optimization (MFO). In this approach, messages are collected during a waiting time and executed in batches to optimize sequences of plan instances of a single integration flow. We introduce a horizontal (value-based) partitioning approach for message batch creation and show how to compute the optimal waiting time. This approach significantly reduces the total execution time of a message sequence and hence, it maximizes the throughput, while accepting moderate latency time.
| Identifer | oai:union.ndltd.org:DRESDEN/oai:qucosa:de:qucosa:82997 |
| Date | 19 January 2023 |
| Creators | Boehm, Matthias, Habich, Dirk, Lehner, Wolfgang |
| Publisher | Springer |
| Source Sets | Hochschulschriftenserver (HSSS) der SLUB Dresden |
| Language | English |
| Detected Language | English |
| Type | info:eu-repo/semantics/acceptedVersion, doc-type:conferenceObject, info:eu-repo/semantics/conferenceObject, doc-type:Text |
| Rights | info:eu-repo/semantics/openAccess |
| Relation | 978-3-642-19801-4, 978-3-642-19802-1, 10.1007/978-3-642-19802-1_3 |
Page generated in 0.0025 seconds