Back to search

Cornell University

Towards Accurate and Scalable Performance Modeling and Benchmarking of Cloud Applications

Abstract

dc:description.abstract

Cloud computing now powers a significant portion of global computation, supporting everything from latency-sensitive interactive services to artificial intelligence (AI) models. As these applications continue to shape the future of cloud infrastructures, understanding their behavior is critical. In this context, accurate and scalable performance modeling and benchmarking have become essential for optimizing system performance and guiding the development of cloud infrastructure. Agile, efficient, and precise modeling and benchmarking tools can provide invaluable insights for fleet design and optimizing efficiency. Within a cloud provider, these tools can support hardware and system optimization (e.g., GPU or ASIC accelerator design), performance characterization and analysis, design space exploration, and bug reproduction. Furthermore, they can be shared with external hardware vendors for early-stage performance testing, evaluation, and joint hardware/software co-design, all while requiring minimal infrastructure support and enabling a streamlined IP-sharing framework. The highly diverse and rapidly evolving landscape of cloud applications presents significant challenges in developing modeling and benchmarking tools that accurately capture performance characteristics. For instance, while open-source benchmarks have seen significant advancements, their update speed, application diversity, and complexity cannot keep pace with the constantly changing and varied applications in real cloud deployments. As a result, engineers and researchers have to manually adapt existing production or open-source workloads into forms suitable for benchmarking. This process requires substantial expertise and a deep understanding of the workloads, making it a non-trivial investment. Consequently, maintaining and updating these benchmarks to keep up with the fast pace of cloud application development incurs high costs. Therefore, there is a strong need for new methodologies that enable efficient and accurate modeling and benchmarking of cloud applications. This thesis presents novel solutions to address these challenges. We first introduce Ditto, an automated cloning framework for end-to-end interactive cloud services, including both monolithic applications and microservices. It begins by capturing the dependency graph across services using distributed tracing, then reconstructs the high-level control and data flow within each service. Finally, Ditto generates system calls and user-level assembly to capture both on-CPU and off-CPU behavior. This process is fully automated, allowing users to clone and benchmark services without needing expertise in their implementation. Our evaluation demonstrates that synthetic applications generated by Ditto respond to changes in input load, platform, resource allocation, and deployment configuration in the same way as the original workloads. Next, we present Mystique, an efficient and scalable framework for generating AI benchmarks. By leveraging execution traces directly captured from production workflows, Mystique generates benchmarks using a "replay-as-benchmark" approach. The trace records runtime information of a model at the operator level, and Mystique faithfully replays it to accurately reproduce the original performance. We demonstrate that our methodology generates AI benchmarks that closely mirror the original applications, both in terms of execution time and system-level metrics, while remaining easy to use and portable across platforms without the need for regeneration. We also highlight several use cases for Mystique, including early-stage platform evaluation, subtrace replay, and scaled-down performance testing. Finally, we discuss Lumos, a trace-driven performance modeling and estimation toolkit for large-scale training of large language models (LLMs). By leveraging built-in profiling tools from machine learning (ML) frameworks, Lumos constructs a comprehensive execution graph to capture the runtime behaviors of LLMs and build accurate performance models. It also provides users with a convenient way to explore various model and deployment configurations through graph manipulation and simulation, streamlining the exploration process. We evaluate Lumos using various GPT-3 model variants on a production-scale cluster and demonstrate that it accurately reproduces and predicts execution times and detailed performance characteristics across different models and configurations.

Degree

thesis:*
Name thesis:degree_name
Ph. D., Electrical and Computer Engineering
Level thesis:degree_level
Doctor of Philosophy
Discipline thesis:degree_discipline
Electrical and Computer Engineering
Grantor
Cornell University
Year dc:date.issued
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Liang, Mingyu
Chair dc:contributor.chair
  • Delimitrou, Christina
Committee members dc:contributor.committeemember
  • Suh, Gookwon Edward
  • Van Renesse, Robbert

Subjects

dc:subject × 5

Rights

dc:rights
Statement dc:rights
  • Attribution 4.0 International
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Bibid
16938449
Dc Identifier Other
ProQuest Submission ID: 14829
ProQuest Publication ID: 31839949

Chain of custody

source
Harvested from
Cornell University
Base URL
ecommons.cornell.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Liang, Mingyu. Towards Accurate and Scalable Performance Modeling and Benchmarking of Cloud Applications. Doctor of Philosophy thesis, Cornell University, 2025. https://hdl.handle.net/1813/117594