Back to search

University of British Columbia

Using Markov Chains for adaptive lifetime management of serverless applications

Abstract

dc:description

In serverless computing, a developer deploys their code without the burden of infrastructure provisioning. The cloud provider scales compute units dynamically to ensure the demand for the deployed application is met. A customer can then execute the deployed code. The latency incurred while a request waits to be assigned the necessary resources before execution is called a cold start. The cloud provider initializes resources before or beyond code execution to avert cold starts, leading to resource wastage. Lifetime management policies seek to balance cold starts with efficient resource management. To make these policies adaptive, we need a mechanism to predict the exact time when the policy needs to kick in. A forecaster makes predictions to create adaptive lifetime management policies. Forecasting involves utilizing historical application traffic and predicting future traffic. This ensures lifetime management policies kick in based on predicted application traffic. However, there is no consensus on which forecaster is the best regarding cold start aversion and resource efficiency. Moreover, no single forecaster is customized per application nor adapts to traffic pattern changes. To ensure customer satisfaction and cloud provider savings, it is important to maintain a low forecasting overhead and high accuracy. We propose a Markov Chain forecaster that combines features from serverless application data to improve accuracy. Features are statistical properties that enable forecasting. We use lightweight features to improve the accuracy of our forecaster. We further create an ensemble Markov Chain forecaster with a subset of features and explore a tradeoff between accuracy and overhead. We create multiple ensemble method forecasters using a lifetime management framework called FeMux, which multiplexes between different Markov Chains based on statistical property changes in the application traffic. Our forecaster's prediction accuracy is measured using simulation based on Azure serverless traces and a custom metric based on cold start time and memory wastage called Representative Unified Metric (RUM). We prototype our forecaster on Knative, a Kubernetes-based serverless platform, and reduce RUM by 45% compared to vanilla Knative, translating to a 37% cold start reduction with less than 2% additional memory. Our forecaster reduces RUM by 30% compared to prior work.

Degree

thesis:*
Name thesis:degree_name
Master of Applied Science - MASc
Level thesis:degree_level
master's
Discipline thesis:degree_discipline
Electrical and Computer Engineering
Grantor dc:publisher
University of British Columbia
Year dc:date
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Munshi, Nalin
Contributors dc:contributor
  • Shahrad, Mohammad

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-NoDerivatives 4.0 International
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/88820

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Munshi, Nalin. Using Markov Chains for adaptive lifetime management of serverless applications. master's thesis, University of British Columbia, 2024. http://hdl.handle.net/2429/88820