Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 13 of 13 for “"inference serving"”.
-
Flashpoint: A Low-latency Serverless Platform for Deep Learning Inference Serving
… (DL) have led to high demand for executing inferences in interactive services such as ChatGPT and GitHub Copilot. However, these interactive services require low-latency inferences, which can only be met with GPUs and result in exorbitant operating costs. For instance, ChatGPT reportedly …
-
A Joint Model Provisioning and Request Dispatch Solution for Mobile Inference Serving at the Edge
… a growing number of mobile clients rely onML inference for making time-sensitive and safety-critical decisions. Therefore, the demand for high-quality and low-latency inference services at the network edge has become the key to the modern intelligent society. This thesis proposes a novel …
-
High-Performance Computational Kernels for Algorithm-System Co-Design in Machine Learning: Enabling Efficient GCN Training and LLM Inference Serving.
… of reliability in large-scale distributed inference. Multi-GPU deployments, while necessary for training and serving massive models, remain vulnerable to hardware failures that can waste compute cycles and disrupt real-time applications. Traditional checkpointing is too slow and …
-
Efficient AI Stack: Deployment-Aware Neural Architecture Search and Serving of Deep Neural Networks
… unpredictable and bursty request arrival rates. Serving models under such conditions requires these systems to strike a careful balance between the latency (R1) and accuracy (R2) requirements of the application and the overall efficiency of utilization of scarce resources (R3). To efficiently …
-
Profiling and Visualizing CPU-Based LLM Inference in llama.cpp
… is therefore essential for improving local LLM inference, but is difficult without structured measurement and analysis. This bachelor’s thesis presents a profiling and visualization tool for characterizing LLM inference on CPUs using an inference serving framework called llama.cpp. The tool …
-
Applications of Predictive Analytics for Machine Learning as a Service
… expansion of cloud-based ML applications and inference services, combined with increasing demands for computational resources, data privacy, and system efficiency, highlights the necessity for scalable, secure, and optimized cloud resource management solutions. This dissertation explores …
-
Data-aware optimizations for efficient analytics over geo-distributed data
… and reduce query latency. We also show that inference serving systems can produce higher-accuracy inferences by dynamically selecting a model based on the data. By considering the properties of geo-distributed data sources, systems can optimally navigate trade-offs between resource usage, …
-
Towards SLO-Compliant and Privacy-Aware Edge Systems
… full internal visibility for white-box ML inference. It is a Pareto-guided scheduler that jointly optimizes model variant selection, adaptive batching, and replica placement across all stages to maintain strict p99 latency SLOs under volatile load. Together, these contributions form a …
-
Offloading and managing change in cloud-assisted machine learning systems
… systems has led to rapidly growing demand for inference, making the design of efficient and scalable systems increasingly important. This dissertation investigates resource allocation and decision-making problems that arise in the operation and maintenance of large-scale deployments of machine …
-
Efficient inference and adaptation in deep learning : from edge systems to large language models
… problem.This thesis investigates efficient inference, training, and adaptation in DL across the model lifecycle from four complementary perspectives: (1) enabling lightweight and energy-efficient inference of DL models on edge devices and distributed Internet of Things (IoT) systems; (2) …
-
Scalable and Efficient Machine Learning as a Service
… machine learning model design, model training, inference serving and provides optimized executions of machine learning tasks in an automated, scalable, and efficient manner.This dissertation proposes three novel approaches for MLaaS, namely SimiGrad, DistQuant, and RRL, to improve the scale and …
-
Causal Inference in the Presence of Heterogeneous Treatment Effects
Causal inference has been widely accepted as a statistical tool in various areas for demystifying causality from data. Treatment effect heterogeneity is a common issue in causal inference which refers to variation in the causal effect of a treatment across different subgroups or individuals within …
-
Cost-Latency Benchmarking for Time-Series Forecasting - Hardware-Aware Evaluation of Deep Learning Architectures for Financial Planning
… at the product level but raises the cost of serving forecasts. The hardware for these workloads is often chosen through heuristics rather than measurement, and existing serving research has focused on computer vision and language rather than time-series forecasting. This thesis develops a …