Back to search

Clemson University

Implementation of a Local LLM Serving System for Agentic AI

Abstract

dc:description.abstract

This thesis presents a system that allows large AI models to run directly on personal devices instead of relying on cloud servers. Recent advances in artificial intelligence, especially large language models (LLMs), have made it possible to build powerful applications such as chatbots, coding assistants, and intelligent agents. However, most of these systems run in the cloud, which raises concerns about privacy, latency, and cost. To address these issues, this work develops a local AI serving system that runs efficiently on a specialized hardware component called a Neural Processing Unit (NPU). The system provides a unified interface that supports multiple types of tasks, including text generation, image under- standing, speech recognition, and embedding-based search. It also supports advanced features such as streaming responses and tool calling, which are essential for building modern AI agents. The system is designed to be compatible with widely used APIs, allowing existing appli- cations to use it without modification. Experimental results show that the system works correctly across different tasks and improves efficiency through techniques such as prompt caching. Overall, this work demonstrates that it is possible to run advanced AI systems locally in a practical and efficient way, enabling faster, more private, and more flexible AI applications.

Degree

thesis:*
Name thesis:degree_name
Master of Science (MS)
Level thesis:degree_level
Thesis
Discipline thesis:degree_discipline
Electrical and Computer Engineering (Holcomb Dept. of)

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Ni, Zhiheng
Contributors dc:contributor
  • Tao Wei
  • Xiaoyong Yuan
  • Fatemeh Afghah

Subjects

dc:subject × 1

Identifiers

dc:identifier.*
Repository record dc:identifier
https://open.clemson.edu/all_theses/4796

Chain of custody

source
Harvested from
Clemson University
Base URL
open.clemson.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Ni, Zhiheng. Implementation of a Local LLM Serving System for Agentic AI. Thesis thesis, https://open.clemson.edu/all_theses/4796