Back to search

Virginia Tech

LLM-Guided Run-Time Parameter Optimization for Energy-Efficient Model Inference

Abstract

dc:description.abstract

The scale of Large Language Models (LLMs) has been increasing over time, and is predicted to continue to increase as LLMs become an integral part of many real world workflows. However, LLMs consume a tremendous amount of energy, which becomes a large concern in the scale of the demand for these tools. Inference engines and serving systems, like vLLM and PyTorch, are used to perform efficient LLM inference. These tools include runtime parameters that have the potential to improve the energy efficiency of inference. The task is choosing the values for those parameters that actually result in lower energy consumption. However, these parameters have complex relationships and trade offs that make this choice difficult and sometimes non intuitive. This choice can also require deep knowledge of the application or inefficient traditional optimization methods. In this work, we created a human-in-the-loop flow with LLM assisted runtime parameter optimization in order to solve this issue. With human created, specific feedback prompting methods, chat based LLMs can iteratively find energy efficient inference parameters faster than traditional search methods. We evaluated LLM as an optimization method on a text prompt workload with vLLM as the inference engine and on an image prompt workload with PyTorch as the inference engine. Across these two systems, we compared the performance of LLM optimization when prompting with baseline prompts that only gave the minimum amount of information and enhanced prompts which included different prompting strategies with the goal of improving performance. LLM optimization that uses the enhnaced prompt strategy, was compared to a more traditional optimization method: Sobol sampling. The enhanced prompting strategy outperforms the baseline across all experiment runs. It achieves an average of 45% reduction in energy-per-token for vLLM inference and an average of 90.7% reduction for PyTorch multimodal inference relative to the default configurations. Compared to Sobol sampling, the LLM guided approach reaches lower energy-per-token values more efficiently in the vLLM inference and achives lower energy-per-image values in the PyTorch setting. Sobol sampling also consistently failed to avoid out of memory configurations in the PyTorch configuration. These results suggest that LLMs may serve as effective optimization agents for reducing the energy consumption of AI inference, without requiring expertise or exhaustive search.

Degree

thesis:*
Name thesis:degree_name
Master of Science
Level thesis:degree_level
masters
Discipline thesis:degree_discipline
Computer Science & Applications
Department dc:contributor.department
Computer Science and#38; Applications
Grantor dc:publisher
Virginia Tech
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Crumpacker, Mary Katelyn
Chair dc:contributor.committeechair
  • Nikolopoulos, Dimitrios S.
Committee members dc:contributor.committeemember
  • Ellis, Margaret O.'Neil
  • Back, Godmar Volker

Subjects

dc:subject × 5

Rights

dc:rights
Statement dc:rights
  • In Copyright
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Other
vt_gsexam:46961

Chain of custody

source
Harvested from
Virginia Tech
Base URL
vtechworks.lib.vt.edu/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Crumpacker, Mary Katelyn. LLM-Guided Run-Time Parameter Optimization for Energy-Efficient Model Inference. masters thesis, Virginia Tech, 2026. https://hdl.handle.net/10919/143610