Back to search

University of Sheffield

Speaking to the Edge: Conversational Control of Smart Home IoT with LLM AI and Deterministic Gatekeeping

Abstract

dc:description.abstract

Smart homes inject additional control layers into domestic objects, often using sensor actuator-microcontroller devices which, when networked, have come to be known as the Internet of Things (IoT). Voice control has increasingly been applied as a user interface here, with systems like Amazon Echo streaming spoken command audio out to cloud-based processing for analysis. Control statements are returned to effect home device state changes, with a spoken output generated and delivered over a smart speaker or similar. Previous versions of this technology have suffered the brittleness problem, where "expert" or "knowledge-based" systems perform acceptably in controlled conditions but fail rapidly given command diversity. The advent of Large Language Models (LLMs) and transformer architectures in speech and language processing has seen flexible, powerful reasoning capabilities become widely available, and brought about the possibility of robust conversational control of smart homes (and other environments), constituting a new wave of Artificial Intelligence (AI) technology. The current state of the art, then, is conversational control via cloud-based services that use the new AI. For smart home control, there are two significant problems with this: the negative security and privacy implications of cloud-based data processing from the home; the unreliability of remote services given older domestic systems that set a high standard in this respect. This motivates the question: what happens if the cloud is replaced with locally-computed conversational control? This thesis addresses that question via six contributions: (1) a taxonomic classification of smart home technologies; (2) a dataset of smart home commands, capturing diverse interaction scenarios; (3) a comparative ASR evaluation under smart home conditions; (4) an analysis of local LLM reasoning for smart home control; (5) a deterministic safety mechanism, validating LLM-generated control outputs; (6) an integrated and evaluated locally-operated smart home assistant, demonstrating its feasibility as a flexible, privacy preserving alternative to cloud-based assistants.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of Sheffield
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Hewitt, Mary
Advisor dc:contributor.advisor
  • Cunningham, Hamish
ORCID person:identifier.orcid

Rights

Language dc:language.dcterms_iso639_2
eng

Identifiers

dc:identifier.*
Dc Identifier Grantnumber
EP/S023062/1

Chain of custody

source
Harvested from
White Rose University Consortium
Base URL
etheses.whiterose.ac.uk/cgi/oai2
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
related terms
citation

Hewitt, Mary. Speaking to the Edge: Conversational Control of Smart Home IoT with LLM AI and Deterministic Gatekeeping. doctoral thesis, University of Sheffield, 2026.