Georgia Institute of Technology
An Analytical Evaluation of Large Language Models for Verifying Structural Damage and Failure Causality Reports
Abstract
dc:description.abstractPost‑disaster structural damage and failure causality reports are often fragmented, inconsistent, and difficult to verify, creating challenges for engineers, emergency responders, and policymakers who must act promptly. While artificial intelligence has shown success in sensor‑ and vision‑based structural health monitoring, the applicability of large language models (LLMs) for verifying post‑disaster structural information remains underexplored. This research evaluates the performance of six commercially available LLM platforms in reporting structural damage, failure causality, casualties, economic impacts, and maintenance histories across five documented structural failure case studies in the United States. Each platform was queried using a standardized set of investigative questions, and responses were evaluated against authoritative forensic investigation reports using a five‑metric scoring framework assessing accuracy, completeness, reliability, recency, and depth. The analysis was supplemented by a structured human evaluation survey examining user perceptions of response quality. Results indicate that LLM performance is strongly dependent on the availability and quality of publicly documented forensic records, with platforms possessing broader access to indexed information (such as Grok and Gemini) consistently outperforming others. Enhanced reasoning modes did not yield measurable improvements in factual accuracy. The study concludes that general‑purpose LLMs are suitable as supplementary orientation tools but are not reliable as standalone verification instruments, underscoring the need for domain‑specific structural engineering language models trained on forensic documentation.
Degree
thesis:*- Name thesis:degree_name
- Civil Engineering, MS
- Department thesis:degree_department
- Civil and Environmental Engineering
- Grantor
- Georgia Institute of Technology
- Year dc:date.issued
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Oluwayemi, Ayojesubolakunmi
- Advisor dc:contributor.advisor
-
- Smyl, Danny
- Committee members dc:contributor.committeemember
-
- Kumar, Aditya
- Sherman, Ryan
Rights
- Language dc:language.iso
- English
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/1853/81626