Back to search

Hosted by Utah State University Libraries

Removing Barriers to Language Sample Analysis: Computer-Automated Microstructure Scoring (CAMS)

Abstract

dc:description.abstract

Language sample analysis (LSA) is an important practice for providing a sensitive and accurate measure of a child’s language abilities. Several research-validated progress-monitoring tools are currently available that are designed to measure language quality through narrative samples in school-age children. While these tools provide clinically important information about a child’s language abilities, they can be time-consuming to code, and challenging to code reliably. In recent years there has been a surge in the use of automated essay scoring (AES) systems for the scoring of high-stakes written assessments, but few programs has been designed to automate clinical assessments. Narrative microstructure, an important indicator of language quality, lends itself to computer automation due to its relatively objective and straightforward assessment. The purpose of the current study was to test the feasibility of developing a series of computer-based rulesets, collectively referred to as CAMS, for automatically scoring narrative microstructure in LSA. CAMS was designed to individually score six microstructure elements related to conjunctions, meta-cognitive and linguistic verbs and sentence elaboration. The accuracy and interrater reliability of CAMS with hand-scores (n = 414) was compared against goldstandard expert scores (n = 50) based on percent overlap and quadratic weighted kappa (QWK). Results indicated that QWK between CAMS and expert-scores was higher than that of hand-scorers on four of the six elements. CAMS also met the literature-based threshold for strong interrater reliability (k > 0.60) for all six elements. CAMS shows promise as a means for automating the scoring of microstructure in LSA and narrative samples.

Degree

thesis:*
Name thesis:degree_name
Master of Science (MS)
Level thesis:degree_level
Thesis
Discipline thesis:degree_discipline
Communicative Disorders and Deaf Education
Grantor dc:publisher
Hosted by Utah State University Libraries

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Fox, Carly
Contributors dc:contributor
  • Sandra Gillam
  • Ron Gillam
  • Sarah Schwartz

Subjects

dc:subject × 6

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
Utah State University
Base URL
digitalcommons.usu.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Fox, Carly. Removing Barriers to Language Sample Analysis: Computer-Automated Microstructure Scoring (CAMS). Thesis thesis, Hosted by Utah State University Libraries, https://digitalcommons.usu.edu/etd/8023