{"id":{"repo_id":"usf","oai_identifier":"oai:digitalcommons.usf.edu:etd-10517"},"canonical_url":"https://search.ndltd.org/etd/usf/oai:digitalcommons.usf.edu:etd-10517","repository":{"repo_id":"usf","name":"University of South Florida","base_url":"https://digitalcommons.usf.edu/do/oai/"},"display":{"title":"Psychometric Characteristics of Academic Language Discourse Analysis Tools","abstract":"Academic language plays a key role in students’ educational success, yet its development in primary grades is poorly understood and often neglected (Snow & Uccelli, 2008). Academic language skills may enhance overall academic performance if targeted early and intensively. However, current methods of assessment are not sufficient to understanding the construct well enough to develop evidence-based intervention strategies. This investigation examined the psychometric properties of two discourse analysis tools designed to directly measure students’ comprehension and production of academic language. Academic language samples (n = 7,887) from a previous cohort-design study (n = 1,040; Kindergarten through third grade participants) were scored using the Narrative Language Measure (NLM) Flowchart and the Expository Language Measure (ELM) Flowchart. A confirmatory factor analysis was used to test two-factor models for both flowcharts. The total scores and subscale scores of the NLM Flowchart demonstrated moderate to strong interrater reliability, moderate convergent validity, and approximate fit with the proposed model (generation χ²(46) = 743.85, p < .001, SRMR = .06, RMSEA = .08, CFI = .88, and TLI = .86; retell χ²(46) = 784.80, p < .001, SRMR = .05, RMSEA = .09, CFI = .91, and TLI = .90). One subscale (i.e., Narrative Structure) showed adequate internal consistency via Cronbach’s alpha. This study found mixed evidence of interrater reliability for the ELM Flowchart, with weak agreement on one subscale (i.e., Passage Structure) and substantial to strong agreement on the other (i.e., Language Complexity). The ELM Flowchart demonstrated moderate convergent validity, but neither subscale reached acceptable levels of internal consistency via Cronbach’s alpha. The appropriateness of using reflective indicator tools to evaluate constructs that may be better suited to a formative model is discussed. Other implications of the findings also are discussed.","abstract_html":"Academic language plays a key role in students’ educational success, yet its development in primary grades is poorly understood and often neglected (Snow &amp; Uccelli, 2008). Academic language skills may enhance overall academic performance if targeted early and intensively. However, current methods of assessment are not sufficient to understanding the construct well enough to develop evidence-based intervention strategies. This investigation examined the psychometric properties of two discourse analysis tools designed to directly measure students’ comprehension and production of academic language. Academic language samples (n = 7,887) from a previous cohort-design study (n = 1,040; Kindergarten through third grade participants) were scored using the Narrative Language Measure (NLM) Flowchart and the Expository Language Measure (ELM) Flowchart. A confirmatory factor analysis was used to test two-factor models for both flowcharts. The total scores and subscale scores of the NLM Flowchart demonstrated moderate to strong interrater reliability, moderate convergent validity, and approximate fit with the proposed model (generation χ²(46) = 743.85, p &lt; .001, SRMR = .06, RMSEA = .08, CFI = .88, and TLI = .86; retell χ²(46) = 784.80, p &lt; .001, SRMR = .05, RMSEA = .09, CFI = .91, and TLI = .90). One subscale (i.e., Narrative Structure) showed adequate internal consistency via Cronbach’s alpha. This study found mixed evidence of interrater reliability for the ELM Flowchart, with weak agreement on one subscale (i.e., Passage Structure) and substantial to strong agreement on the other (i.e., Language Complexity). The ELM Flowchart demonstrated moderate convergent validity, but neither subscale reached acceptable levels of internal consistency via Cronbach’s alpha. The appropriateness of using reflective indicator tools to evaluate constructs that may be better suited to a formative model is discussed. Other implications of the findings also are discussed.","abstract_has_math":false,"creators":["Claar, Courtney (Cici) Brianna"],"institution":"University of South Florida","degree_name":"Education Specialist (Ed.S.)","degree_level":"Ed. Specalist","degree_discipline":"Educational Measurement and Research","degree_department":null,"school":null,"contributors":["Trina Spencer, Ph.D., BCBA-D","Jose Castillo, Ph.D.","Yi-Jui Iva Chen, Ph.D."],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-06-28T07:00:00Z","date_published":"2022-06-28T07:00:00Z","updated_at":"2026-11-19T08:47:49Z","subjects":["academic language","language comprehension","language measures","language sampling","Arts and Humanities","Educational Assessment, Evaluation, and Research","Elementary Education and Teaching"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://digitalcommons.usf.edu/etd/9320","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://digitalcommons.usf.edu/etd/9320","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://digitalcommons.usf.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Adigitalcommons.usf.edu%3Aetd-10517","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Trina Spencer, Ph.D., BCBA-D","Jose Castillo, Ph.D.","Yi-Jui Iva Chen, Ph.D."]},{"key":"dc:creator","label":"Author","values":["Claar, Courtney (Cici) Brianna"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2022-06-28T07:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Educational Measurement and Research"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Ed. Specalist","*Ed.S."]},{"key":"thesis:degree_name","label":"Degree Name","values":["Education Specialist (Ed.S.)"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of South Florida"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["academic language","language comprehension","language measures","language sampling","Arts and Humanities","Educational Assessment, Evaluation, and Research","Elementary Education and Teaching"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.usf.edu/etd/9320"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Academic language plays a key role in students’ educational success, yet its development in primary grades is poorly understood and often neglected (Snow & Uccelli, 2008). Academic language skills may enhance overall academic performance if targeted early and intensively. However, current methods of assessment are not sufficient to understanding the construct well enough to develop evidence-based intervention strategies. This investigation examined the psychometric properties of two discourse analysis tools designed to directly measure students’ comprehension and production of academic language. Academic language samples (n = 7,887) from a previous cohort-design study (n = 1,040; Kindergarten through third grade participants) were scored using the Narrative Language Measure (NLM) Flowchart and the Expository Language Measure (ELM) Flowchart. A confirmatory factor analysis was used to test two-factor models for both flowcharts. The total scores and subscale scores of the NLM Flowchart demonstrated moderate to strong interrater reliability, moderate convergent validity, and approximate fit with the proposed model (generation χ²(46) = 743.85, p < .001, SRMR = .06, RMSEA = .08, CFI = .88, and TLI = .86; retell χ²(46) = 784.80, p < .001, SRMR = .05, RMSEA = .09, CFI = .91, and TLI = .90). One subscale (i.e., Narrative Structure) showed adequate internal consistency via Cronbach’s alpha. This study found mixed evidence of interrater reliability for the ELM Flowchart, with weak agreement on one subscale (i.e., Passage Structure) and substantial to strong agreement on the other (i.e., Language Complexity). The ELM Flowchart demonstrated moderate convergent validity, but neither subscale reached acceptable levels of internal consistency via Cronbach’s alpha. The appropriateness of using reflective indicator tools to evaluate constructs that may be better suited to a formative model is discussed. Other implications of the findings also are discussed."]},{"key":"dc:title","label":"Title","values":["Psychometric Characteristics of Academic Language Discourse Analysis Tools"]}]}],"canonical_facts":{"dc:contributor":["Trina Spencer, Ph.D., BCBA-D","Jose Castillo, Ph.D.","Yi-Jui Iva Chen, Ph.D."],"dc:creator":["Claar, Courtney (Cici) Brianna"],"dc:date":["2022-06-28T07:00:00Z"],"dc:description":["Academic language plays a key role in students’ educational success, yet its development in primary grades is poorly understood and often neglected (Snow & Uccelli, 2008). Academic language skills may enhance overall academic performance if targeted early and intensively. However, current methods of assessment are not sufficient to understanding the construct well enough to develop evidence-based intervention strategies. This investigation examined the psychometric properties of two discourse analysis tools designed to directly measure students’ comprehension and production of academic language. Academic language samples (n = 7,887) from a previous cohort-design study (n = 1,040; Kindergarten through third grade participants) were scored using the Narrative Language Measure (NLM) Flowchart and the Expository Language Measure (ELM) Flowchart. A confirmatory factor analysis was used to test two-factor models for both flowcharts. The total scores and subscale scores of the NLM Flowchart demonstrated moderate to strong interrater reliability, moderate convergent validity, and approximate fit with the proposed model (generation χ²(46) = 743.85, p < .001, SRMR = .06, RMSEA = .08, CFI = .88, and TLI = .86; retell χ²(46) = 784.80, p < .001, SRMR = .05, RMSEA = .09, CFI = .91, and TLI = .90). One subscale (i.e., Narrative Structure) showed adequate internal consistency via Cronbach’s alpha. This study found mixed evidence of interrater reliability for the ELM Flowchart, with weak agreement on one subscale (i.e., Passage Structure) and substantial to strong agreement on the other (i.e., Language Complexity). The ELM Flowchart demonstrated moderate convergent validity, but neither subscale reached acceptable levels of internal consistency via Cronbach’s alpha. The appropriateness of using reflective indicator tools to evaluate constructs that may be better suited to a formative model is discussed. Other implications of the findings also are discussed."],"dc:identifier":["https://digitalcommons.usf.edu/etd/9320"],"dc:subject":["academic language","language comprehension","language measures","language sampling","Arts and Humanities","Educational Assessment, Evaluation, and Research","Elementary Education and Teaching"],"dc:title":["Psychometric Characteristics of Academic Language Discourse Analysis Tools"],"thesis:degree_discipline":["Educational Measurement and Research"],"thesis:degree_level":["Ed. Specalist","*Ed.S."],"thesis:degree_name":["Education Specialist (Ed.S.)"],"thesis:institution_name":["University of South Florida"]},"updated_at":"2026-11-19T08:47:49Z"}