Back to search

University of North Texas

A Quasi-Experimental Study of Inter-rater Reliability When Awarding Exceptional Quality Points on the Texas Teacher Appraisal Instrument

Abstract

dc:description

This study investigates the inter-rater reliability of appraisers who award exceptional quality points on the Texas Teacher Appraisal Instrument. Inter-rater reliability was measured when appraisers scored exceptional quality points after viewing a videotaped lesson. Comparisons were made between appraisers when grouped according to elementary or secondary certification, sex, years of administrative experience, and type of training. A total of 707 subjects from 56 school districts participated in the study. Five research hypotheses were formulated with the .05 level of significance for acceptance. All hypotheses were tested by correlation of coefficients, multiple response procedures, frequencies, and percentages. The data measuring inter-rater reliability of the appraisers in training imply that there is very little reliability in the awarding of exceptional quality points on the Texas Teacher Appraisal Instrument. The findings of this study are that certification, sex, administrative experience, and type of training made no significant differences when scoring the instrument. Therefore, it is concluded that the scoring of exceptional quality points is a subjective, professional judgment made by each appraiser when observing a teacher. Since no significant reliability was found, the scoring of exceptional quality points cannot be supported as a reliable means of determining the quality of teaching in Texas schools. Generally, elementary certified appraisers awarded fewer exceptional quality points than secondary appraisers, males awarded slightly more points than females, appraisers indicated no noticeable trend because of years of administrative experience, and less experienced appraisers had the tendency to award more points than experienced appraisers. Therefore, inter-rater reliability in awarding exceptional quality points cannot be expected on a consistent basis. Each appraiser, regardless of certification, sex, years of administrative experience or training, will use his or her own professional judgment when scoring the instrument.

Degree

thesis:*
Grantor dc:publisher
University of North Texas
Year dc:date
1989

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Dobbs, Louann
Contributors dc:contributor
  • Horvat, John J.
  • Campbell, Lloyd P.
  • Halstead, Francis E., 1930-
  • Brookshire, William K.

Subjects

dc:subject × 7

Rights

dc:rights
Statement dc:rights
  • Public
  • Dobbs, Louann
  • Copyright
  • Copyright is held by the author, unless otherwise noted. All rights reserved.
Language dc:language
English

Identifiers

dc:identifier.*
Identifier
local-cont-no: 1002714322-Dobbs
call-no: 379 N81d no.3018
untcat: b1450894
ark: ark:/67531/metadc331185

Chain of custody

source
Harvested from
University of North Texas
Base URL
digital.library.unt.edu/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Dobbs, Louann. A Quasi-Experimental Study of Inter-rater Reliability When Awarding Exceptional Quality Points on the Texas Teacher Appraisal Instrument. University of North Texas, 1989. https://digital.library.unt.edu/ark:/67531/metadc331185/