{"id":{"repo_id":"utahstate","oai_identifier":"oai:digitalcommons.usu.edu:etd-1779"},"canonical_url":"https://search.ndltd.org/etd/utahstate/oai:digitalcommons.usu.edu:etd-1779","repository":{"repo_id":"utahstate","name":"Utah State University","base_url":"https://digitalcommons.usu.edu/do/oai/"},"display":{"title":"Psychometric Properties of Postsecondary Students' Course Evaluations","abstract":"Several experts in the area of postsecondary student evaluations of courses have concluded that they are stable or reliable measures as well as being measures that provide ways of making valid inferences regarding teacher effectiveness. Often these experts have offered these conclusions without supporting evidence. Surprisingly, a thorough review of the literature revealed very few reported test-retest reliability studies of course evaluations and the results from these studies are contradictory. In the area of validity, the conclusions offered by scholars who conducted meta-analyses of mutlisection course studies are inconsistent. This leads to the following two research questions: 1. What is the test-retest reliability over a 3-week period of the course evaluation currently employed at Utah State University? 2. Can results of the course evaluation employed at Utah State University be used to make valid inferences about a teacher's effectiveness? Two parts of a study were conducted to answer these questions. First, a test-retest reliability part was conducted with students from courses at Utah State University, employing a 3-week time lapse between administrations of the course evaluations. Second, a multisection course validity part was conducted using existing student ratings data and final examination scores for 100 sections of MATH 1010 over a 5-year period. Correlational analyses were conducted on the resulting data from both studies. Test-retest reliability coefficients ranging from 0.64 to 0.94 were found. In the second study, the correlation coefficients from the validity study ranged from -0.39 to 0.71, with a mean coefficient of 0.14 and 0.11 for final examination score by instructor rating and final examination score by course rating, respectively. Results from both parts of the study suggest that the course evaluation used at USU is not reliable and that results of the course evaluation do not provide information that can be used to make valid inferences regarding teacher effectiveness.","abstract_html":"Several experts in the area of postsecondary student evaluations of courses have concluded that they are stable or reliable measures as well as being measures that provide ways of making valid inferences regarding teacher effectiveness. Often these experts have offered these conclusions without supporting evidence. Surprisingly, a thorough review of the literature revealed very few reported test-retest reliability studies of course evaluations and the results from these studies are contradictory. In the area of validity, the conclusions offered by scholars who conducted meta-analyses of mutlisection course studies are inconsistent. This leads to the following two research questions: 1. What is the test-retest reliability over a 3-week period of the course evaluation currently employed at Utah State University? 2. Can results of the course evaluation employed at Utah State University be used to make valid inferences about a teacher&#x27;s effectiveness? Two parts of a study were conducted to answer these questions. First, a test-retest reliability part was conducted with students from courses at Utah State University, employing a 3-week time lapse between administrations of the course evaluations. Second, a multisection course validity part was conducted using existing student ratings data and final examination scores for 100 sections of MATH 1010 over a 5-year period. Correlational analyses were conducted on the resulting data from both studies. Test-retest reliability coefficients ranging from 0.64 to 0.94 were found. In the second study, the correlation coefficients from the validity study ranged from -0.39 to 0.71, with a mean coefficient of 0.14 and 0.11 for final examination score by instructor rating and final examination score by course rating, respectively. Results from both parts of the study suggest that the course evaluation used at USU is not reliable and that results of the course evaluation do not provide information that can be used to make valid inferences regarding teacher effectiveness.","abstract_has_math":false,"creators":["Drysdale, Michael J."],"institution":"Hosted by Utah State University Libraries","degree_name":"Doctor of Philosophy (PhD)","degree_level":"Dissertation","degree_discipline":"Psychology","degree_department":null,"school":null,"contributors":["Karl R. White","J. Nicholls Eastmond","James T. Dorward"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":null,"date_issued":"","date_published":null,"updated_at":"2026-11-19T08:54:13Z","subjects":["course evaluation","reliability","student ratings","validity","Educational Assessment, Evaluation, and Research","Educational Psychology","Statistics and Probability"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[{"key":"dc:identifier.doi","label":"DOI","values":["https://doi.org/10.26076/39a4-cbfe"],"render_values":[{"text":"https://doi.org/10.26076/39a4-cbfe","href":"https://doi.org/10.26076/39a4-cbfe","code":true}]}],"person_identifier_entries":[]},"links":{"outbound_url":"https://digitalcommons.usu.edu/etd/783","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://digitalcommons.usu.edu/etd/783","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://digitalcommons.usu.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Adigitalcommons.usu.edu%3Aetd-1779","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Karl R. White","J. Nicholls Eastmond","James T. Dorward"]},{"key":"dc:creator","label":"Author","values":["Drysdale, Michael J."]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.available","label":"Dc Date Available","values":["2020-04-14T07:00:00Z"]},{"key":"dc:date.created","label":"Dc Date Created","values":["2010-12-01T08:00:00Z"]},{"key":"dc:publisher","label":"Institution","values":["Hosted by Utah State University Libraries"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Psychology"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctor of Philosophy (PhD)"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["course evaluation","reliability","student ratings","validity","Educational Assessment, Evaluation, and Research","Educational Psychology","Statistics and Probability"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.usu.edu/etd/783"]},{"key":"dc:identifier.doi","label":"DOI","values":["https://doi.org/10.26076/39a4-cbfe"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Several experts in the area of postsecondary student evaluations of courses have concluded that they are stable or reliable measures as well as being measures that provide ways of making valid inferences regarding teacher effectiveness. Often these experts have offered these conclusions without supporting evidence. Surprisingly, a thorough review of the literature revealed very few reported test-retest reliability studies of course evaluations and the results from these studies are contradictory. In the area of validity, the conclusions offered by scholars who conducted meta-analyses of mutlisection course studies are inconsistent. This leads to the following two research questions: 1. What is the test-retest reliability over a 3-week period of the course evaluation currently employed at Utah State University? 2. Can results of the course evaluation employed at Utah State University be used to make valid inferences about a teacher's effectiveness? Two parts of a study were conducted to answer these questions. First, a test-retest reliability part was conducted with students from courses at Utah State University, employing a 3-week time lapse between administrations of the course evaluations. Second, a multisection course validity part was conducted using existing student ratings data and final examination scores for 100 sections of MATH 1010 over a 5-year period. Correlational analyses were conducted on the resulting data from both studies. Test-retest reliability coefficients ranging from 0.64 to 0.94 were found. In the second study, the correlation coefficients from the validity study ranged from -0.39 to 0.71, with a mean coefficient of 0.14 and 0.11 for final examination score by instructor rating and final examination score by course rating, respectively. Results from both parts of the study suggest that the course evaluation used at USU is not reliable and that results of the course evaluation do not provide information that can be used to make valid inferences regarding teacher effectiveness."]},{"key":"dc:description.note","label":"Dc Description Note","values":["<p>This work made publicly available electronically on November 29, 2010.</p>"]},{"key":"dc:title","label":"Title","values":["Psychometric Properties of Postsecondary Students' Course Evaluations"]}]}],"canonical_facts":{"dc:contributor":["Karl R. White","J. Nicholls Eastmond","James T. Dorward"],"dc:creator":["Drysdale, Michael J."],"dc:date.available":["2020-04-14T07:00:00Z"],"dc:date.created":["2010-12-01T08:00:00Z"],"dc:description.abstract":["Several experts in the area of postsecondary student evaluations of courses have concluded that they are stable or reliable measures as well as being measures that provide ways of making valid inferences regarding teacher effectiveness. Often these experts have offered these conclusions without supporting evidence. Surprisingly, a thorough review of the literature revealed very few reported test-retest reliability studies of course evaluations and the results from these studies are contradictory. In the area of validity, the conclusions offered by scholars who conducted meta-analyses of mutlisection course studies are inconsistent. This leads to the following two research questions: 1. What is the test-retest reliability over a 3-week period of the course evaluation currently employed at Utah State University? 2. Can results of the course evaluation employed at Utah State University be used to make valid inferences about a teacher's effectiveness? Two parts of a study were conducted to answer these questions. First, a test-retest reliability part was conducted with students from courses at Utah State University, employing a 3-week time lapse between administrations of the course evaluations. Second, a multisection course validity part was conducted using existing student ratings data and final examination scores for 100 sections of MATH 1010 over a 5-year period. Correlational analyses were conducted on the resulting data from both studies. Test-retest reliability coefficients ranging from 0.64 to 0.94 were found. In the second study, the correlation coefficients from the validity study ranged from -0.39 to 0.71, with a mean coefficient of 0.14 and 0.11 for final examination score by instructor rating and final examination score by course rating, respectively. Results from both parts of the study suggest that the course evaluation used at USU is not reliable and that results of the course evaluation do not provide information that can be used to make valid inferences regarding teacher effectiveness."],"dc:description.note":["<p>This work made publicly available electronically on November 29, 2010.</p>"],"dc:identifier":["https://digitalcommons.usu.edu/etd/783"],"dc:identifier.doi":["https://doi.org/10.26076/39a4-cbfe"],"dc:publisher":["Hosted by Utah State University Libraries"],"dc:subject":["course evaluation","reliability","student ratings","validity","Educational Assessment, Evaluation, and Research","Educational Psychology","Statistics and Probability"],"dc:title":["Psychometric Properties of Postsecondary Students' Course Evaluations"],"thesis:degree_discipline":["Psychology"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Doctor of Philosophy (PhD)"]},"updated_at":"2026-11-19T08:54:13Z"}