{"id":{"repo_id":"eiu","oai_identifier":"oai:thekeep.eiu.edu:theses-5987"},"canonical_url":"https://search.ndltd.org/etd/eiu/oai:thekeep.eiu.edu:theses-5987","repository":{"repo_id":"eiu","name":"Eastern Illinois University","base_url":"https://thekeep.eiu.edu/do/oai/"},"display":{"title":"Assessing the Reliability of the Five-in-20 Classroom Observation Tool","abstract":"Effective classroom management practices are crucial to fostering a positive learning environment, student achievement, and student social-emotional development, as well as teacher job satisfaction. The Five-in-20 Classroom Observation Tool (FCOT) was developed from the 21 evidence-based strategies identified by Simonsen and colleagues (2008) to assess and support teachers’ classroom management; however, its psychometric properties are unknown. The present study looks at the interobserver agreement of the FCOT, specifically at how consistent observers’ ratings are with each other and how consistent observers’ ratings are with a standard coded copy of the FCOT. Forty-four participants, using the FCOT, rated the same 20-minute video of classroom instruction. Their ratings were compared using intraclass correlation coefficient and alpha coefficient. Intraclass correlation coefficient yielded excellent agreement for average measures and moderate agreement for single measures for both strategy endorsement and quality rating. Cronbach’s alpha also suggested strong reliability for strategy and quality. The participants’ ratings were compared to the standard coded FCOT using Cohen’s kappa and percent agreement. The kappa value for strategy endorsement suggested substantial agreement, while the kappa value for quality rating suggested moderate agreement. Percent agreement for strategy endorsement was calculated at 85.37% and for quality rating at 60.24%. Implications and suggestions for future research are provided.","abstract_html":"Effective classroom management practices are crucial to fostering a positive learning environment, student achievement, and student social-emotional development, as well as teacher job satisfaction. The Five-in-20 Classroom Observation Tool (FCOT) was developed from the 21 evidence-based strategies identified by Simonsen and colleagues (2008) to assess and support teachers’ classroom management; however, its psychometric properties are unknown. The present study looks at the interobserver agreement of the FCOT, specifically at how consistent observers’ ratings are with each other and how consistent observers’ ratings are with a standard coded copy of the FCOT. Forty-four participants, using the FCOT, rated the same 20-minute video of classroom instruction. Their ratings were compared using intraclass correlation coefficient and alpha coefficient. Intraclass correlation coefficient yielded excellent agreement for average measures and moderate agreement for single measures for both strategy endorsement and quality rating. Cronbach’s alpha also suggested strong reliability for strategy and quality. The participants’ ratings were compared to the standard coded FCOT using Cohen’s kappa and percent agreement. The kappa value for strategy endorsement suggested substantial agreement, while the kappa value for quality rating suggested moderate agreement. Percent agreement for strategy endorsement was calculated at 85.37% and for quality rating at 60.24%. Implications and suggestions for future research are provided.","abstract_has_math":false,"creators":["Buechlein, Danielle"],"institution":null,"degree_name":"Specialist in School Psychology","degree_level":null,"degree_discipline":"Psychology","degree_department":null,"school":null,"contributors":["Margaret T. Floress","Gary L. Canivez","Hao-Jan Luh"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":null,"date_issued":"","date_published":null,"updated_at":"2026-11-19T07:54:14Z","subjects":["classroom management","direct observation","evidence-based strategies","consultation","reliability","Curriculum and Instruction","Early Childhood Education","Elementary Education","Secondary Education","Special Education and Teaching","Teacher Education and Professional Development"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://thekeep.eiu.edu/theses/4978","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://thekeep.eiu.edu/theses/4978","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://thekeep.eiu.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Athekeep.eiu.edu%3Atheses-5987","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Margaret T. Floress","Gary L. Canivez","Hao-Jan Luh"]},{"key":"dc:creator","label":"Author","values":["Buechlein, Danielle"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.available","label":"Dc Date Available","values":["2023-05-19T07:00:00Z"]},{"key":"dc:date.created","label":"Dc Date Created","values":["2023-01-01T08:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Psychology"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Specialist in School Psychology"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["classroom management","direct observation","evidence-based strategies","consultation","reliability","Curriculum and Instruction","Early Childhood Education","Elementary Education","Secondary Education","Special Education and Teaching","Teacher Education and Professional Development"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://thekeep.eiu.edu/theses/4978"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Effective classroom management practices are crucial to fostering a positive learning environment, student achievement, and student social-emotional development, as well as teacher job satisfaction. The Five-in-20 Classroom Observation Tool (FCOT) was developed from the 21 evidence-based strategies identified by Simonsen and colleagues (2008) to assess and support teachers’ classroom management; however, its psychometric properties are unknown. The present study looks at the interobserver agreement of the FCOT, specifically at how consistent observers’ ratings are with each other and how consistent observers’ ratings are with a standard coded copy of the FCOT. Forty-four participants, using the FCOT, rated the same 20-minute video of classroom instruction. Their ratings were compared using intraclass correlation coefficient and alpha coefficient. Intraclass correlation coefficient yielded excellent agreement for average measures and moderate agreement for single measures for both strategy endorsement and quality rating. Cronbach’s alpha also suggested strong reliability for strategy and quality. The participants’ ratings were compared to the standard coded FCOT using Cohen’s kappa and percent agreement. The kappa value for strategy endorsement suggested substantial agreement, while the kappa value for quality rating suggested moderate agreement. Percent agreement for strategy endorsement was calculated at 85.37% and for quality rating at 60.24%. Implications and suggestions for future research are provided."]},{"key":"dc:title","label":"Title","values":["Assessing the Reliability of the Five-in-20 Classroom Observation Tool"]}]}],"canonical_facts":{"dc:contributor":["Margaret T. Floress","Gary L. Canivez","Hao-Jan Luh"],"dc:creator":["Buechlein, Danielle"],"dc:date.available":["2023-05-19T07:00:00Z"],"dc:date.created":["2023-01-01T08:00:00Z"],"dc:description.abstract":["Effective classroom management practices are crucial to fostering a positive learning environment, student achievement, and student social-emotional development, as well as teacher job satisfaction. The Five-in-20 Classroom Observation Tool (FCOT) was developed from the 21 evidence-based strategies identified by Simonsen and colleagues (2008) to assess and support teachers’ classroom management; however, its psychometric properties are unknown. The present study looks at the interobserver agreement of the FCOT, specifically at how consistent observers’ ratings are with each other and how consistent observers’ ratings are with a standard coded copy of the FCOT. Forty-four participants, using the FCOT, rated the same 20-minute video of classroom instruction. Their ratings were compared using intraclass correlation coefficient and alpha coefficient. Intraclass correlation coefficient yielded excellent agreement for average measures and moderate agreement for single measures for both strategy endorsement and quality rating. Cronbach’s alpha also suggested strong reliability for strategy and quality. The participants’ ratings were compared to the standard coded FCOT using Cohen’s kappa and percent agreement. The kappa value for strategy endorsement suggested substantial agreement, while the kappa value for quality rating suggested moderate agreement. Percent agreement for strategy endorsement was calculated at 85.37% and for quality rating at 60.24%. Implications and suggestions for future research are provided."],"dc:identifier":["https://thekeep.eiu.edu/theses/4978"],"dc:subject":["classroom management","direct observation","evidence-based strategies","consultation","reliability","Curriculum and Instruction","Early Childhood Education","Elementary Education","Secondary Education","Special Education and Teaching","Teacher Education and Professional Development"],"dc:title":["Assessing the Reliability of the Five-in-20 Classroom Observation Tool"],"thesis:degree_discipline":["Psychology"],"thesis:degree_name":["Specialist in School Psychology"]},"updated_at":"2026-11-19T07:54:14Z"}