Back to search

University of South Florida

Development and Initial Validation of the In-The-Moment Asynchronous Student Engagement Questionnaire (ITMASEQ) in Higher Education

Abstract

dc:description

This dissertation focuses on the development and initial validation of the In-The-Moment Asynchronous Student Engagement Questionnaire (ITMASEQ), designed to measure undergraduate student engagement—emotional, cognitive, and behavioral—in asynchronous online learning environments. The study responds to the growing need for dynamic, in-the-moment assessments of student engagement, a construct critical to understanding academic success and retention, particularly in online education. The ITMASEQ included a total of 25 questions: Five contextual questions (study day, study place, company during the study, primary online activity, type of primary online activity), eight emotional engagement questions (enjoyment, feeling calm, anxious, focused, distracted, etc.), seven cognitive engagement questions (interest, challenge, mental effort, etc.) and five behavioral questions (persistency, following directions, etc.). The Experience Sampling Method (ESM) was employed to capture real-time engagement data across multiple interactions: student-content, student-instructor, student-student, and student-AI. The data were collected for 8 weeks, once a week from 83 students who took asynchronous online courses from the College of Education for the main study. The rigorous item development and review process using multiple expert panels and student feedback about the clarity and relevance of the items strengthened the content validity of the ITMASEQ. Cronbach’s alphas for the three engagement subscales (emotional, cognitive, and behavioral) across the eight time periods were generally low with emotional engagement showing moderate reliabilities (> .70) for five of the eight time periods and cognitive engagement showing moderate reliability for one time point. Reliability was below .70 for behavioral engagement for each time point. While the study offers some initial and promising insights into the structure of the ITMASEQ, the scale is not yet ready for practical application. Further refinement is needed to enhance its robustness and improve its practicality in effectively capturing the construct. Given that the current version of the ITMASEQ includes more items than most Experience Sampling Method (ESM) studies, its practicality warrants reconsideration. While the comprehensive nature of the tool initially seemed advantageous for capturing a wide range of data, a shorter version could offer substantial benefits, particularly in terms of usability and reducing respondent burden.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy (Ph.D.)
Level thesis:degree_level
Dissertation
Discipline thesis:degree_discipline
Educational Measurement and Research
Grantor
University of South Florida
Year dc:date
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Topsakal, Elif
Contributors dc:contributor
  • Robert Dedrick, Ph.D.
  • Eunsook Kim, Ph.D.
  • John Ferron, Ph.D.
  • Sanghoon Park, Ph.D.

Subjects

dc:subject × 8

Identifiers

dc:identifier.*
Repository record dc:identifier
https://digitalcommons.usf.edu/etd/11205

Chain of custody

source
Harvested from
University of South Florida
Base URL
digitalcommons.usf.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Topsakal, Elif. Development and Initial Validation of the In-The-Moment Asynchronous Student Engagement Questionnaire (ITMASEQ) in Higher Education. Dissertation thesis, University of South Florida, 2024. https://digitalcommons.usf.edu/etd/11205