<p>As pedagogical innovations
continue to be developed and adopted in engineering education, it is important
to understand how these innovations affect the students’ experiences and
achievements. A common data analysis
practice when evaluating educational innovations is to aggregate the data from
all of the students together. However,
this data aggregation inherently biases the results toward the characteristics
of the dominant student group, leaving the experiences of minority groups
largely unexplored. In this
dissertation, I investigate the students’ experiences and achievements in an
undergraduate dynamics course, and I intentionally use analysis methods that
disaggregate the data to better understand the behaviors and performance of
smaller subgroups of students, not just the majority.</p>
<p> This
dissertation presents three studies that examine: 1) the validity, reliability,
and fairness of a standardized set of conceptual questions on the final exam,
with a focus on gender fairness, 2) how and why the students use the available
resources, and 3) how the students’ holistic resource usage patterns relate to
their academic achievement. My
motivation for choosing these studies was that conceptual assessments and
customized resources are two key components of the learning environment for the
dynamics course. However, the quality of
the conceptual exam questions used for the course had yet to be evaluated. Similarly, the learning environment for the
course incorporates many customized resources, including a custom-written
“lecturebook” (a hybrid of a textbook and a workbook) and an extensive online
library of videos, but little was known about how the students used these
resources, or how the students’ pattern of resource usage related to their
performance in the course. </p>
<p> The
first study in this dissertation used multiple-group confirmatory factor
analysis to investigate item-level gender bias in a 12-item Abbreviated
Dynamics Concept Inventory (aDCI), which was a set of standardized conceptual
questions included on the final exam.
The results suggested that two items were slightly biased against women,
with stereotypically-masculine contexts and content as possible sources of the
bias. The bias in the aDCI items likely
unfairly lowered some women’s final exam scores, highlighting the need for
engineering educators to consider the fairness of their assessments.</p>
<p> The
second study used a cluster analysis of survey responses to identify nine
archetypical patterns of resource usage, all of which differed from the average
resource-usage pattern of the aggregated sample. An analysis of forty-four student interviews,
organized by resource-usage cluster, determined that students exhibited their
resource-usage behaviors largely because of how they perceived the resource’s
availability, accessibility, and quality.
The results illustrate that there is no “typical” way in which the
students used the resources, so it is important for instructors to consider a
wide array of usage behaviors when designing a course’s learning environment
and resources.</p>
<p> The
third study utilized a multiple regression analysis to find that <i>on average</i> a student’s resource-usage
pattern is not related to their achievement when controlling for many other
demographic, cognitive, and non-cognitive factors that can affect resource
usage and performance. However, two
individual resource-usage patterns were significantly related to achievement. Students who primarily used their lecturebook
and their peers for support performed better than their similar peers in other
resource-usage clusters. Conversely,
students who rarely used their lecturebook had lower course grades than their
peers. Drawing from the results of the
second study, general study-habit suggestions for the students in the course
were extracted from the qualitative themes found in the interviews of the
students in these two clusters.</p>
<p> Overall,
the results of these three studies highlight how the experiences and
achievements of smaller groups of students would go unnoticed if analytical
methods that only utilized aggregated data were used. While the setting of this research is
specific to the assessments and resources of a given dynamics course, the
methods used to disaggregate the data to gain insights about different
subgroups of students are applicable to many engineering education
contexts. My hope is that this work
inspires more researchers to consider the experiences of all students, not just
those of the majority.</p>
| Identifer | oai:union.ndltd.org:purdue.edu/oai:figshare.com:article/7427465 |
| Date | 17 January 2019 |
| Creators | Nick A. Stites (5930300) |
| Source Sets | Purdue University |
| Detected Language | English |
| Type | Text, Thesis |
| Rights | CC BY 4.0 |
| Relation | https://figshare.com/articles/BEYOND_AGGREGATED_DATA_A_STUDY_OF_GROUP_DIFFERENCES_IN_CONCEPTUAL_UNDERSTANDING_AND_RESOURCE_USAGE_IN_AN_UNDERGRADUATE_DYNAMICS_COURSE/7427465 |
Page generated in 0.0022 seconds