In the last few years, many schools and universities have incorporated personal
digital assistants (PDAs) into their teaching curricula, in an attempt to enhance
students' learning experience and reduce instructors' workload. One of the
most common uses of PDAs in the classroom is as a test administrator. This
study compared the usability effectiveness, efficiency, and satisfaction of a
PDA-based quiz application to that of standard paper-and-pencil quizzes in a
university course in order to determine whether it was advisable to invest time
and money in PDA-based testing. The effects of computer anxiety, age, gender,
and ethnicity on usability were also evaluated, to ascertain that these factors do
not discriminate against individuals taking PDA-based tests.
Five quizzes were administered to students participating in an engineering
introductory course. Of these, students took two PDA-based quizzes and three
paper-and-pencil quizzes. One PDA-based quiz and one paper-and-pencil quiz
were compared in terms of their effectiveness, measured as students' quiz
scores and through a mental workload questionnaire; their efficiency, which
was the time it took students to complete each quiz; and their satisfaction,
evaluated using a subjective user satisfaction questionnaire. Computer anxiety
was also measured, using an additional questionnaire.
It was hypothesized that the PDA-based quiz would be more effective and
efficient than the paper-and-pencil quiz and that students' satisfaction with the
PDA-based quiz would be greater. The study showed the PDA-based quiz to be
more efficient, that is, students completed it in less time than they needed to
complete the paper-and-pencil quiz. No differences in effectiveness and
satisfaction were found between the two quiz types.
It was also hypothesized that for PDA-based quizzes, as computer anxiety
increased, effectiveness and satisfaction would decrease; for paper-and-pencil
quizzes there would be no relationship between computer anxiety and
effectiveness and no relationship between computer anxiety and satisfaction.
Findings showed an increase in quiz score (increase in effectiveness) and an
increase in mental workload (decrease in effectiveness) as computer anxiety
increased for both quiz types. No relationship was found between computer
anxiety and satisfaction for either paper-and-pencil or PDA-based quizzes.
The final hypothesis suggested that user satisfaction would be positively
correlated with effectiveness (quiz score and mental workload) for both PDA-based
and paper-and-pencil quizzes. No relationship was found between quiz
score and satisfaction for either quiz type. User satisfaction was positively
correlated with mental workload, regardless of quiz type.
The usability comparison of paper-and-pencil and PDA-based quizzes found
the latter to be equal, if not superior, to the former. The effort students put into
taking the quiz was the same, regardless of administration method, and scores
were not affected. In addition, different demographic groups performed almost
equally well in both quiz types (white students' PDA-based quiz scores were
slightly lower than those of the other ethnic groups). Computer anxiety was not
affected by the quiz type. For these reasons, as well as other advantages to both
students (e.g. real-time scoring) and teachers (e.g. spending less time on
grading), PDAs are an attractive test administration option for schools and
universities. / Graduation date: 2004
| Identifer | oai:union.ndltd.org:ORGSU/oai:ir.library.oregonstate.edu:1957/30800 |
| Date | 17 July 2003 |
| Creators | Segall, Noa |
| Contributors | Porter, J. David |
| Source Sets | Oregon State University |
| Language | en_US |
| Detected Language | English |
| Type | Thesis/Dissertation |
Page generated in 0.0015 seconds