The option to bring-your-own-device (BYOD) to educational settings is becoming more prevalent as mobile technologies are more accessible than ever, yet little research has been done to examine the effect of those devices on language assessment. In this study, participants (n=175) were divided by stratified random sampling into four groups. Using a Latin square design to control for ordering, two forms of a multiple-choice reading and listening exam were administered over two days. On each day, participants took one test on a BYOD mobile device and one on a computer. A repeated measures ANOVA was used to determine the effect that device type had on score. During the administration of the test, the BYOD condition revealed a number of difficulties that would caution against full-scale adoption for high stakes testing, but the test scores on the computer and BYOD mobile version of the exam were not significantly different in either skill area.