This study evaluated 227 students attending 12 classes of the Apprentice Medical Services Specialist Resident Course. Six classes containing a total of 109 students took the Block One Tests in the traditional paper and pencil form. Another six classes containing a total of 118 students took the same Block One Tests on computers. A confidence level of .99 and level of signifi­cance of .01 was established. An independent samples t-test was conducted on the sample. Additionally, a one-way analysis of variance was performed between the classes administered the Block One Tests on computers. Several other frequencies and comparisons of Block One Test scores and other variables were accomplished. The variables examined included test versions, shifts, student age, student source, and education levels. The study found no significant difference between test administration modes. This study concluded that computer-administering tests identical to those typically administered in the traditional paper and pencil manner had no significant effect on achievement. It is important to note, however, that the conclusion may only be valid if the computer-administered test contains exactly the same test items, in the same order and format, with the same layout, structure, and choices as the traditional paper and pencil test. In other words, unless the tests are identical in every possible way except the actual test administration mode this conclusion may not be applicable.