The importance given to vocabulary in second language instruction and in theories of second language acquisition has increased greatly in the last fifteen years. It is thus important for second language teachers/researchers to have valid, useful methods of assessing the vocabulary needs and vocabulary knowledge of their students as well as valid and useful methods of assessing the efficacy of various methods and techniques of teaching and learning vocabulary. This study examines the usefulness and validity of a relatively new type of checklist vocabulary test method known as the 'YES/NO' method. In the YES/NO method, nonsense words are listed together with real test words. A subject's test score is calculated by applying both the percentage of real words checked and the percentage of nonsense words checked to a mathematical formula. Sixty-six students enrolled in a college-level Englishfor- academic-purposes (EAP) program took three vocabulary tests. Correlation was calculated between the participants' scores on the 'Structure' and 'Listening' subsections of the Comprehensive English Language Test (CELT) and the participants' scores on all three vocabulary tests scored both as YES/NO tests and as simple-checklist tests. The following three findings were noteworthy: (1) correlation between CELT subtest scores and vocabulary test scores was more consistent and stronger when the vocabulary tests were scored as simple-checklist tests compared to when they were scored as YES/NO tests, (2) few students scored above recommended exit-level scores on the CELT subtests but below 5,000 on the vocabulary tests, and (3) a 120-real-word vocabulary test correlated more consistently and strongly with the CELT subtests than either of two 60-real-word vocabulary tests. Three conclusions were made: (1) adult EAP students preparing to study at English-medium institutions of higher education need knowledge of the 5,000 most-frequent words [lemmas] of English, (2) the YES/NO method of testing the L2 vocabulary knowledge of adult EAP students is not better than the simple-checklist method, and (3) a good direction for work on the improvement of tests intended to measure the L2 vocabulary knowledge of adult EAP students may be to explore how to elicit valid responses on long simple-checklist tests.