Objective. To use cognitive interviewing techniques to determine faculty and student interpretation of a subset of items from the AACP faculty and graduating student surveys. Methods. Students and faculty were interviewed individually in a private room. The interviewer asked each respondent for his/her interpretation of 15 randomly selected items from the graduating student survey or 20 items from the faculty survey. Results. While many items were interpreted consistently by respondents, the researchers identified several items that were either difficult to interpret or produced differing interpretations. Conclusion. Several interpretational inconsistencies and ambiguities were discovered that could compromise the usefulness of certain survey items.