Context: Scapular motion evaluation is a necessary component of the upper extremity exam. Several methods exist, but most lack good reliability or diagnostic accuracy. The yes/no scapular dyskinesis test has the best of both measures but is untested on inexperienced clinicians. Objective: The purpose of this study was to evaluate the reliability and diagnostic accuracy of the yes/no scapular dyskinesis test when used by graduate assistant athletic trainers. Participants: The participants were college-aged students with no prior history of upper extremity fracture or nerve injury. Data Collection and Results: Participants were evaluated for scapular dyskinesis by a physician and 10 graduate assistant athletic trainers. Ratings were for normal or dyskinetic and then determination of side. Reliability was calculated using Gwet’s AC1 statistic and diagnostic accuracy from standard 2x2 contingency tables. Results: Reliability was moderate (AC1 = 0.48, p < 0.0025, 95% CI [0.147, 0.812]) when side was not accounted for and moderate (AC1 = 0.43, p < 0.0001, 95% CI [0.242, 0.632]) when side-per-side decisions were made. Sensitivity and negative predictive values were low to moderate (34.4%-66.2%, 8.9%-74.1% ). Specificity and positive predictive values were moderate to high (50%-85.2%, 51.5%-95.2%). Accuracy was moderate (65.2%-69.4%) and positive and negative likelihood ratios were low (1.325-2.333, 0.675-0.769). Conclusions: The reliability in this study was on par with previously published studies. Measures of diagnostic accuracy met or exceeded previous results. Clinically, to avoid false negative results and enhance the use of positive results, it appears necessary to combine methods and begin the evaluation with a gross assessment of whether or not dyskinesis is present and if it is to then evaluate which side is dyskinetic.