This study analyzed the required inclusion of school test scores in the yearly evaluation of school principals within the current school reform and standards and accountability movements of both the federal and state departments of education. Extant data from a single school district in Oregon was used for this study, and included: (a) district-wide elementary principal summative performance evaluation scores, (b) district-wide fourth and fifth grade fall and spring reading curriculum-based measures scores curriculum-based measures, (c) 2013 and 2014 spring reading scores from the Oregon Assessment of Knowledge and Skills (OAKS-R), and (d) student demographic variables. The student non-academic predictor variables (demographic risk factors) included in the study were (a) attendance, (b) English Language Proficiency (ELP), (c) Free and Reduced Meals (FARMS), (d) percent Other-than-White, and (e) Special Education. Multiple regression analyses were used to determine which assessment and/or non-assessment factors accounted for differences between principals’ summative evaluation scores. The results indicated that Summative Principal Ratings are poor predictors of the academic success of all students whether using large-scale summative assessment (OAKS-R) or formative assessments (easyCBM), with all measures only accounting for a miniscule portion of the Summative Principal Rating variance. However, demographic variables were slightly more related to the Summative Principal Rating. Practical implications of using student test scores to hold principals accountable for the academic results of all students are discussed in relation to district administrative policy and placement procedures for administrators and teachers, examining the behaviors and practices of teachers’ whose students have shown the most gains, and using these successful teacher practices a basis for teacher-to-teacher district-wide professional development. Finally, suggestions for future research in the areas of improving principal evaluation systems and the study of direct and indirect impacts principals have on student success and achievements are discussed.