In this work, the Bauschinger effect is shown to be intimately tied not only to plasticity but to damage as well. The plasticity-damage effect on the Bauschinger effect is demonstrated by employing different definitions (Bauschinger Stress Parameter, Bauschinger Effect Parameter, the Ratio of Forward-to-Reverse Yield, and the Ratio of Kinematic-to-Isotropic Hardening) for two differently processed aluminum alloys (rolled and cast) in which specimens were tested to different prestrain levels under tension and compression. Damage progression from second phase particles and inclusions that were generally equiaxed for the cast A356-T6 aluminum alloy and elongated for the rolled 7075 aluminum alloy was quantified from interrupted experiments. Observations showed that the Bauschinger effect had larger values for compression prestrains when compared to tension. The Bauschinger effect was also found to be a function of damage to particles/inclusions, dislocation/particle interaction, the work hardening rate, and the Bauschinger effect definition.