Modern semiconductor fabrication techniques allow for the fabrication of semiconductor heterostructures which host electron transport with a minimum of scattering sites. In such devices, electrons populate a two-dimensional electron gas (2DEG) in which electrons propagate in exactly two dimensions, and may be further confined by potential barriers to form electron billiards. At sub-Kelvin temperatures, electron trajectories are determined largely by reflections from the billiard walls, while net conduction through the device depends on quantum mechanical wave interference. Measurements of magnetoconductance fluctuations (MCF) serve as a probe of dynamics within the electron billiard. Many prior studies have utilized heterostructures employing the modulation doping architecture, in which the 2DEG is spatially removed from the donor atoms to minimize electron scattering. Theoretical studies have claimed that MCF will be fractal when the confinement potential defining the billiard is soft-walled, regardless of the presence of smooth potentials within the billiard such as those introduced by remote ionized donors. The small-angle scattering sites resulting from these potentials are often disregarded as negligible; we use MCF measurements to investigate such claims. To probe the effect of remote ionized donor scattering on the phase space in electron billiards, we compare MCF measured on billiards in a modulation-doped heterostructure to those measured on billiards in an undoped heterostructure, in which this potential landscape is believed to be absent. Fractal studies are performed on these MCF traces, and we find that MCF measured on the undoped billiards do not exhibit measurably different fractal characteristics than those measured on the modulation-doped billiards. Having confirmed that the potential landscapes in modulation-doped heterostructures do not affect the electron phase space, we then investigate the effect of these impurities on the distribution of electron trajectories through the billiards. By employing thermal cycling experiments, we demonstrate that this distribution is highly sensitive to the precise potential landscape within the billiard, suggesting that modulation-doped heterostructures do not support fully ballistic electron transport. We compare our MCF correlation data with the dynamics of charge transfer within heterostructure systems to make qualitative conclusions regarding these dynamics.