Heterogeneous line broadening and spectral diffusion of the fluorescence emission spectra of perylene diimide molecules have been investigated by means of time dependent single molecule spectroscopy. The influence of temperature and environment has been studied and reveals strong correlation to spectral diffusion processes. We followed the freezing of the molecular mobility of quasi free molecules on the surface upon temperature lowering and by embedding into a poly(methyl methacrylate) (PMMA) polymer. Thereby changes of optical transition energies as a result of both intramolecular changes of conformation and external induced dynamics by the surrounding polymer matrix could be observed. Simulations of spectral fluctuations within a two-level system (TLS) model showed good agreement with the experimental findings.