Aluminum laminates of high and technical purity layers were produced by accumulative roll bonding (ARB) at room temperature. To study the thermal stability, the laminates after 2 to 9 ARB cycles were annealed between 100 and 400 °C for one hour. Changes of the microstructure were analyzed by electron backscatter diffraction. For low ARB cycle numbers (4 or below) and 300 °C annealing temperature, the deformed technical pure layers start to recrystallize while the high-purity coarse recrystallized layers experience intralayer grain growth. For higher ARB cycle numbers (6 and 8) and an annealing temperature of 300 °C or above, the ultra-fine grained layers of technical purity are consumed by the layer overlapping growth of high-purity grains producing a banded grain structure. For 9 ARB cycles and at an annealing temperature of 400 °C, a globular grain structure develops with grain sizes larger than twice the layer thickness. The effect of impurities on recrystallization and grain growth of ARB laminates is discussed with regard to tailoring its microstructure by heat treatment. For further analyses, the results are compared with Potts model simulations finding a rather good qualitative agreement with the experimental data albeit some simplified model assumptions.