Relatively complex Augmented Reality (AR) algorithms are becoming widely available due to advancements in affordable mobile computer hardware. To take advantage of this a new method is developed for tracking 2D regions without a prior knowledge of an environment and without developing a computationally expensive world model. In the method of this paper, affinely invariant planar regions in a scene are found using the Maximally Stable Extremal Region (MSER) detector. A region is selected by the user to define a search space, and then the Scale Invariant Feature Transform (SIFT) is used to detect affine invariant keypoints in the region. If three or more keypoint matches across frames are found, the affine transform A of a region is calculated. A 2D image is then transformed by A causing it to appear stationary on the 2D region being tracked. The search region is tracked by transforming the previous search region by A, defining a new location, size, and shape for the search region. Testing reveals that the method is robust to tracking planar surfaces despite affine changes in the geometry of a scene. Many real world surfaces provide adequate texture for successful augmentation of a scene. Regions found multiple frames are consistent with one another, with a mean cross-correlation of 0.608 relating augmented regions. The system can handle up to a 45° out of plane viewpoint change with respect to the camera. Although rotational changes appear to skew the affine transform slightly, translational and scale based have little distortion and provide convincing augmentations of graphics onto the real world.