Recovering a linear combination of exponential signals characterized by parameters is highly significant in many MR imaging applications such as parameter mapping and spectroscopy. The parameters carry useful clinical information and can act as biomarkers for various cardiovascular and neurological disorders. However, their accurate estimation requires a large number of high spatial resolution images, resulting in long scan time. One of the ways to reduce scan time is by acquiring undersampled measurements. The recovery of images is usually posed as an optimization problem, which is regularized by functions enforcing sparsity, smoothness or low rank structure. Recently structured matrix priors have gained prominence in many MRI applications because of their superior performance over the aforementioned conventional priors. However, none of them are designed to exploit the smooth exponential structure of the 3D dataset. In this thesis, we exploit the exponential structure of the signal at every pixel location and the spatial smoothness of the parameters to derive a 3D annihilation relation in the Fourier domain. This relation translates into a product of a Hankel/Toeplitz structured matrix, formed from the k-t samples, and a vector of filter coefficients. We show that this matrix has a low rank structure, which is exploited to recover the images from undersampled measurements. We demonstrate the proposed method on the problem of MR parameter mapping. We compare the algorithm with the state-of-the-art methods and observe that the proposed reconstructions and parameter maps have fewer artifacts and errors. We extend the structured low rank framework to correct field inhomogeneity artifacts in MR images. We introduce novel approaches for field map compensation for data acquired using Cartesian and non-Cartesian trajectories. We adopt the time segmentation approach and reformulate the artifact correction problem into a recovery of time series of images from undersampled measurements. Upon recovery, the first image of the series will correspond to the distortion-free image. With the above re-formulation, we can assume that the signal at every pixel follows an exponential signal characterized by field map and the damping constant R2*. We exploit the smooth exponential structure of the 3D dataset to derive a low rank structured matrix prior, similar to the parameter mapping case. We demonstrate the algorithm on spherical MR phantom and human data and show that the artifacts are greatly reduced compared to the uncorrected images. Finally, we develop a structured matrix recovery framework to accelerate cardiac breath-held MRI. We model the cardiac image data as a 3D piecewise constant function. We assume that the zeros of a 3D trigonometric polynomial coincides with the edges of the image data, resulting in a Fourier domain annihilation relation. This relation can be compactly expressed in terms of a structured low rank matrix. We exploit this low rank property to recover the cardiac images from undersampled measurements. We demonstrate the superiority of the proposed technique over conventional sparsity and smoothness based methods. Though the model assumed here is not exponential, yet the proposed algorithm is closely related to that developed for parameter mapping. The direct implementation of the algorithms has a high memory demand and computational complexity due to the formation and storage of a large multi-fold Toeplitz matrix. Till date, the practical utility of such algorithms on high dimensional datasets has been limited due to the aforementioned reasons. We address these issues by introducing novel Fourier domain approximations which result in a fast and memory efficient algorithm for the above-mentioned applications. Such approximations allow us to work with large datasets efficiently and eliminate the need to store the Toeplitz matrix. We note that the algorithm developed for exponential recovery is general enough to be applied to other applications beyond MRI.