Abstract We are interested in studying Gaussian Markov random fields as correlation priors for Bayesian inversion. We construct the correlation priors to be discretisation-invariant, which means, loosely speaking, that the discrete priors converge to continuous priors at the discretisation limit. We construct the priors with stochastic partial differential equations, which guarantees computational efficiency via sparse matrix approximations. The stationary correlation priors have a clear statistical interpretation through the autocorrelation function. We also consider how to make structural model of an unknown object with anisotropic and inhomogeneous Gaussian Markov random fields. Finally we consider these fields on unstructured meshes, which are needed on complex domains. The publications in this thesis contain fundamental mathematical and computational results of correlation priors. We have considered one application in this thesis, the electrical impedance tomography. These fundamental results and application provide a platform for engineers and researchers to use correlation priors in other inverse problem applications.