The discovery of exoplanets represents one of the greatest scientific revolutions in history, and exoplanetary science has rapidly become uniquely positioned to address profound questions about the origins of life, and about humanity's place (and future) in the cosmos. Since the discovery of the first exoplanet over two decades ago, the radial velocity (RV) method has been one of the most productive techniques for discovering new planets. It has also become indispensable for characterising exoplanets detected via other techniques, notably transit photometry. Unfortunately, signals intrinsic to stars themselves - especially magnetic activity signals - can induce RV variations that can drown out or even mimic planetary signals. Modelling and thus mitigating these signals is notoriously difficult, which represents a major obstacle to using next-generation instruments to detect lower mass planets, planets with longer periods, and planets around more magnetically-active stars. Enter Gaussian processes (GPs), which have a number of features that make them very well suited to the joint modelling of stochastic activity processes and dynamical (e.g. planetary) signals. In this thesis, I leverage GPs to enable the study of smaller planets around a wider variety of stars than has previously been possible. In particular, I develop a principled and sophisticated Bayesian framework, based on GPs, for modelling RV time series jointly with ancillary activity-sensitive proxies, thus allowing activity signals to be constrained and disentangled from genuine planetary signals. I show that my framework succeeds even in cases where existing techniques would fail to detect planets, e.g. the case of a weak planetary signal with period identical to its host star's rotation period. In a first application of the framework, I demonstrate that Alpha Centauri Bb - until 2016, thought to be the closest exoplanet to Earth, and also the lowest minimum-mass exoplanet around a Sun-like star - was, in fact, an astrophysical false positive. Next, I use the framework to re-characterise the well-studied Kepler-10 system, thereby resolving a mystery surrounding the mass of planet Kepler-10c. I also use the framework to help discover or characterise various exoplanets. Finally, the activity modelling framework aside, I also present in outline form a few promising applications of GPs in the context of modelling stellar signals and studying exoplanets, viz. GPs for (i) enhanced characterisation of stellar rotation; (ii) generating realistic synthetic observations, and modelling in a systematic way the effects of an observing window function; and (iii) ultra-precise extraction of RV shifts directly from observed spectra, without requiring template cross-correlation.