Competition over access for mates has led to the evolution of many striking examples of morphological traits and behaviour in animals. The rapid development of the sexual selection field in recent decades have dramatically advanced our understanding of what traits make individuals more successful in attracting mates and how preferences for mates evolve over time. However, till now, research in this field has put less emphasis on the mechanisms that underlie variation in mate choice and sexual behaviour. Cognitive processes could potentially be key drivers of individual variation in mating preferences and sexual behaviours and therefore deserve further investigation. In this thesis, I used guppies artificially selected for relative brain size as the model system to study the association between brain size, cognitive ability and various aspects of mate choice. Previous studies in this model system showed that large-brained individuals of both sexes outperformed small-brained individuals in cognitive tests. Here I quantified their sexual behaviours and mating preferences to provide novel empirical data concerning the association between brain size, cognitive ability and sexual selection. In dichotomous choice preference tests based on visual cues, comparisons between large-brained and small-brained guppies showed important differences in their assessment of mate quality. These results are not driven by pre-existing visual biases caused by the artificial selection since further investigation of the visual capacity of these fish detected no differences between large-brained and small-brained individuals in their sensitivity to colour or in their capacity to resolve spatial detail. I also quantified sexual behaviour in male guppies artificially selected for relative brain size and found no difference in the behaviours of large-brained and small-brained males in a single male-single female non-competitive scenario. On the contrary, in a more complex social setting I found a reduction in large-brained males in the rate of courtship towards females and dominance displays towards other males when exposed to different degrees of predation threat and different numbers of male competitors. However, this reduction in behavioural intensity did not result in a lower access to copulation with females for large-brained males. I likewise evaluated female sexual behaviour and found that large-brained females had higher behavioural flexibility such that they decreased their receptiveness towards males more strongly under higher levels of predation threat. Together, these results provide novel empirical evidence that brain size and cognitive ability are tightly linked to mating preferences and sexual behaviours. These findings suggest that brain size and cognitive ability might be important mechanisms behind variation in mating preferences and in sexually selected traits across and within species. / <p>At the time of the doctoral defense, the following papers were unpublished and had a status as follows: Paper 2: Manuscript. Paper 3: Manuscript. Paper 5: Manuscript.</p>