Bismuth is among the most studied of all elements, but its behaviour under pressure exhibits myriad unexpected puzzles even after many decades of research. Bismuth narrowly avoids being an insulator: a Peierls-type distortion almost completely gaps the electronic energy bands, producing a rhombohedral metal with a tiny overlap of conduction and valence bands. The resulting solitary free electron per 100,000 atoms can travel large distances in high-purity crystals, leading to a host of unusual properties. We show that the rhombohedral structure can be tuned with pressure, driving the carrier concentration to nearly zero. We compare our measurements to recent experimental advances implying the formation of novel electronic order driven by the pairing of low-density electrons and holes, and show evidence for a previously unseen phase at very low temperatures in the semiconducting state. We also present a method for calculating the carrier density and resistivity as a function of pressure, based on phenomenological band parameters and a simple charge-balance argument, and demonstrate that this approach can quite well describe most - but not all - of the observed behaviour of the resistivity. At higher pressures, bismuth undergoes a transition into a quasiperiodic host-guest structure. Here, two distinct crystal lattices coexist and interpenetrate, but the lattice parameters are incommensurate. This crystal thus lacks a single unit cell - an unexpected complexity for a simple element. The discovery of such unusual structures in elements is a new phenomenon and their physical properties are rather unexplored. We present experimental measurements of the resistivity and magnetic susceptibility in the incommensurate host-guest state. We argue that the experimental data (in particular, the shape of the normal-state electrical resistivity, and the high value of the low-temperature upper critical field) may be evidence for strong electron-phonon coupling. This strong coupling is consistent with theoretical predictions which suggest the presence of a low-energy phonon mode arising due to the vanishing energy cost of moving guest atoms through the host lattice.