Spin-polarized first-principles electronic structure and total energy calculations have been performed to better understand the magnetic properties of Co doped ZnO (ZnO:Co) with vacancies and Ga co-dopants. The paramagnetic state of ZnO:Co, in which Co ions lose their magnetic moments, has been found to be unstable. The total energy results show that acceptor-like Zn vacancies and donor-like Ga co-dopants render the anti-ferromagnetic (AFM) and ferromagnetic (FM) states to be more favorable, respectively. With O vacancies, ZnO:Co has been found to be in the weak FM state. These magnetic properties can be understood by the calculated O- and Zn-vacancies and Ga-co-dopant induced changes of the electronic structure, which suggest that AFM and FM Co-Co couplings are mediated by O 2p-Co majority (¡ô)-spin 3d hybridized states in the valence band of ZnO and O-vacancy-derived p states or Ga sp states in the ZnO band gap, respectively. For ZnO:Co with Zn vacancies (Ga co-dopants) the AFM (FM) coupling outweighs the FM (AFM) coupling and results in the AFM (FM) state, while for ZnO:Co with O vacancies, both the FM and AFM couplings are enhanced by similar degrees and result in the weak FM state. This study reveals a competition between FM and AFM couplings in ZnO:Co with vacancies and Ga co-dopants, the detailed balancing between which determines the magnetic properties of these materials.