Environmental constraints can limit a population to a certain size, which is usually called the carrying capacity of a habitat. Besides to this 'external' factor, which is mainly determined by the limitation of resources, we investigate here another set of population-intrinsic factors that can limit a population size significantly below the maximum sustainable size. Firstly, density-independent mortality is a prominent factor in all organisms that show age-related and/or accidental death. Secondly, in sexually reproducing organisms the sex ratio and the success of pairing is important for finding reproductive partners. Using a simple model, we demonstrate how sex ratio, mating success and gender-specific mortality can strongly affect the speed of population growth and the maximum population size. In addition, we demonstrate that density-independent mortality, which is often neglected in population models, adds a very important feature to the system: it strongly enhances the negative influence of unbiased sex ratios and inefficient pairing to the maximum sustainable population size. A decrease of the maximum population size significantly affects a population's survival chance in inter-specific competition. Thus, we conclude that the inclusion of density-independent mortality is crucial, especially for models of species that reproduce sexually. We show that density-independent mortality, together with biased sex ratios, can significantly lower the abilities of a population to survive in conditions of strong inter-specific competition and due to the Allee effect. We emphasize that population models should incorporate the sex ratio, male success and density-independent mortality to make plausible predictions of the population dynamics in a gender-structured population. We show that the population size is limited by these intrinsic factors. This is of high ecological significance, because it means that there will always be resources available in any habitat that allows other species (e.g., invaders) to use these resources and settle successfully, if they are sufficiently adapted.