Fragmentation of forest habitat by urban and rural development has had profound effects on the distribution and abundance of many native species; however, little is known about the ecological processes driving patterns in community structure (species richness and composition) of host-specialised herbivores in modified habitats. I examined patterns in community structure of 9 specialist and 19 generalist invertebrate herbivores of cabbage trees (Cordyline australis Laxmanniaceae) across a highly-modified landscape. I found that, although species richness of specialists was highest in forest sites, the majority of host-specialised species were not restricted to forest habitats and were as widespread as many generalists. In terms of site occupancy, only two specialist and four generalist species were rare. I show that patterns of species occupancy and abundance reflect differing susceptibility to habitat modification, with landscape-level variation an important predictor of abundance for nearly all species. When species occurrences and life history traits were considered I did not find strong evidence for the importance of dispersal ability, which suggests that habitat variability had a stronger organising effect on the community. In a replicated common garden experiment, I found distinct regional patterns in the community structure of the specialist invertebrates occurring on different phylogenetic groups of C. australis. In contrast, community structure of generalist herbivores did not differ significantly among host genotypes. I speculate these patterns are due to historical changes in the distribution of cabbage trees in the Southern phylogenetic region that caused specialised herbivores to become locally adapted on populations of low genetic diversity following expansion after the last glacial maximum. However, this consistent selection pressure did not occur in the Northern region where C. australis habitat has been more consistently available over the past tens of thousands of years, reflected in higher host genetic diversity. This study has advanced our understanding of the patterns in community structure of an indigenous, host-specialised fauna in a highly modified and fragmented urban and rural landscapes.