Understanding how urbanization alters functional interactions among pollinators and plants is critically important given increasing anthropogenic land use and declines in pollinator populations. Pollinators often exhibit short-term specialization, and visit plants of the same species during one foraging trip. This facilitates plant receipt of conspecific pollen -- pollen on a pollinator that is the same species as the plant on which the pollinator was foraging. Conspecific pollen receipt facilitates plant reproductive success and is thus important to plant and pollinator persistence. We investigated how urbanization affects short term specialization of insect pollinators by examining pollen loads on insects' bodies and identifying the number and species of pollen grains on insects caught in urban habitat fragments and natural areas. We then assessed possible drivers of differences between urban and natural areas, including frequency dependence in foraging, species richness and diversity of the plant and pollinator communities, floral abundance, and the presence of invasive plant species. Pollinators were more specialized in urban fragments than in natural areas, despite no differences in the species richness of plant communities across site types. These differences were likely driven by higher specialization of common pollinators, which were more abundant in urban sites. Pollinators were also more specialized when foraging on invasive plants across sites, and floral abundance of invasive plants was higher in urban sites. Our findings reveal strong effects of urbanization on pollinator fidelity to individual plant species and have implications for the maintenance of plant species diversity in small habitat fragments. The higher fidelity of pollinators to invasive plants suggests that native species may receive fewer visits by pollinators. Therefore, native plant species diversity may decline in urban sites without continued augmentation of urban flora or removal of invasive species.