Host-shift speciation and cospeciation are the two major processes driving symbiont macroevolutionary diversification. Cospeciation is expected to be frequent in vertically transmitted and host-specific symbionts, and leads to congruent host-symbiont phylogenies. However, the cophylogenetic dynamics of many groups of highly specialized host-specific symbionts is largely unstudied. Thus, the relevance of cospeciation vs. host-shift speciation remains largely unknown. Here, we investigated this question by performing the largest cophylogenetic study of feather mites to date, using both distance and event-based cophylogenetic methods. For these analyses, we inferred phylogenies based on all protein coding genes of the mitochondrial genome of Proctophyllodes and Trouessartia feather mite species living on European passerine birds. Results show high incongruence among bird and feather mite phylogenies, because of extensive host-switching. We conclude that host-shift speciation, rather than cospeciation, may be the main driver of symbiont diversification even for highly specialized symbionts with low host-switching potential.