Originally posted as a preprint on April 4, 2021
It is thought that two species can coexist if they use different resources present in the environment, yet this assumes that species are completely reproductively isolated. We simulate coexistence outcomes for two sympatric species that are ecologically differentiated but have incomplete reproductive isolation. The consequences of interbreeding crucially depend on hybrid fitness. When hybrid fitness is high, just a small rate of hybridization can lead to collapse of two species into one. Low hybrid fitness can cause population declines, making extinction of one or both species likely. High intrinsic growth rates result in higher reproductive rates when populations are below carrying capacity, reducing the probability of extinction and increasing the probability of stable coexistence at moderate levels of assortative mating and hybrid fitness. Very strong but incomplete assortative mating can induce low hybrid fitness via a mating disadvantage to rare genotypes, and this can stabilize coexistence of two species at high but incomplete levels of assortative mating. Given these results and evidence that it may take many millions of years of divergence before related species become sympatric, we postulate that coexistence of closely related species is more often limited by insufficient assortative mating than by insufficient ecological differentiation.