Outgroup (cladistics)

An outgroup that is nested within the ingroup will, when used to root the phylogeny, result in incorrect conclusions about phylogenetic relationships and trait evolution.

Choosing a closely related outgroup relative to the ingroup is more useful when looking at subtle differences, while choosing an unduly distant outgroup can result in mistaking convergent evolution for a direct evolutionary relationship due to a common ancestor.

For example, Jarvis et al. (2014) used humans and crocodiles as outgroups while resolving the early branches of the avian phylogeny.

Although there are algorithmic approaches to identify the outgroups with maximum global parsimony, they are often limited by failing to reflect the continuous, quantitative nature of certain character states.

[12] Character states are traits, either ancestral or derived, that affect the construction of branching patterns in a phylogenetic tree.