In mathematics, and especially algebraic geometry, stability is a notion which characterises when a geometric object, for example a point, an algebraic variety, a vector bundle, or a sheaf, has some desirable properties for the purpose of classifying them.
The exact characterisation of what it means to be stable depends on the type of geometric object, but all such examples share the property of having a minimal amount of internal symmetry, that is such stable objects have few automorphisms.
However, high amounts of symmetry are not desirable when one is attempting to classify geometric objects by constructing moduli spaces of them, because the symmetries of these objects cause the formation of singularities, and obstruct the existence of universal families.
[1] However the ideas behind Mumford's work go back to the invariant theory of David Hilbert in 1893, and the fundamental concepts involved date back even to the work of Bernhard Riemann on constructing moduli spaces of Riemann surfaces.
Here extremal is generally meant in the sense of the calculus of variations, in that such objects minimize some functional.