[1][2] This result has been called the fundamental theorem of cyclic groups.
[3][4] For every finite group G of order n, the following statements are equivalent: If either (and thus both) are true, it follows that there exists exactly one subgroup of order d, for any divisor of n. This statement is known by various names such as characterization by subgroups.
However, the Klein group has more than one subgroup of order 2, so it does not meet the conditions of the characterization.
The infinite cyclic group is isomorphic to the additive subgroup Z of the integers.
More generally, a finitely generated group is cyclic if and only if its lattice of subgroups is distributive and an arbitrary group is locally cyclic if and only its lattice of subgroups is distributive.