Born and raised on a farm in Centreville, Maryland, Whiteley grew up around horses and from a very early age was intent on racing them.
Long reticent about talking to the media, leading up to the 1967 Kentucky Derby, Whiteley finally gave in as a result of all the hoopla surrounding Damascus.
Actor Sam Shepard played the role of Whiteley in the made-for-television movie Ruffian which aired June 9, 2007, on the ABC television network.
In 1977, he conditioned Forego for another successful season that saw him earn his fourth consecutive Eclipse Award for Outstanding Older Male Horse.
Beginning in 1965, he wintered his horses at Marion duPont Scott's training Center in Camden, South Carolina, which set a trend with other horsemen that continues to this day.
Whiteley's son, David, followed in his footsteps as a successful trainer of a number of graded stakes race winners.