[1][2] His maternal grandfather, William Pinkney, had been a famous United States politician, administrator, and diplomat.

Whyte's early education involved instruction by a private tutor, who had been personal secretary to Napoleon Bonaparte.

[citation needed] From 1842 to 1844, Whyte was unable to attend college as a result of his family's poor financial situation, and began work at the banking firm of Peabody, Riggs and Co. in Baltimore.

After nearly a decade out of the political arena, Whyte was asked by then-governor Thomas Swann to fill the remainder of resigning senator Reverdy Johnson's term from July 13, 1868, to March 3, 1869.

In 1906, Whyte was appointed by Maryland Governor Edwin Warfield to fill the Senate seat vacancy caused by the death of Arthur P. Gorman.