Having become interested in the various Native American tribes, Bliss learned a number of their languages and studied their cultures.

Born in Whitehall, New York, he was the son of Captain John Bliss (of Lebanon, New Hampshire) and Olive Hall Simonds (of Todd County, Kentucky).

In 1845, Bliss took part in the United States military occupation of the Republic of Texas, prior to its annexation.

Between April 1846 and November 1847, he took part in the Mexican War, including fighting in the battles of Palo Alto, Resaca de la Palma and Buena Vista.

[1] During his entire service in Texas and Mexico, he served as chief of staff to Major General Zachary Taylor.

On December 5, 1848, at Baton Rouge, Louisiana, Bliss married President-elect Zachary Taylor's youngest daughter, Mary Elizabeth (1824–1909).

Taylor appointed Bliss as Private Secretary to the President (he was able to take leave from the Army for this assignment).

Following a summer visit to New Orleans on behalf of University of Louisiana, Bliss contracted yellow fever.