During the American Civil War, Baldwin believed his primary loyalty was to his state, and served as one of Virginia's representatives to the First and Second Confederate Congresses.

During the Presidential Election of 1860, Baldwin canvassed for John Bell, the Constitutional Unionist candidate, who won in Virginia but drew far fewer electoral college votes than either Republican Abraham Lincoln or Democrat Stephen A. Douglas.

On April 4, 1861, Baldwin represented the Convention's Unionist leadership at a secret one-hour interview with President Abraham Lincoln at the White House.

[5] The Unionists had temporarily managed to avoid secession while Baldwin met with Lincoln, but their majority collapsed shortly after his return to Richmond.

He initially served as a militia colonel and inspector general of Virginia State Troops, accepting a commission on April 23, 1861.

After giving his loyalty oath to the federal government, he was elected to the Virginia House of Delegates under the post-war military provost in 1865-1867.

"[11] When the Virginia Constitutional Convention of 1868 proposed to restrict former Confederates from holding further offices, which went beyond the terms of surrender at Appomattox Court House and caused considerable controversy within the Commonwealth, Baldwin joined with his brother-in-law Alexander H. H. Stuart and the Committee of Nine and met with General Ulysses S. Grant.

Newly elected President Grant also met with provisional governor Henry H. Wells and businessmen Gilbert C. Walker and Franklin Stearns, then gave General (Secretary of War) John M. Schofield (and his successor General Canby) orders to allow separate votes on those two controversial provisions (each of which lost) and the new state Constitution without them (that passed overwhelmingly).