He was born in Charlestown, New Hampshire on April 17, 1830 to Henry H. Sylvester and Elizabeth Hubbard.

He attended school at Phillips Exeter Academy, enrolled at Yale College with the Class of 1851 but left at the end of his sophomore year, and graduated at the Law School of Ann Arbor.

In 1854, he came to Iowa and continued his law studies with Judge James Grant and John F. Dillon at Davenport.

He went south and was for some time editor of the Memphis Argus, and secretary of the Howard Association.

He was the originator of the Memorial Bridge project over the Potomac to commemorate the war and link the North with the South.