Brannan held command of the Department of Key West at Fort Zachary Taylor, Florida, part of the Union effort to hold federal installations within Confederate territories early in the war.

Brannan was scandalized by the highly publicized disappearance of his first wife, Eliza Crane Brannan, daughter of Colonel Ichabod B. Crane, in 1858; she mysteriously disappeared after taking a ferry from Staten Island to Lower Manhattan and was initially presumed to have committed suicide or been murdered, but it was later discovered that she had secretly fled to Europe and married another United States Army artillery officer, First Lieutenant Powell T. Wyman.

[2] Shortly after graduation from the military academy, Brannan joined his artillery company at Plattsburgh, New York, where he was present during the border dispute with Canada during that time.

[3] During the Mexican–American War, Brannan's artillery company joined the siege of Veracruz; from there, he participated in skirmishing at La Hoya and the battles of Cerro Gordo, Contreras and Churubusco.

[3] Upon the outbreak of the Civil War in 1861, Brannan was appointed a brigadier general of United States Volunteers on September 28, 1861.

He left his artillery company on detached service for the duration of the war, never returning before he was promoted to the Regular Army rank of major on August 1, 1863.

He received a Regular Army brevet promotion to the rank of lieutenant colonel for his service during the battle for Jacksonville, Florida.

In 1863, Brannan led an infantry division under Major General William Rosecrans in the Tullahoma Campaign where he fought at Hoover's Gap.

[3] After the Civil War, Brannan was mustered out of the volunteer forces and reverted to his permanent Regular Army rank of major with the 1st U.S.

While Brannan was posted in Key West, Florida, from 1856, Eliza and their daughter lived in Staten Island, New York, with her mother.

In 1860, however, Eliza Brannan contacted her brother (Dr. Charles Crane) and notified him that she was, in fact, alive: having originally fled to Italy, she was now remarried and living in Paris, France.

In 1860, First Lieutenant Wyman was denied a leave of absence to visit his lover, and instead resigned his commission and traveled to Italy to join (and later marry) Eliza Brannan.

At some point between 1860 and 1862, Wyman and Eliza Brannan returned to the United States, where they lived openly as a married couple.

Meanwhile, Powell Wyman accepted a volunteer commission as colonel of the 16th Massachusetts Infantry in August 1861, and he was killed in action in June 1862 at the Battle of Glendale.