Not receiving Senator James Buchanan's appointment to West Point (which instead went to future Civil War general William B. Franklin), Haller responded to a summons to go to Washington, D.C., where he was commissioned as a lieutenant in the U.S. 4th Infantry Regiment.

Haller fought Seminole Indians in Florida in 1840–1841 and later served with distinction at Monterrey, Veracruz, and other battles during the Mexican–American War, officering in the same regiment as Ulysses S. Grant.

Wells alleged that Haller had toasted him "Here's to a Northern Confederation and a Southern one while Lincoln is President", blamed the President for casualties at Fredericksburg, and in a later dispute offered that a "Black Republican" would be a more suitable quarters-mate for him; Haller denied giving such a toast entirely, and protested that he did not blame the government for "disasters" at Fredericksburg but only the change in strategy caused by Lincoln's removal of General George B.

[2] He and his wife eventually returned to the American West, settling on Whidbey Island in Coupeville, Washington Territory in 1866 where he built a home on Front Street and started a business that extended credit to pioneer families.

After retiring from the Army in 1882, Haller returned to the Pacific Northwest and settled in Seattle where he built a three-story, eighteen-room mansion named "Castlemount" and became part of the growing city's business and industry community.

The home, one of thirty other buildings on Whidbey Island left from the area's early settlement era dating between the 1850s and 1870s, has been the subject of a campaign by a local historic preservation group seeking to purchase and restore the house.