Fisher homesteaded in Edwinton (Bismarck) in 1872 and made his fortune in banking, real estate and liquor sales.

In 1955, the North Dakota legislature approved House Bill 761 providing $200,000 for a new governor's residence to be built because the current mansion was in a state of disrepair.

The architects were originally Leonhard & Askew of Bismarck,[1] but were replaced by Ritterbush Brothers when their design proved too expensive.

The 2015 North Dakota Legislative Assembly passed Senate Bill 2304, which authorized up to $4 million in special funds (for use on the Capitol grounds only) along with up to $1 million in private contributions for the demolition of the current Governor's residence and the construction of a new home on the same site (on the southwest corner of the Capitol grounds).

The former residence was found to have major structural, safety, and ADA compliance concerns, and estimates to repair the home came in at nearly $3 million, without addressing handicap accessibility issues.