The building, like many others built in Madison during the early 1900s, was designed in the Beaux-Arts style.

The building was developed by Leonard Gay, for whom it was originally named, and designed by architect James R. Law, Jr., who later served as mayor of Madison from 1932 to 1943.

Because the height of the building interfered with views of the Capitol, its construction drew opposition.

The city's landscape architect, John Nolen, led an unsuccessful campaign to stop its construction.

After the building was completed, a 90-foot (27 m) height limit was enacted for buildings around the Capitol; as a result, the Gay Building remained Madison's tallest (other than the Capitol) until the Wisconsin Supreme Court struck down the height limit law in 1923, which allowed construction of the taller Belmont Hotel.