[1] The building was initially constructed starting in 1908 by Emerson Monroe Scarbrough, Sr., a former Confederate soldier, to house the flagship location for his department store.

[3] In 1893 Scarbrough left R.H. Hicks in charge of the store and moved with his family to Austin, expanding the business to a new location in the Kreisle Building at the 400 block of Congress Avenue.

[9] In 1931, the Scarbrough sons hired architects Wyatt C. Hedrick and Edwin C. Kreisle for a renovation of the building.

The Scarbrough Building was completely remodeled in an Art Deco facade, with shop windows widened from 6 ft. (2 m).

Punch Bowl Social, an entertainment venue, signed a lease for the ground floor and basement level of the building in 2019.