He started his first business in Madison, Georgia, forming the firm of High & Berney.
He later moved to Atlanta for greater opportunity, where he partnered with E. D. Herring to open up a dry goods store at 40 Whitehall Street (now Peachtree St.
In 1887 High's department store opened a new location on the west side of Whitehall.
By 1889 High was able to buy the southwest corner of Whitehall and Hunter (now King Blvd.)
and erect a four-story brick building with an ornate interior for his department store.