[2][3] The facility is located on a 135,000 sf site within the former facilities of the Imperial Tobacco Company, of Great Britain and Ireland, The company had been a leader in the N.C. tobacco industry from the late 19th century, for about 65 years.

The tobacco company's 180-foot overhead steel conveyor, 80-foot water tower, and 90-foot brick chimney all remain, and are incorporated in the museum's logo.

While Imperial Tobacco constructed numerous facilities, from 1903 to 1923, from warehouses to curing furnaces, the building that is now the Imperial Centre itself, formerly known as the McDonald Building, housed the company's ordering operations as well as its hand and machine stemming work, where the central vein or stem of harvested tobacco leaves were removed.

When Hurricane Floyd inundated Rocky Mount in September 1999, many of the city's cultural facilities were destroyed, including its downtown Children's Museum and Arts Center as well as the Playhouse community theater in the Nashville Road area.

Rocky Mount chose to recreate a new facility at the Imperial Tobacco site.