Due to Belgium's occupation by Nazi Germany during World War II, however, the building was instead donated to the Virginia Union University in Richmond.

The complex served first as a new soldier processing location for the United States Army, then later as a gym, library, and classroom space for Virginia Union.

[3] The red tiles and black slate that comprise the structure's exterior were sourced from Flanders and the Ardennes, respectively, and the plate glass that features prominently at the site came from the Walloon Region.

[3][5] The tower's base features two Art Deco bas relief panels representing the culture of the Belgian Congo.

[7] Former president Herbert Hoover was the guest of honor at the opening night dinner, and was praised for his humanitarian efforts in Belgium during the First World War.

[4] The gift of the $700,000 building still required significant fundraising by the school, as the process of disassembly in New York, transport, and reconstruction in Richmond was expected to cost $800,000.

[14] It was eventually rebuilt largely as it appeared in New York and adapted to the university's needs, though fund shortages necessitated the complex to be constructed in phases.

[12] The tower, located on the southwest corner of the structure, was named after Robert Lee Vann, a Pittsburgh-area publisher and alum of Virginia Union.

[5] During the building's gradual reconstruction on the Virginia Union campus in the 1940s, it served as a processing station for the United States Army.

The bas reliefs carved into the tower proved controversial around this time; some felt the depiction of the Belgian Congo glorified colonialism, and concerns about the figures' nudity resulted in shrubs being planted to shield the art from the street.