[2] Located in the Ybor City Historic District, the building previously served as the second site of the Cooperative El Primero Progresso (officially the Agrupacion Benefica y Cultural del Centro Obrero),[3] or the Labor Temple, a place for Ybor City's cigar and restaurant workers to engage in union activities and organization.

[2] A different building—a two-story, white-stuccoed structure built in 1925 and located at 1614 8th Ave—had served as the Agrupacion Benefica y Cultural del Centro Obrero since 1928.

[10] It opened as a saloon under the Castle name that year on Guavaween,[2] an annual Halloween celebration that took place in Ybor City on the last Saturday of October.

[10] One night, former Castle manager John Landsman hosted a rave in the upstairs floor of the building, which was typically used as a space for union workers' birthday and retirement parties.

[2] An area on the ground floor that was previously used as offices for a cigar workers' union is now known as "the Dungeon", a dance and party space decorated in skulls and chains.

[2] According to Tom Gold, a founding resident DJ at the Castle, the flowing water feature was eventually removed due to clubgoers spilling drinks and vomiting into the moat.

[11] In 2017, Christopher Spata of the Tampa Bay Times described the Castle as an internationally renowned nightclub, and "one of the premiere dance clubs for goth and industrial music on earth.