The area became a transportation hub with rail service developed by Henry Flagler and was also a cigar making center that included Greek and Syrian immigrants.

In 1902, the state legislature passed a new constitution, adopting barriers to voter registration and voting that resulted in the deliberate disfranchisement of African Americans to exclude them from politics.

The southern part of the neighborhood was developed as a major railroad hub in the late 19th century; several rail lines met at Union Station (now adapted for use as the Prime F. Osborn III Convention Center).

A vibrant music and entertainment scene emerged, attracting many nationally renowned jazz artists to play at local black clubs on and off Ashely Street.

The Wynn/Egmont Hotel was known to provide the best accommodations for touring performers; the Boston Chop House, Mama’s Restaurant and Hayes Luncheonette served good food; the Lenape Bar and Manuel's Taproom were the favorite watering holes.

With the end of legal segregation following civil rights legislation in the mid-1960s, many residents left the area to pursue newer housing and work opportunities elsewhere.

During the 1980s the crack cocaine epidemic hit hard among struggling residents of LaVilla, resulting in an increase in crime and furthering the decline.

[8] According to General Counsel Rick Mullaney, who was chief of staff under Mayor Ed Austin, the area became "nothing but crack houses, prostitution and crime.

"[9] The 1993 River City Renaissance plan crafted by Mayor Ed Austin allocated millions of dollars to renovating and developing LaVilla.

The southern part of the neighborhood was once a major railroad hub, with several rail lines meeting at Union Station (now the Prime F. Osborn III Convention Center), and probably not coincidentally, was also for a period Jacksonville's primary red light district.