His father, Ratliff Rodney Brewer, had been a farmer, plantation manager and overseer, and a captain in the Confederate Army during the Civil War.

His middle name came from his grandfather, Leroy Brewer (1793–1851), a Mississippi Delta pioneer who migrated from Elbert County, Georgia during the Treaty of Dancing Rabbit Creek.

His uncle Leroy Jasper Brewer (1833–1911), who was mayor of Holcomb at the time of Earl's gubernatorial election, died just weeks before his nephew's inauguration.

In 1924, after an unsuccessful Senate run, he took the case of Martha Lum, a local Chinese American girl who had been denied admission to the Rosedale schools since she was not white.

[5]: 99–106  Brewer appealed to the U.S. Supreme Court but was unable to handle the case at that point, so he handed it off to a younger lawyer, and the Mississippi Supreme Court's decision was upheld, with Chief Justice William Howard Taft writing an opinion that greatly broadened the scope of acceptable school segregation nationwide.