After relocating to Paris, France, Brown became known within the gay nightlife of the time for his flamboyant lifestyle and his interest in the arts, performing in a cabaret.

By his seventh fight, December 13 of that same year, he beat Sailor Patchett by a fifteen-round decision, to earn the Panamanian 'Isthmus' flyweight title.

[7] Brown began campaigning extensively across the United States before he suffered his first loss, at the hands of Jimmy Russo on December 6, 1924, by decision in twelve.

He would later avenge that defeat, and he beat Davey Abad and Willie LaMorte before being disqualified in the first round against Frankie Murray on June 11, 1925.

Despite that setback, he kept on campaigning successfully, fighting for the first time in his career at Madison Square Garden on May 21, 1926, against Teddy Silva.

He beat Gregorio Vidal by a fifteen-round decision to win the vacant NYSAC and lineal bantamweight titles, at the Queensboro Stadium, Long Island.

[15] On February 19, 1934, Brown defended his IBU title against Young Perez at the Palais des Sports, Paris, in what would be the first of three encounters between the two boxers.

On June 1, 1935, Brown lost the title to Baltasar Sangchili by a fifteen-round decision, at the Plaza de Toros, in Valencia, Spain.

Suffering from the prolonged effects of drug use, he was persuaded by Jean Cocteau to detox, receiving treatment at the Sainte-Anne Asylum,[19] and begin training for a comeback to boxing.

[20] He had a rematch with Sangchili on March 4, 1938, avenging his earlier loss with a fifteen-round decision to win the vacant IBU bantamweight title,[18] but by this time the International Boxing Union was no longer recognized in the United States.

[9] His rematch win over Sangchili is believed to be his last great night, and, bowing to Cocteau's wishes, Brown vowed to retire after one more fight.

With the advent of the World War II, Brown moved to the United States, settled in Harlem and tried to find work of the cabaret sort he performed in Paris when not fighting.

[9] Brown went on fighting until 1942, challenging unsuccessfully for the Panamanian Featherweight title on September 30, 1942, when he drew with Leocadio Torres, but retiring as a winner, defeating Kid Fortune by a decision in ten rounds on December 4 of the same year.

[2] He went back to New York afterward and, in his late 40s, took a lot of beatings while serving as a sparring partner for up-and-comers at a gym in Harlem, making a dollar a round.

He died there on April 11,[2] unaware that not long before, one of the newspapers in Paris had begun talks about organizing a fund drive to pay for his trip home.

[25] He joined Josephine Baker's La Revue Nègre as a tap-dancer, and made his cabaret debut as a song-and-dance man at the Caprice Viennoise.