A native of Memphis, Tennessee, Brown was obtained by Detroit from the Negro league Kansas City Monarchs in 1961.

Brown spent eight years in the minor leagues, making it to the majors in 1969 after hitting .356 of that season for the Triple-A Toledo Mud Hens and hitting two home runs against the Tigers during an exhibition game.

His first major league hit was a home run at Yankee Stadium.

He also contributed to the Tigers American League East title in 1972, collecting a hit and two RBIs in two at-bats against the Oakland Athletics.

He was often mistakenly identified as the brother of roommate Gates Brown, to whom he bore no relation.