He joined the United States Army in 1943 and served in the 69th Infantry Division in the European theatre of World War II.

Under special legislation passed during the fruit-fly crisis, the department was authorized to seize or to destroy infested products and to stop interstate and intrastate traffic to enforce the law.

He even famously put his hand into a fire ant mound at the urging of a television reporter while news cameras rolled.

However, his opponent in the 1982 Democratic primary election, the liberal journalist and commentator Jim Hightower, accused him of manufacturing the fire ant crisis to win reelection.

Brown was also embarrassed when during a public speech in April 1982 he called Booker T. Washington, "That great black nigger, er, educator."