He was given the nickname "Sailor" by teammates due to his "rolling gait and muscularly stocky build".

[3] Brown was born on 7 November 1915 in Great Yarmouth, Norfolk and was educated at St Peter's and Priory School.

[2] He played for Great Yarmouth Town and Gorleston as a youth,[2] before making a move to the Football League with Charlton Athletic in August 1934.

[4][5] During the war, he served as a sergeant in the Royal Air Force and was a member of the Greenwich auxiliary police.

[6] Following the war, Brown played for Charlton in the FA Cup Final on 27 April 1946, which was lost 4–1 to Derby County at Wembley Stadium.