Born in Wolverhampton, West Midlands,[7][8] Goldson attended Thomas Telford School[9] and was awarded his first professional contract midway through his youth scholarship with Shrewsbury Town in May 2010.

[11] With captain Ian Sharps suspended,[9] he made his full debut on 1 March, in a 3–0 home defeat to Bury,[12] and featured sporadically the following season as Shrewsbury won promotion to League One.

[17] Although his loan was later being extended by a further month,[18] Goldson was recalled by his parent club on 9 January 2014 due to an injury to fellow defender Darren Jones.

Despite receiving offers from Premier League clubs, Goldson signed a new two-year contract extension on 4 July 2014, having been convinced to stay by new manager Micky Mellon.

[20] He captained Shrewsbury Town for a whole match for the first time on 12 August 2014, in a League Cup first round victory over Championship club Blackpool, in the absence of regular skipper Liam Lawrence from the starting eleven.

[27] He was first included in a Seagulls squad a month later, as an unused substitute in a goalless Championship draw at his hometown club Wolverhampton Wanderers, a result which kept his team at the top of the table.

[28] After Brighton's 2–2 draw at Derby County on 12 December, Goldson alleged racial abuse by a home fan at Pride Park, which was investigated by the police.

[37] Goldson was on the fringes of Brighton's team in 2017–18, totalling only eight appearances, but scored in their 3–1 home win over Coventry City in the FA Cup fifth round on 17 February 2018.