Baker grew up on a wheat and sheep farm in Pingaring, Western Australia, a town 341 kilometres (212 mi) east of Perth.

[9] Following that season Baker was one of just 12 Western Australians and 80 young players across the country to be invited to the national draft combine in Melbourne in October.

[12][13][14] Baker was passed over in national, pre-season and rookie drafts that year, before making a switch to play senior football with rival WAFL club Subiaco in 2017.

[15] At the time he recognised the need to improve physically, citing poor sprint testing at the national combine as a factor in his failure to be drafted.

[6] In addition, he worked on improving his inconsistent kicking caused by a bad ball-drop technique and a lack of positional versatility that saw him play exclusively as an inside midfielder.

[17] He played a total of 13 matches at senior level including a "shining" performance in Subiaco's losing grand final in which he recorded 19 disposals, six inside-50s and five tackles.

[29][30] Baker earned an AFL debut on the back of the performance, turning in 12 disposals, five score involvements and two behinds in his side's round 19 win over Collingwood at the MCG.

[31][32][33] He held his senior spot the following week, adding 14 disposals and another two behinds before being omitted from the club's round 21 side to face Gold Coast.

[41][42] He accepted a one match suspension for the act and subsequently missed the club's losing semi-final against Essendon that saw the VFL side eliminated from the finals series.

[75][76] After initially being named by AFL Media as a candidate to fall out of the club's best 22, the mid-December retirement of fellow defender Alex Rance saw Baker increasingly likely to retain his spot in the Richmond backline.

[80][81] Baker recorded 16 disposals in a round 1 win over Carlton when the home and away season began, but under extraordinary conditions imposed on the league as a result of the rapid progression of the coronavirus pandemic into Australia.

[88][89] Baker contributed 12 disposals when the season resumed in June after an 11-week hiatus, before missing round 3's match against Hawthorn to attend a funeral in his home state of Western Australia.

[23][94][95] He contributed a best on ground performance in round 10, earning nine coaches association award votes for 580 metres gained and a then-career-high 26 disposals.

[107][108][109] After a season in which he played 20 of a possible 21 games and won another premiership medal, Baker also placed sixth in the club's best and fairest award.

[23][111][112] Despite recording 27 disposals and being named by AFL Media as one of his side's best player while in defence the week prior, Baker was shifted to half forward from round 6 in replacement for the injured Kane Lambert.

[23][113][114][115] Baker played in his 50th career match in round 8's loss to Geelong, before the next week being named as a full-time midfielder in light of injuries to Shane Edwards, Shai Bolton, Trent Cotchin and Dion Prestia.

[116][117][118] He was outstanding in the role, recording career-highs in disposals (33), inside-50s (nine) and clearances (seven) to earn eight coaches votes as second-best on ground during the comeback win over Greater Western Sydney.

[123][124] Following the 2024 AFL season, Baker requested a move back to his home state of Western Australia, later naming West Coast as his club of choice.