Oliver's professional debut for Hearts was in a Scottish League Cup match against Raith Rovers on 27 August 2013, coming on as a substitute in the 55th minute, replacing midfielder Scott Robinson.
[2] Four days later Oliver made his league debut for the club, coming on as substitute for Callum Tapping in the 61st minute in a 2–0 defeat to Inverness Caledonian Thistle.
[3] Under the management of Gary Locke, Oliver was one of the youngsters to get an opportunity in the first team, as a result of Hearts entering administration, as well as having an injury crisis at that particular time.
[10] On 19 December 2014, Oliver's loan spell at Ochilview ended early, as he was recalled by Hearts after appearing in 11 league matches and scoring two goals.
[13] In November 2018, Oliver sustained a broken collarbone in a challenge with Peterhead's ex-Ton midfielder Jamie Stevenson, that kept him out of first-team action for two months.