[1] He began his career with Westmarch before joining local Scottish Football League Second Division club Abercorn.

[5] Having spent time with Kilbarchan on loan,[6] Dalrymple scored on his debut for Hearts in a 1–1 draw with Edinburgh rivals Hibernian on 11 October 1902.

[7][8] Dalrymple received a runners-up medal after Hearts lost the second replay by two goals to nil on 25 April 1903, with 32,000 spectators in attendance.

[11] He made his debut in a 1–0 win at West Ham United on 1 September 1903,[12] and scored his first goal at the end of the month as the club defeated Portsmouth 3–1.

He scored 44 goals in 106 appearances over the next three years,[18] and helped the club reach the semi-finals of the FA Cup in 1908, where they were beaten 6–0 by Newcastle United at Anfield.