He was the son of Sir William Graham of Kincardine by Lady Mary Stewart, daughter of King Robert III of Scotland.
Before rising to the rank of bishop, Patrick for many years controlled the parish church of Kinneil.
Although Patrick paid for the bishopric of Brechin, his election was acknowledged by Pope Pius II, who appointed him to the see sometime before 29 March 1463.
He commissioned one John Huseman, Dean of the church of St. Patroclus in Soest in the diocese of Cologne, to investigate charges (of insanity) made against Archbishop Patrick.
The result was that Archbishop Patrick was condemned to confine himself to a monastery, residing first at Inchcolm, then Dunfermline, before being imprisoned in Lochleven Castle.