In 1794 Pryse was commissioned as an Ensign into the Berkshire Militia (of which his father was the lieutenant-colonel)[4] and served as High Sheriff of Cardiganshire in 1799.
[2] By 1812, he became involved in the contest for Cardigan Boroughs, and proposed Herbert Evans as candidate against the sitting member, John Vaughan of Crosswood.
Roland Thorne suggests that had Pryse, as heir to the Gogerddan interest, wished to stand himself, Evans would have withdrawn in his favour.
A county meeting was held at Cardigan and a vote taken between the two candidates, Pryse and William Edward Powell of Nanteos.
In order to 'preserve the peace of the county', Pryse then withdrew on the understanding that Powell would support his candidature when a vacancy arose in Cardigan Boroughs.
[6] His father, Edward Loveden, died on 6 January 1822, and Pryse inherited a life interest in his Berkshire estates.
[1] During his thirty-year parliamentary career, Pryse was regarded as a supporter of moderate reform who advocated Liberal principles.
[11] The campaign in Aberystwyth was reported to be strongly in favour of Pryse, where the "overwhelming feeling [was] the injustice of turning out a long tried representative to make room for a thorough stranger".
In Cardigan too, Pryse was greeted by crowds of supporters and his colours were flown from the masts of ships in the harbour.