Little is known of Fysher's early life, beyond the fact that he was baptised in Grantham, Lincolnshire, on 30 December 1698, the home town of his father, who was also called Robert.
In 1715, at the age of 16, he matriculated at the University of Oxford as a member of Christ Church, obtaining his Bachelor of Arts degree in 1718.
He transferred to Oriel College, Oxford in 1723, was promoted to Master of Arts in 1724, and was awarded a Bachelor of Medicine degree in 1725.
As well as his work as Librarian, he assisted Oriel College by acting as Dean for three terms (1727, 1731–1732 and 1734–1735) and as both Junior and Senior Treasurer.
Ill-health towards the end of his life inhibited his work at the library; he died on 4 November 1749 in Sevenhampton, Wiltshire, and was buried in the University Church of St Mary the Virgin.