Built on Gem Street in Gosta Green in Birmingham, it was a red brick and stone church designed by Thomas Rickman and Richard Charles Hussey in the Gothic style.

It was built to commemorate Henry Ryder, Bishop of Lichfield[2] and was consecrated in 1838.

[3] In 1925 the parish of St Mary's Church, Whittall Street, Birmingham was united with Bishop Ryder, and in 1939 part of the parish and the benefice of St Bartholomew’s Church, Birmingham, were united.

Gem Street also no longer exists, but the church was located in the middle of the modern Aston University campus.

The first organ in the church cost £270 (equivalent to £31,000 in 2023)[1] and was built by Theodore Charles Bates of London and opened on 7 December 1841.