A church in the locality is first recorded in the 12th century, though the date of its founding and its dedication are now unknown.

At this time, Whitchurch was a chapel of ease to Llandaff Cathedral, and would remain so until the 19th century.

In the 15th or 16th century, a porch was added, followed later still by a nave, then a belfry of Bath stone.

[1] As Whitchurch gradually evolved into a suburb over the 19th century, the population of the district continued to expand.

Clarke, who became the vicar of St Mary's in 1876, was dissatisfied by the inadequate (and increasingly rundown) church, and began to agitate for a replacement.