[2][3] He was one of six brothers who represented Taranaki at cricket and rugby, including:[3] Alfred Bayly was educated at New Plymouth Boys' High School.

[1] He married Mary Eleanor Georgina Cashel on 30 September 1895, at St Paul's Church, Auckland,[4] and they went on to have five daughters, three of whom died as children.

[3] A utility back, Bayly made his first-class debut as a 16-year-old, appearing for West Coast (North Island) against the touring New South Wales team in 1882.

After missing selection for national side in 1896, Bayly was recalled in 1897 to captain the team on their tour of Australia, making nine appearances.

In all, Bayly played 20 games for New Zealand, including 10 as captain, scoring six tries and one drop goal for a total of 20 points.