Born in Liverpool, Merseyside, Bayliss joined Rochdale and progressed through the ranks, making his Football League debut in the 1994–95 season.

He spent four seasons at Kenilworth Road, making 37 league appearances and also had nine games on loan to Chester City before being released in the summer of 2005.

Bayliss remained registered as a player until the end of the 2010–2011 season, though his appearances were largely restricted to low-profile cup matches or filling up the substitutes bench.

Following allegations made against him, Darren Sheridan was first suspended and then left Barrow in February 2012, leaving Bayliss as the sole manager of the club.

As of May 2020[update], Bayliss was working as a scout for Millwall, alongside a part-time job with the NHS in ex-offender rehabilitation.