Bayliss was originally a seventh-round draft pick of the Kansas City Royals in 2002, and he made his major league debut with that team on June 21, 2005.

He and minor league reliever Chad Blackwell were traded to the Pittsburgh Pirates for starting pitcher Mark Redman on December 7.

Bayliss did not make the Toronto Blue Jays 40-man roster and started the 2009 season with the Triple A Las Vegas 51s.

On July 28, 2009, Bayliss was released by the Blue Jays so that he could sign with The Saitama Seibu Lions of the Japan Pacific League.

On December 15, 2010, Bayliss got out of Free Agency and the Tampa Bay Rays inked him to a minor league deal.