He was signed as an undrafted free agent by the Pittsburgh Penguins in 2014 and won back-to-back Stanley Cups in his first two seasons with the organization.

On March 13, 2016, Sheary recorded his first multi-goal game, scoring 2 goals against the New York Rangers at Madison Square Garden against goaltender Henrik Lundqvist.

On June 1, 2016, Sheary scored the game-winner, in overtime, to win against the San Jose Sharks 2–1 in the Stanley Cup Finals.

In the 2016–17 NHL season, Sheary continued to play alongside Sidney Crosby for the entire year, getting 23 goals and 30 assists for 53 points.

[5] On June 27, 2018, Sheary, along with teammate Matt Hunwick, were traded to the Buffalo Sabres for a conditional fourth-round pick in the 2019 NHL Entry Draft.

[6] In the 2018–19 season, Sheary played in a top-nine forward role, contributing with 14 goals and 34 points through 78 games as the Sabres missed the post-season.

At the NHL trade deadline, Sheary was re-acquired by the Pittsburgh Penguins along with Evan Rodrigues in exchange for Dominik Kahun on February 24, 2020.