His father was the goalkeeper on the Offaly and Leinster teams while his granduncles, Jim and Joe, played with the county and won two junior All-Ireland titles in 1923 and 1929.
Carroll grew up on the family farm and was educated at Coolderry national school and the Presentation College in Birr.
He enjoyed much success and won two senior county titles in 1977 and 1980, captaining the side in 1980 and winning the man of the match award.
The following year Carroll won his second Leinster title, a win which allowed Offaly advance directly into the All-Ireland final.
Galway provided the opposition in the championship decider and were the favourites, however, Carroll scored a vital in the first half to keep Offaly in the match.
Cork provided the opposition in the subsequent centenary year All-Ireland final at Semple Stadium.