[4] Carroll spent the final four years of his playing career with Bonbeach, serving as Player-coach of the club and retiring as a player in 2008.
Shortly after retiring from football as a player, Carroll became the new coach of TAC Cup team Gippsland Power.
In just his second year as coach of Gippsland, Carroll guided the Power to the TAC Cup Grand Final in 2010, losing to Calder Cannons 116–58.
[7][8] Carroll's first season as Box Hill's coach was marked with inconsistency, with the Hawks winning 7 games and losing 10 to finish in 8th place on the VFL ladder, narrowly qualifying for the VFL Finals, where the Hawks were defeated by Werribee 89–63 in the first week of the finals.
Despite going into the match as underdogs, the Hawks defeated the Cats 99–78 to achieve just their second VFL premiership, and their first since 2001, in front of a crowd of 15,119 people at Etihad Stadium.
[9] As a result of this achievement, Carroll was appointed Head Coach of Australian Football League club Hawthorn's Development Academy.
[5] John played just 4 seasons in the VFL, making the league's team of the year on each occasion, before retiring prematurely due to a debilitating back injury after moving to South Australian club Woodville-West Torrens.