After initially playing for Wealdstone, Baptiste made his breakthrough with Hayes in 2000, where he scored six goals in 14 league games and caught the eye of Luton Town.
In the summer of 2003, Baptiste joined Stevenage Borough, following manager Graham Westley's move, where he stayed until February 2004 scoring four league goals.
In September 2008, he left Havant & Waterlooville to play for Maidenhead United, spending the first half of the season there before signing for AFC Wimbledon on 31 March 2009.
While playing for Harrow, Baptiste infamously missed an open goal after taking the ball past the opposition defence and goalkeeper.
Baptiste scored on his debut for Isthmian League team Wingate & Finchley from the substitute bench on 20 October 2012 against Lewes in a 3–1 defeat.