Born in New Orleans, Louisiana, he was the son of former MLB outfielder and longtime minor-league manager Larry Gilbert; his brother Tookie also played in the majors.

Charlie Gilbert served in the United States Navy in the Pacific Theatre of World War II.

In his third MLB game, on April 23, 1940, he hit a pair of home runs against the Boston Bees in an 8–3 Brooklyn victory at Ebbets Field.

[2] He thus became the first player, and one of three men in the history of the Dodgers' franchise, to have a multi-home-run game in his first five starts; the others are Cody Bellinger and Yasiel Puig.

Gilbert's daughter Jan went on the memorialize her late father in Goodbye, Dad, a series of mixed-media works utilizing family photographs.