[3] In 2001, he led Team USA to a gold medal game victory over Venezuela in the International Baseball Federation's AA World Youth Championships.

He graduated from Lakewood Ranch High School in Bradenton, Florida in 2003 after leading his team to the state 5A title his senior year.

[4][5] He subsequently transferred to Lakewood Ranch High School where he finished his education and amateur baseball career.

As a result of the incident, Milledge was passed over in the 2003 amateur draft until the Mets selected him as the twelfth overall pick in the first round.

The delay in signing Milledge prevented him from beginning his professional career until shortly before the end of the minor league season in 2003.

In 2004, he was slated to start the year with the Single–A Capital City Bombers, but he suffered a fractured metacarpal on his right hand in spring training, missing the first six weeks of the season.

In 65 games with the Bombers, he hit .337 with 13 home runs, 58 RBI and 23 stolen bases, earning him a promotion to the St. Lucie Mets in August.

Milledge made his major league debut on May 30, 2006, as the starting right fielder for the Mets in a 7–2 loss to the Arizona Diamondbacks.

In May, it was reported that Milledge appeared in a rap song, "Bend Ya Knees", by Manny D, a childhood friend.

Up until the end of June, he had been batting third in the lineup in place of the injured Ryan Zimmerman, posting a .245 average, 7 home runs, and 32 RBI for the season.

Because of his slow start, where he batted .167 with 1 RBI and no extra base hits in 24 at-bats, he was optioned down to AAA Syracuse on April 15, 2009.

On June 30, 2009, the Nationals traded Milledge and Joel Hanrahan to the Pittsburgh Pirates for Nyjer Morgan and Sean Burnett.

[11] Milledge, still rehabbing from an injury that occurred at Triple–A Syracuse when the trade was made, was assigned to the Pirates' Gulf Coast League team in Bradenton, Florida to complete his rehab, and then moved to the Pirates' Triple–A affiliate, the Indianapolis Indians, with a short stint at the Lynchburg Hillcats during the Triple–A all-star break.

When he was an active player, he would spend his offseasons attending youth football and basketball games in an effort to convince black children to give baseball a try.

[22] Following his retirement, Milledge opened Manatee Intercity Baseball in Bradenton, Florida, whose mission is to give minority kids an opportunity to learn and play the game.