Holly attended Clairton High School where in his senior year made 45 receptions for 900 yards and seven touchdowns.
During his time there he played in 49 games making 85 tackles, nine interceptions and one touchdown on defense.
Holly was selected by the San Francisco 49ers in the seventh round (215th overall) in the 2005 NFL draft.
In his first season with the Browns he made 14 appearances and 12 starts during which he made 56 tackles and five interceptions which led the team, (one of his interceptions was returned for a touchdown versus the Pittsburgh Steelers on November 19).
Despite tearing two knee ligaments during offseason workouts on May 21, the Browns did not rescind their tender and Holly signed it the day after his injury.