Casserly was born in 1949 and grew up in River Edge, New Jersey before attending Bergen Catholic High School.

During his early years as a scout, he unearthed free agents Joe Jacoby and Jeff Bostic, who were original members of the famed "Hogs" offensive line and key components of Washington's first two Super Bowl teams.

The Washington Redskins elevated Casserly to assistant general manager in 1982 and the club went on to capture its first Super Bowl.

That experience was the subject of the Warner Bros. feature film, "The Replacements," that starred Keanu Reeves and Gene Hackman.

[citation needed] Promoted to general manager after Beathard resigned in 1989,[4] Casserly sustained the Washington Redskin’s history of uncovering high-quality players in the later rounds of the draft.

[citation needed] He used a fifth-round draft pick in 1990 to select Southwest Louisiana quarterback Brian Mitchell.

Washington Redskins then converted Mitchell into a running back/kick returner, where he later joined Jim Brown as the only players in NFL history to lead the league in combined net yards four times.

During the 1999 off-season, Casserly acquired veteran quarterback Brad Johnson who responded with a Pro Bowl season.

He still managed to obtain the player that Washington wanted, selecting future Pro Bowl and Hall of Fame cornerback Champ Bailey.

Mario Williams, who many criticized for being the #1 overall pick in 2006 (behind college standouts Reggie Bush and Vince Young), made the Pro Bowl in the 2008 season.

Casserly was criticized for a number of personnel moves, including trading second and third round picks to the Oakland Raiders for Phillip Buchanon in 2005.