Rapp's major league debut was on July 10, 1992, as he pitched two scoreless innings against the Montreal Expos.

During the strike shortened 1994 season, Rapp compiled a record of 7–8 with a 3.85 ERA, two complete games and his first career shutout.

After starting the season 4–6 with an ERA of 4.47, Rapp was traded to the team which first drafted him, the San Francisco Giants for minor league pitchers Brandon Leese and Bobby Rector.

The Royals granted Rapp free agency in December 1998 and he signed another one-year deal, this with the Boston Red Sox on January 11, 1999.

After his one season in Boston, Rapp was granted free agency on November 2, 1999, and signed with the Baltimore Orioles on January 28, 2000.

Rapp started the 2000 campaign 3–0, then learned that his father was ill with a lethal bacterial infection, sustained from cutting his leg while fishing.

Emotionally drained, Rapp returned to the Orioles on August 18 and pitched against the Royals, retiring 17 consecutive batters from the first inning onwards.

Rapp's final season in the majors proved to be 2001; he finished the campaign with a record of 5–12 and an ERA of 4.68, with one complete game.

General manager Dave Littlefield said that Rapp was not given his unconditional release due to the fact of his complaints but because he "Could not pitch deep into games.