While at KB, he made his debut for the Danish youth national teams (in November 2000), and played four matches at the 2002 UEFA European Under-17 Football Championship.

He debuted on 23 April 2005 in a match against FC Nordsjælland where he replaced an injured Ole Tobiasen in the right-back position.

Kvist was called up for the national team by manager Morten Olsen for a friendly tour of the United States, El Salvador and Honduras in January 2007.

He was in the starting lineup for all three games of the tour, including a 3–1 loss to the United States national team on 20 January 2007.

Kvist's form led to the club's captain (and player with most appearances), Hjalte Bo Nørregaard, losing his position in the first eleven.

[11] On 1 September 2014, Kvist moved on a free transfer to Championship side Wigan Athletic signing a one-year deal.

[15] At the same time, he would also sit on the board of Copenhagen's e-sports venture, North, just as he would help the A national team with the negotiations with Danish Football Association.

Copenhagen manager Ståle Solbakken was fired on 10 October 2020, Kvist was appointed interim director of sports for the club.