It was originally located next to St Stephen's Chapel and was rebuilt and enlarged by James Wyatt in the early 19th century.

After the burning of Parliament in 1834 it was rebuilt by Charles Barry as part of the new Palace of Westminster in the Perpendicular Gothic Revival style.

[8] At a dinner at the home of Lord Camden in 1808, The 1st Duke of Cumberland and Teviotdale told Abbot that the Whig MP George Tierney would be complaining in the Commons about the expenditure of £70,000 on the Speaker's House (equivalent to £8,074,430 in 2023).

[12] After the burning of Parliament in 1834, it was rebuilt by Charles Barry as part of the new Palace of Westminster in the Perpendicular Gothic Revival style.

[17] The Speaker's House was described in the 1878 book Old and New London as "of considerable extent, comprising from sixty to seventy rooms" with the "staircase, its carvings, tile-paving, and brass-work, is exceedingly effective and elegant, and everywhere there is a large amount of painted and gilded decoration".

[12] The court is entered by two "not very imposing" archways, as described in an article in The Illustrated London News, which said that "spacious as the area which presents itself, and lofty as are the buildings which form its four sides, the appearance of the house as a whole is not particularly striking".

A band above the arch is inscribed with the Christian text "Domine salvum fac regem" ("Lord save the king").

A grand staircase leads from the entrance hall; tall standard lamps adorn the steps at the bottom of the stairs.

Its ceiling is divided into richly carved and gilded bays, with square panels bearing the arms of the Houses of York and Lancaster, and the Portcullis of Westminster.

It is richly decorated with emblems of three kingdoms, the arms of Queen Victoria, crowns and the portcullis with the monogram "V.R.".

[12] The Speaker's House was refurbished in the 1980s under Sir Robert Cooke, who served as Special Advisor to the Palace of Westminster from 1979 until 1987.

[13] A canopied bed in the Speaker's House is intended for the British monarch to sleep in the night before their coronation.

The case was argued before the Lord Chancellor, John Copley, who ruled that Manners-Sutton's claim was unsustainable as the Crown could not be held responsible for the negligence of its agents.

[21] A collection of painted portraits of the speakers dating back to the 1800s are displayed in a function room in the residence.

[13] During their 1956 visit to the UK, the Soviet First Secretary Nikita Khrushchev and the Soviet Premier Nikolai Bulganin attended a dinner at Speaker's House with Speaker William Morrison and 39 others, including Prime Minister Anthony Eden, Leader of the House of Commons Rab Butler, Foreign Secretary Selwyn Lloyd, Leader of the Labour Party Hugh Gaitskell and Lord Chancellor Viscount Kilmuir.

[31] Boothroyd met her waxwork dummy at Speaker's House in June 1998 before its unveiling at Madame Tussauds.

Bercow was elected speaker after the resignation of Michael Martin in the wake of the MPs expenses scandal.

Bercow personally paid for a children's climbing frame and a Wendy house for Speaker's Green.

[19] Bercow's wife Sally described the view from Speaker's House as "incredibly sexy, particularly at night with the moon and the glow from the old gas lamps".

[34][35] The Aber Valley Male Voice Choir celebrated their golden anniversary with a performance at Speaker's House in 2009.

[36] Pupils from the London Welsh School sang songs at the door of Speaker's House to celebrate St David's Day in March 2015.

The practice was maintained by John Bercow during his speakership when an event to mark the centenary of the 1914 Home Rule Bill was due to be held at his residence with attendees invited by the Irish embassy in London.