The house is a wooden, three-story building in Colonial Revival style located on a hill on Coaster's Harbor Island, overlooking Coaster's Harbor, Dewey Field, and Narragansett Bay.

The house was built in 1896 by local Newport, Rhode Island, architect and builder Creighton Withers at the cost of $16,226.

Built originally as "Quarters B" for the commandant of the Naval Training Station, Newport, the first president of the Naval War College to occupy it was Rear Admiral French Ensor Chadwick, in June 1903.

Every Naval War College president except for Charles Stillman Sperry (1903–1906) has lived in the house since that time.

Among the most famous residents of the house have been Admirals William Sims, Raymond A. Spruance, Stansfield Turner, and James Stockdale.