[1] William Lawrence Scott was born on July 2, 1828, in Washington, D.C., to Mary Ann Lewis (died 1879) and Colonel Robert Scott (U.S. Army) (1798–1835),[2] of Virginia, a graduate of the United States Military Academy at West Point, who was detailed to the nation's capital at the time of his son's birth.

His elder brother, Robert Wainright Scott (1827–1866),[4] was educated at the Naval Academy at Annapolis, served with distinction in the U.S. Civil War, and died while commander of the USS Saginaw at Acapulco, Mexico, on January 5, 1866.

He served as chairman of the United States House Committee on Expenditures in the Department of the Navy during the Fiftieth Congress.

[11] In June 1883, Scott bought the 2,650-acre (10.7 km2) Hollywood Farm on the Chesapeake Bay in Virginia from the heirs of the late Governor Littleton Tazewell for $55,000.

Scott bought the land primarily to establish a terminus, a harbor and a town for the services of his railroad, the New York, Pennsylvania and Norfolk.

Scott immediately deeded part of his 2,650-acre (10.7 km2) purchase to the railroad and the following year, in 1884, he laid out the Town of Cape Charles, Virginia, on 135 acres (0.55 km2).

[18] He was interred in a mausoleum designed by E.L. Pelton, an Erie architect, completed in 1889 at a cost of $40,000 (equivalent to $1,356,444 today).

They divorced in 1925 and later that same year, she married Sumner Welles (1892–1961), the Under Secretary of State from 1937 to 1943 during Franklin D. Roosevelt's presidency.