The Glenelg Manor was built on a part of land patented as "Dorsey's Grove" in 1721.

The original structure of the house dates from circa 1740 to the second half of the 18th century, and may have been built by Ephraim Howard.

[1] The original building was rented in 1954 when the Glenelg Country School was founded by Kingdon Gould, Jr. and his wife Mary Thorne Gould, along with Mr. and Mrs. John T. Mason, Jr., Judge James Macgill and Mr. and Mrs. William Shippen.

Subsequent Headmasters/Headmistresses: Beatrice Pfefferkorn (1956–1959); Edward L. Jones (1959–1964); Peter T. Terry (1964–1966); Thomas J. Barlow (interim head in 1966 for Peter Terry); Frederic W. Rhinelander (1966–1977); Charles H. Miller, Jr. (1977–1990); Ryland O. Chapman III (1990–2007); Gregory J. Ventre (2007–2021); and Matthew J. Walsh (2021–present).

In 2014, County Executive Ken Ulman proposed CR-121-2014 in his last weeks of pre-election activities.