Swarthmoor Village was named after its location on the Swarth Moor, which was formerly a marsh-covered moorland.

The Swarth Moor's name was derived from the word sweart, meaning black.

[1] Before 1850 there were just a few farmsteads on Swarth Moor, and Swarthmoor Hall, which is located to the east of today's village.

George Fox (1624–1691), a founder of the Quakers, came to the area in 1652 and was later allowed by Judge Thomas Fell (1598–1658) to use Swarthmoor Hall as a meeting place.

[2] Situated on the outskirts of the village is Swarthmoor Hall, where in 1652 George Fox founded the Quaker movement.