Of mixed race, Purvis and his brothers inherited considerable wealth from their native British father after his death in 1826.

Purvis's parents had lived in a common law marriage, prevented from marrying because his mother was a mixed race free woman of color, of Sub-Saharan African and Jewish descent.

His maternal grandparents were Dido Badaraka, a former enslaved woman, and Baron Judah, a Jewish American native of Charleston.

He described her as dark-skinned with tightly curled hair, she was most likely of Sub-Saharan African descent as evidenced by her name and her physical description.

[2][3] Harriet's father was Baron Judah, born in Charleston of Ashkenazi and Sephardi Jewish descent.

[1][5] William Purvis and Harriet Judah lived together as husband and wife, but racial law prevented their marriage.

William intended to consolidate his business affairs and return with his family to Britain, where he thought his sons would have better opportunities.

Robert and Joseph inherited increased shares of the estate; they used their wealth to support their political activism and public service.

[citation needed] In 1833, Purvis helped abolitionist William Lloyd Garrison establish the American Anti-Slavery Society in Philadelphia and signed its "Declaration of Sentiments".

Free men of color in Pennsylvania did not regain suffrage until after the ratification of the Fifteenth Amendment in 1870, following the Civil War.

As a supporter of the Underground Railroad, Purvis served as chairman of the General Vigilance Committee from 1852 to 1857, which gave direct aid to fugitive slaves.

Byberry Hall, which still stands today, hosted anti-slavery speakers and was across the street from Purvis's home.

Purvis was a friend of both Thomas and his son William Henry Dorsey, who was an artist and collector of Black history.

William compiled hundreds of scrapbooks of Black history during the 19th century and built a collection that he laid out in his home in Philadelphia.

By the end of the Civil War, which gained the emancipation of slaves and suffrage for black men, Purvis had reached his late 50s and became less active in political affairs.

The mostly Irish rioters set fires and attacked firefighters and police as they went, heading for Purvis's home, where they protested outside for forty hours.

Purvis and his home were reportedly saved from the Irish mob solely by a Catholic priest's intervention.