Williams wrote The Journal of Llewellin Penrose, Seaman, believed to be partly autobiographical, about a Welsh sailor who is cast away in the Americas.

Particularly of note to recent historians are three portraits tentatively attributed to him at the beginning of his career: the Mohawk chief Theyanoguin in 1755, the radical abolitionist vegetarian Benjamin Lay around 1758, and the Black astronomer Francis Williams around 1760.

Traveling with the Hallam Company he lived for a time in Jamaica, where he possibly painted Francis Williams, and afterwards in New York City, where he was married to Mary Mare, the sister of artist John Mare Jr. Williams returned to England in 1776, where he lived in London for some time and renewed his friendship with Benjamin West.

Williams left his personal property to Thomas Eagles, a gentleman who helped him gain admission to the almshouse, where he lived until his death.

John Eagles, son of Thomas, prepared an edited copy of this for publication, including 37 watercolours mainly by Nicholas Pocock, intended for engraving.