His grandfather, Sir John Wedderburn, 5th Baronet of Blackness, was involved with the Jacobite rising of 1745, and was convicted of treason.
The punishment for this was threefold: the death penalty, the confiscation of all his estates (he had property at Inveresk), and the attainting of his family, including the baronetcy.
The former, John Wedderburn of Ballendean, is notable for the civil case brought under Scots law by his former slave Joseph Knight.
When the mixed-race Robert Wedderburn showed up at the family seat seeking to claim kinship, he was sent away with a flea in his ear.
Andrew remained in Europe, inherited his father's estates and set up as a sugar broker (Wedderburn and Company).
In 1825 the HBC moved its trading post for that region from Spokane House and named the new site Fort Colvile (NB spelling).
It was abandoned in 1870, and flooded over by the Grand Coulee Dam in 1940; the Colville people derive their name from their association with this trading post.