Small is featured as one of the central figures in American artist John Trumbull's notable painting, The Death of General Warren at the Battle of Bunker's Hill, June 17, 1775; versions were completed in the early 19th century.

At an early age Small's family purchased a commission for him to enter the Scots Brigade, and he served with them in the Netherlands.

[3] In 1756, Small obtained a commission in the British Army, becoming a lieutenant in the Black Watch just before its departure for North America to fight in the Seven Years' War.

[3] Lieutenant Small went to North America to fight for Great Britain against French Canadian forces in the Seven Years' War.

He sailed with his regiment to take part in the Invasion of Martinique and the British expedition against Cuba, as Great Britain tried to take more power in the Caribbean during the Seven Years' War.

When the Black Watch left in 1767 for Europe, most of the men of that regiment, who had earlier volunteered to stay in North America, joined the Fusiliers in order to serve under Small.

[1] It may have been during the interval between the Seven Years' War and the American Revolution that Small began to acquire his estate, Selmah Hall, in Nova Scotia.

Promoted to the rank of major, Small was sent to Nova Scotia to raise at his own expense the Young Royal Highlanders Regiment.

At the end of the war, the Crown aided many of its soldiers with land grants to settle in Nova Scotia and Upper Canada (present-day Ontario) in lieu of pay, to encourage British colonization of the area.

Small also lived there for a time, constructing a manor house called "Selmah Hall"; the community of Selma, Nova Scotia, was named for his property.

[4] After the war, Small is thought to have advised American artist John Trumbull on his painting The Death of General Warren at the Battle of Bunker Hill.

[5] Trumbull painted Small at the centre of the action, holding a fellow soldier's Brown Bess musket to prevent him from bayoneting his friend, American Patriot Joseph Warren.