His well-publicised crimes, escape from arrest, recapture and trial led to his transportation to the Australian penal colony of Van Diemen's Land (now known as Tasmania).
Solomon married Ann (or Hannah) Julian on 7 January 1807 in the Great Synagogue, Duke's Place, London.
Solomon used the shop to carry on business as a receiver of stolen goods, known as a fence, becoming one of the most active Londoners in the "trade".
On 17 April 1810, Solomon and a man named Joel Joseph were caught stealing a pocket book (valued at 4 shillings) and £40 in bank notes from Thomas Dodd outside Westminster Hall (the site of Parliament) where a large crowd had gathered for a public meeting.
Joseph attempted to get rid of the evidence by eating the bank notes while Solomon tried to ditch the notebook.
However, for reasons that are no longer clear, he remained in England, imprisoned in the prison hulk Zetland for four years, before being released in error or escaping.
The judge allowed Solomon's four youngest children (all under the age of ten) to accompany Ann on the transport ship.
Their two oldest sons, John, 20, and Moses, 19, sailed to Sydney and then to Van Diemen's Land independently to be with their mother.
Solomon's father was also charged with theft, but the court allowed his sentence to be respited because of his age (Henry claimed "I am upwards of seventy years old".
Hobart, Van Diemen's Land's capital, was the enforced home of many of Solomon's old criminal colleagues and customers.
Ann Solomon had initially been assigned as a servant to police officer Richard Newman, but quarrels broke out and she was sent to the Van Diemen's Land Female House of Correction.
The judge approved Solomon's release because of a technical fault in the London warrants, but fixed bail at £2,000, with four sureties of £500.
Police arrested Solomon and placed him on board the ship Prince Regent to be sent back to England.
Solomon's trial at the Old Bailey in June 1830 caused a sensation and was extensively reported in the newspapers and the pamphlets of the day.
Solomon was tried at the Old Bailey on eight charges of receiving stolen goods, found guilty on two, and sentenced to transportation for fourteen years.
[1] When Solomon was released from prison, he took up residence at New Norfolk and tried to reunite with his family, but the two elder sons seem to have left Van Diemen's Land by then.
[1] Solomon died on 3 September 1850 and was buried the next day in the Jewish cemetery in Harrington Street, Hobart.
It had been officially closed in 1872, and following the seizure of the property by the state in 1945, what memorials remained were removed as an apartment complex was built on the site over the next decade.