The Smiths bought land in what was then the countryside outside New York City and planned to build an estate, which they called Mount Vernon, in honor of President Washington.

Along with General Francisco de Miranda, Colonel Smith raised private funds, procured weapons, and recruited soldiers of fortune to liberate Venezuela from Spanish colonial rule.

On February 2, 1806, a force of 200 filibusters, including Smith's son William Steuben, set sail to Venezuela on a chartered merchant vessel, the Leander, armored by Samuel G. Ogden.

On April 28, a failed attempt to land in Ocumare de la Costa resulted in two Spanish vessels capturing the Bacchus and Bee.

Sixty men, including Smith's son, were taken prisoner and put on trial in Puerto Cabello for piracy,[3] and ten were sentenced to death by hanging.

Miranda, aboard the Leander and escorted by the packet ship HMS Lilly, escaped to the British islands of Grenada, Barbados, and Trinidad, where the governor Sir Thomas Hislop, 1st Baronet agreed to provide some support for a second attempt to invade Venezuela.

In the aftermath of the failed expedition, Colonel Smith and Ogden were indicted by a federal grand jury in New York for violating the Neutrality Act of 1794 and put on trial.

Colonel Smith claimed his orders came from President Thomas Jefferson and US Secretary of State James Madison, who refused to appear in court.