Founders Bill Gray and Ron Smith named the restaurant after Henry Lee III, known as "Light Horse Harry", a historical figure from the American Revolution.

[4][1][6] Light Horse Harry led the successful Battle of Paulus Hook, which helped to retain Jersey City as a stronghold for the Continental Army.

[1] Founders and brothers-in-law Bill Gray and Ron Smith started the establishment after becoming familiar with the relevant history while restoring two neighboring buildings.

[3] Dishes at the restaurant have included lobster salad and pan-roasted salmon, in addition to "a crisp, greaseless version of fish and chips".

[10] Light Horse Tavern received favorable mention in a 2003 review for The Jersey Journal, which noted, "With its beautiful decor, try to stay focused on the food.

[3][note 1] David Corcoran wrote favorably of the restaurant in a 2007 review for The New York Times, and commented of the establishment's bar: "Long and lively, a scene all its own.

[16] Amaechi Okoro wrote favorably of the dining and bar at the establishment in a 2008 piece for The Waterfront Journal, and called it a "classy alternative" to louder facilities.

[5] Jon Whiten of The Jersey City Independent recommended the restaurant among "This Weekend’s Best Bets" in June 2010, for its wine tasting fundraiser with sommelier Roland Arnold.