His paternal grandparents were John Frederick Frelinghuysen (1776–1833), a lawyer and brigadier general in the War of 1812, and his second wife, Elizabeth Mercereau Van Vechten.

[1] In 1921, President Warren G. Harding signed the Knox–Porter Resolution, officially ending America's involvement in World War I at Frelinghuysen's estate in Raritan, New Jersey.

Together they had three children: His wife's portrait and that of Joseph Jr, were painted in 1916 by the Swiss-born American artist Adolfo Müller-Ury; it is today in the Newark Museum, New Jersey.

Frelinghuysen owned an 88-foot houseboat called Victoria that Harding used for 12 days after he won the 1920 election for President, but before he was inaugurated in March 1921.

The marker remains in a patch of grass near a Burger King parking lot along Route 28, just north of the Somerville traffic circle.