Joseph was the eldest son of Prince Meinrad II of Hohenzollern-Sigmaringen (1673–1715) from his marriage to Johanna Katharina von Montfort (1678–1759), daughter of Count Johann Anton I of Montfort-Tettnang.

Shortly before 1720, Joseph joined the Austrian army, where he held the rank of General of the Cavalry and later Fieldmarshal Lieutenant of the imperial Swabian Circle.

In Oettingen on 20 May 1722 Joseph married firstly with Maria Franziska Louise (21 May 1703 – 29 November 1737), daughter of Prince Franz Albrecht of Oettingen-Spielberg, who brought him a considerable fortune.

They had six children: On 5 July 1738 Joseph married secondly with Maria Judith Katharina Philippina (30 April 1718 – 9 May 1743), daughter of Count Franz Anton of Closen, Baron of Arnstorf.

They had three children: On 22 October 1743 Joseph married thirdly with Maria Theresa (3 March 1696 – 7 May 1761), daughter of Count Franz Christoph of Waldburg at Trauchburg.