[5] Besides his father, his paternal uncles were Martin, John, and John Jacob Zabriskie,[a] all descended from Albrycht Zaborowski, who left Ducal Prussia after the Thirty Years' War, and came to New Amsterdam on the Dutch ship De Vos ship in 1662.

[6] Through his uncle Martin,[b] he was also a first cousin of Eliot Zborowski, who married Margaret Astor Carey, a granddaughter of William Backhouse Astor, Sr.[7] His maternal grandparents were Captain William M. Titus, who served in the War of 1812, and Maria (née Gardner) Titus.

His office was located at 52 Beaver Street in a building built by his grandfather, William M. Titus, on land once owned by his great-grandfather, Thomas Gardner, "a wealthy resident of Paramus, and who was somewhat eccentric in disposition.

[10] He ran against the sitting Assistant Treasurer of the United States, Hamilton Fish II, who was a son of former U.S.

[2] In addition, the Zabriskie's owned two country estates, Province Island on Lake Memphramagog and Blithewood at Annandale-on-Hudson, New York.

[18] In 1899, Zabriskie purchased Blithewood from John Bard's 1,000-acre estate and commissioned Francis L. V. Hoppin, a McKim, Mead and White alumnus, to tear down the old home and build a new 42-room Georgian-style manor house in the Beaux-Arts style.

Bard had purchased Blithewood in 1853 from Robert Donaldson Jr., who had hired landscape designer Andrew Jackson Downing to transform the grounds upon his 1835 purchase of the 92-acre estate, then known as Annandale (owned by John Church Cruger, the father of Stephen Van Rensselaer Cruger and son-in-law of Stephen Van Rensselaer).

[19] In June 1895, Zabriskie was married to Frances Hunter (1866–1951) at the Madison Square Presbyterian Church.

[c][20] Together, were the parents of two children:[21] He was a noted collector of medals, giving speeches and writing a book on the topic.