At the age of fourteen he became a sailor on the Yankee clipper Southern Cross which was captured and destroyed by the Confederate State Army steamer Florida in 1863, during the American Civil War.

Although he disapproved of the overthrow of the Kingdom of Hawaii in 1893, he looked favorably on the American annexation in 1898, which he believed would bring long-term stability to the islands.

His Hawaiian Fiber Company, which operated a sisal plantation on the Ewa coral plain in southwestern Oahu, was ultimately also a failure.

Youngest son Harold Garfield Dillingham (October 9, 1881 – December 19, 1971) married Margaret Bayard Hyde-Smith in 1908 and had 5 children.

Youngest daughter Marion Eleanor Dillingham (September 23, 1883 – January 19, 1972) married John Pinney Erdman in 1904[7] and had five children.