[2][1] Joseph's brother Thomas Nickerson (1810–1892) also served as a railroad executive, notably as president of the Atchison, Topeka and Santa Fe Railway (ATSF) between 1874 and 1880.

[3] In 1880, when Joseph Nickerson died as one of the richest men in Boston,[4][1] Albert inherited several million dollars and was made a director of the ATSF and the Mexican Central Railway.

[2] His friendship with the president was a leading cause of Cleveland purchasing Gray Gables, which adjoined Nickerson's "Great Hill" estate in Marion, Massachusetts.

[1][2] Partially as a result of these two setbacks, Nickerson sold the house to his brother George and he moved to an estate on Buzzards Bay in Marion, Massachusetts.

[2][9][1] In Marion, Nickerson bought a large estate known as "Great Hill," with 40 rooms and seven square miles of forests and winding driveways.

[5] Russell Sturgis called it "one of the best pieces existing of the peculiar Romanesque sculpture of semi-Byzantine character which Mr. Richardson's work introduced to this community.

"[12] The castle has a number of interesting architectural elements but its best known is its numerous secret passages[13] and "legendary underground mazes and hallways".

[5] The escape routes and secret hiding places were presumably added because Nickerson was concerned that people "would try to murder him for his abundant wealth.

"[5][b] It was built on top of a rocky hill "so that the Castle and the River appeared magically to carriages or cars arriving through the forested Pine Street entrance.

[1] His funeral at St. Paul's Church attracted a large number of mourners, and special trains were run from Boston to accommodate them all.

[2][c] The Panic of 1893 greatly reduced his fortune, and his family was forced to sell off his 60,000 shares in the Atchison railroad to hold onto their property.