His father died a short time later, leaving a widow and five children in straitened circumstances.
He was again declared elected to the 91st New York State Legislature (Kings Co., 6th D.), and took his seat at the beginning of the session, but his election was contested by Democrat John Raber who was seated on March 13, 1868.
The same year he supported Seth Low at the first mayoral election of the consolidated City of New York, against the regular Republican candidate Benjamin F. Tracy, and fell in disgrace with the Republican party machine.
Worth suffered from poor health after 1898, and during his last years spent the winters in Hot Springs, Arkansas.
On February 21, 1905, a short time after returning from the Oaklawn Park Race Track, he died suddenly in his room at the Eastman Hotel in Hot Springs, "supposedly of heart failure.