At the time of their marriage, Caroline required Fillmore to sign a prenuptial agreement.

They are believed to have had a happy marriage, and Caroline greatly enjoyed her newfound status as the wife of a former president, though her mental and physical health began to decline in the 1860s.

Her husband's sudden death in 1874 (he had enjoyed relatively good health — especially when compared to hers — until just shortly before his fatal stroke) only succeeded in making her more infirm, eccentric, and temperamental.

In her final years she frequently changed her will, and upon her death on August 11, 1881, aged 67, suits were initiated by various members of the Fillmore family contesting her directives.

She is buried in the Fillmore family plot at Forest Lawn Cemetery in Buffalo.