The museum is named after Agnes Newton Keith, an American author known for her three autobiographical accounts of life in British North Borneo.

[1] In 1934, Agnes married Harry Keith, a British official assigned for forest conservation in Sandakan.

[4][5] At one event she saw the apparition with family; saying goodbye to husband and taking her own baby before returning to the road in front of the house with body keep moving forward while her head managed to keep straight looking over behind (towards Agnes position).

[4][5] The same horror experienced earlier by Agnes also felt by the next house owner following Agnes return to the United States; a new female owner named Rosemary (living in the house from 1955 to 1967), the wife of G L Carson while sleeping alone in the house awaken by the sounds of a room door located at the end of the house being closed several times in early dawn and hearing footsteps that getting louder moving towards her room before shockingly saw an unrecognisable female apparition suddenly appeared at the end of her bed with eyes staring at her when she began to open her own eyes that causing her to be quickly fainted.

The house provides insights to life during the administration of British North Borneo and is furnished with a reproduction of colonial furniture and antiques.