He was born in Christiania as the son of goldsmith Jacob Tostrup (1806–1890) and his wife Lina Hjorthøy (1821–1890).
He was a great-grandson of priest and topographer Hugo Fredrik Hjorthøy and a granduncle of Jakob Tostrup Prytz.
He was thus brought into the goldsmith profession at an early age, and studied in Oslo, Berlin, Paris and Copenhagen between 1858 and 1865.
[2] He also cooperated with architects such as Henrik Thrap-Meyer, Christian Christie and Torolf Prytz in making new designs.
He was a co-founder of Norwegian Museum of Decorative Arts and Design in 1876, and was a member of the board of directors for the rest of his life.