He joined the Malayan Civil Services (MCS) in 1928, a year after passing his qualification exam in London.

From 1941 to 1942, he served as a Company Commander in the 1st Battalion, Federated Malay States Volunteer Force (FMSVF), earning the rank of captain and becoming a prisoner of war during WWII.

[4][6] In 1958, he was appointed as the first Keeper of Public Records, an office that would eventually become the Arkib Negara, or National Archives.

[6] In 1959, he became the first director of the then-Federation of Malaya's own national museum, where he assembled a team tasked with retrieving Malayan artefacts from cities such as Lisbon and London to be displayed there.

[3][4] Sheppard's careers include:[10] Among his other positions are:[10] The Mubin Sheppard Memorial Prize was established under his name in 1996 by the Malaysia Heritage Trust to stimulate students' awareness of "the need to conserve Malaysia's built heritage and to encourage research and writings on various aspects of conservation and preservation".