Born in Czernowitz in Austria-Hungary (today Chernivtsi, Ukraine), Sherf made aliyah in 1925.

In 1947 he was appointed secretary of the Situation Committee, which helped create the administrative apparatus of the new state.

In the 1965 elections he was voted into the Knesset on the Alignment's list.

He held both posts until after the 1969 elections, when he became Minister of Housing.

He was not re-elected in the 1973 elections and lost his place in the cabinet the following year when Golda Meir resigned as Prime Minister.