With the 1504 victory of Ulrich, Duke of Wurttemberg in the Landshut War of Succession the castle became a fief of Württemberg under Marshal Konrad Thumb of Neuburg in 1507.

His son Hans Konrad Thumb of Neuburg sold the fief in 1527 to Philip of Hirnheim, the Reformer of Gruppenbach.

Until 1888 the site was home to the state forestry office, at which point it was acquired by the Weinberger farmer Christian Hildt.

In 1924, Siegfried Levy, from Kornwestheim, who with his family owned the company Salamander Shoes, acquired the facility in which he established a stud farm.

On 5 October 1937 the castle was "Aryanized" (nationalized, i.e., seized by the Nazis and the Jewish owners fled to South Africa.

In 1939, the old medieval bailey was rebuilt and parts of the castle were demolished to produce an "Ordensburg" for the Nazi party.

While most buildings in the grounds date back to the renaissance castle, the staircase tower is marked by recent rebuilding.

Until well into the 20th century, the facade of the residential building and the staircase tower was just a timber balustrade, but the former archways of the commercial areas on the ground floor are now largely walled.