First mentioned in the Landbuch of Karl IV in 1375, the Kleinmachnow played an important role at the Bäke beek / creek crossing, secured by multiple medieval castles.

Kleinmachnow is bordered by the Teltow Canal in the south and by the city boundary of Berlin in the north, west and east, forming a kind of indentation in the district of Steglitz-Zehlendorf, which is a neighbouring borough.

Located in the southwestern part of Berlin and adjacent Brandenburg, Kleinmachnow sits on the Teltow Ground Moraine Plateau, carved into a turbulent relief of small-scale chains of hills formed from boulder clay and meltwater channels, interspersed with puddles and pools.

Geologically, the youngest Vistula Ice Age, about 21,000 years ago, created a loose erosion in the Bäke Valley that made the construction of the Teltow Canal between the Seeberg and Weinberg possible.

The Buschgraben is a glacial meltwater channel, located on the southwest border of Berlin, stretching in a north–south direction between the localities of Berlin-Zehlendorf and Kleinmachnow, ending at the Teltowkanal northwest of Teltow.

The Meiereipfuhl in the Bannwald formerly held five ponds, this has since reduced with the Duellpfuhl an der Ginsterheide, Pferdepfuhl, and Pfuhl am Jägerstieg/Ecke Wolfswerder serving as receptacles for roadway surface water today.

On the night of 2–3 December 1943, a Lancaster plane of the Royal Australian Air Force (flight LM 316) was shot down over Kleinmachnow, one of those killed being the well-known Norwegian writer and poet Nordahl Grieg, at the time serving as a war correspondent.

Since German reunification Kleinmachnow has seen a major increase of population, while the restitution of numerous plots[clarification needed] in favour of those owners who had fled from the GDR led to fierce conflicts.

Subsequently, due to the increase of restitution claims by the original property owners, as well as the allure of the area, the number of inhabitants doubled from the mid-1990s until today.

In 1539, Joachim II, the Elector of Brandenburg, initiated the Reformation, resulting in the conversion of Bradenburg from a primarily Catholic region to a mainly Protestant area.

Since the reunification of Germany in 1990, debate has persisted over the most effective administrative arrangement; ranging from merely informal cooperation, through contractual collaboration, to full amalgamation into a single large district.

In 2021, the district of Teltow joined forces with Kleinmachnow to create a four-day program for an international contingent of athletes participating in the Special Olympics World Sumer Games 2023, Berlin.