On December 29, 1945, a group of twenty Romanian mathematicians from various institutions in Bucharest led by Dimitrie Pompeiu held a meeting at the University of Bucharest to establish the Institute of Mathematical Sciences with the aim of "promoting scientific research in mathematical sciences, through communications, talks, publications, congresses, and other means proper to this aim".

Provoked by a verbal disagreement with Miron Nicolescu, in April 1975, Ceaușescu issued a decree to close down the Institute.

The ensuing disruption of scientific life led to the eventual departure from Romania of a number of leading mathematicians, including Ciprian Foias and Dan-Virgil Voiculescu.

In 1978, with help from Zoia Ceaușescu, some of the former members of the Institute were hired into a newly established Mathematical section of the National Institute Scientific and Technical Creation (Institutul Național pentru Creație Științifică și Tehnică, INCREST), previously known as the Institute of Fluid Mechanics and Aerospace Research (Institutul de Mecanică a Fluidelor și Cercetări Aerospațiale, IMFCA), currently a private company (INAV S.A.), owned by Grupul S.C.R.

The Institute publishes two scientific journals (Mathematical Reports and Revue roumaine de Mathématiques pures et appliquées), and organizes many specialized conferences and congresses.