His father was a first-generation nobleman of the church estate, and his older brother Alexander Ostrovsky became a noted playwright.
He entered government service in the office of the Civil Governor of Simbirsk, assisting Auditor General V. A. Tatarinov in the introduction of auditing reforms.
He was assigned to perform audits in Vladimir, Tula, Kaluga, Yaroslavl, Kostroma, Pskov, Vilna, Grodno, Kovno Province, and the former Kingdom of Poland (then a Russian possession).
Ostrovsky was an honorary member of the Imperial Academy of Sciences, Chairman of the Museum of Applied Knowledge in St. Petersburg, and (from May 15, 1883) an Actual Privy Counsellor, a civilian rank equivalent to full general or admiral.
His funeral was held July 28 in the Holy Spirit Church of Alexander Nevsky Monastery, and was attended by Ivan Durnovo (Chairman of the Council of Ministers) and others.