In 1872, Barlow took office as State Attorney General and Olney helped with the prosecution of the members of the Tweed Ring.
In 1875, Olney ran on the Tammany ticket for New York County District Attorney but was defeated by the incumbent Republican Benjamin K. Phelps.
John McKeon and the resignation of Wheeler H. Peckham after only a week in office, Governor Grover Cleveland appointed Olney in December 1883 as D.A.
Instead of relying on the city’s police force, which Mandelbaum had reportedly been bribing for decades, Olney hired the Pinkerton Detective Agency, to conduct a sting operation that led to her arrest.
In 1919, he was appointed by Surrogate Fowler as referee to investigate claims against the estate of the deceased actress Anna Held.