His father died soon afterwards, and he left school at age 16 to begin working in the publishing business, where he eventually became a proofreader.

Active in the Democratic Party, from his early 20s he was a key member of the Tammany Hall organization, beginning as a messenger for "Boss" William Tweed, and serving as confidential secretary for Henry W. Genet, Tweed's Tammany Hall successor.

His administration was largely defined by the 1894 Lexow Investigation, which uncovered widespread police department corruption that was directly linked to Tammany.

Gilroy did not run for reelection, and was succeeded in 1895 by reform candidate William L. Strong, who ran with the backing of Republicans and anti-Tammany Democrats.

He had a daughter, Frances E. Gilroy, who married Edward A. Maher Jr.[1] He died on December 1, 1911, at his home on Ocean Avenue, Far Rockaway, Long Island and he was buried at Woodlawn Cemetery in the Bronx.