Born in Bellefonte, Pennsylvania, Holly read law to enter the bar in 1891.
He was an assistant state's attorney of Cook County, Illinois from 1914 to 1916, thereafter returning to private practice in Chicago until 1933.
[1] Holly received a recess appointment from President Franklin D. Roosevelt on November 8, 1933, to a seat on the United States District Court for the Northern District of Illinois vacated by Judge George Albert Carpenter.
He was confirmed by the United States Senate on February 20, 1934, and received his commission on March 1, 1934.
He assumed senior status on December 31, 1943. his service terminated on January 30, 1958, due to his death.