Noted for possessing a sharp mind and an enthusiasm for legal research, he was also a staunch supporter of the authority of precedent throughout his career, and would write landmark opinions in cases such as Elk v. Wilkins and United States v. Wong Kim Ark.

[7] In December 1881, President Chester A. Arthur nominated Gray to fill the vacancy on the U.S. Supreme Court created by the death of Nathan Clifford.

[2] The United States Senate quickly confirmed his appointment, and on January 9, 1882, he officially joined the Court.

[8] Additionally, he was one of the few Supreme Court appointees in the latter half of the 19th century who had not previously been a politician, and he maintained the opinion that law and politics were entirely separate fields.

[2] The 8–1 decision rested largely on prior court cases as well as an assessment of what the Framers of the Constitution intended to achieve (i.e. their original intent) through their grant of certain "Enumerated powers" to Congress in Article I, Section 8.

The most enduring of his written opinions is the one he authored in Mutual Life Insurance Co. of New York v. Hillmon (1892), which held that a declarant's out-of-court statement of his intention to do something or go somewhere in the future is admissible under the "state-of-mind" hearsay exception.

After the first hearing, Gray wrote that he sided with the defendant (Farmer's Loan & Trust), arguing that the tax was indeed constitutional.