James Gilbert Baker (November 11, 1914 – June 29, 2005) was an American astronomer and designer of optics systems.

He was born in Louisville, Kentucky to Jesse B. Baker and Hattie M. Stallard, the fourth child of that couple.

After the start of World War II, he was recruited to be a civilian optical designer for the Army's newly formed aerial reconnaissance branch under Colonel George William Goddard.

Living in Cambridge, Massachusetts, from 1946 until 1949 he was an associate professor at Harvard University, as well as continuing his research into optics he had pursued during the war.

He and Edwin Land were instrumental in persuading President Dwight Eisenhower to have the U-2 spy plane built.

He died in Bedford, New Hampshire at the age of 90, and was survived by his wife Elizabeth, and his three sons, a daughter, and five grandchildren.