He enlisted in the United States Naval Reserve on 12 October 1939, was appointed Aviation Cadet in 1940 and underwent flight training, and was commissioned as an ensign.

He later transferred for temporary duty to Torpedo Squadron 8 (VT-8) aboard the aircraft carrier USS Hornet and reached the rank of lieutenant, junior grade.

As a Douglas TBD Devastator torpedo bomber pilot in VT-8, Gray took part in the pivotal Battle of Midway on 4 June 1942.

From there John P. Gray proceeded via the Panama Canal Zone to San Diego, California, arriving 19 June 1945.

She departed for Pearl Harbor, Territory of Hawaii, 21 June 1945, and spent July 1945 training underwater demolition teams on the island of Maui.