He studied art in the United States, including West Point and then France and the far east, and came to produce many landscapes during his extended tours of these places.

[4] Winckworth had a nephew, Walter Gay, who was also a noted painter of French subjects and spent much time painting in France.

After he settled he often traveled to the White Mountains in New Hampshire, often visiting the artist's colony at West Compton.

After twenty-four years of work, Gay auctioned off his entire collection of paintings in 1874 to fund a four-year visit to Japan, and was one of the first Americans to take up residence there.

During his four year tour in Japan, he produced a number of paintings with subjects that included canals, temples, and old castles, and which have been described as his Orientalist work.

[6]<[2] In the 1850s Gay exhibited his paintings at the Boston Athenæum which were displayed alongside works by notable European masters such as William Holman Hunt, and Narcisse Virgilio Díaz and Jean-François Millet, serving as "proud reminders" that Boston artists were well-versed in the latest art trends in Europe.

In artist Samuel Isham's opinion, Gay did not achieve the measure of fame he deserved, maintaining that his paintings were characterized by "simplicity and truth".