It is situated to the west of the town within 22 acres (8.9 ha) of grounds.

The lower site on Lady Edith's Drive closed on 23 June 2017.

The present building designed by Keith Scott of Building Design Partnership's Preston office, was built by the North Riding Education Committee in the late 1950s, around the same time as Scarborough Technical College (now called Yorkshire Coast College).

The school at this time had specialist nautical studies courses, which were aimed at pupils wishing to pursue a career at sea.

Previously a community school administered by North Yorkshire County Council, in March 2019 Graham School converted to academy status.