Although it is a geographically discrete hamlet, some consider it as part of the larger village of Port Isaac located around the headland, upon which it relies for most services.

The appearance of the hamlet has changed little in two centuries, with its stone and slate buildings recognised for their character and the rugged local coastline as a heritage coast.

The parish has just over 1,000 residents, including a significant proportion of retired people, but the hamlet itself consists mostly of second homes and holiday lets.

In the 16th century, the English cartographer John Norden called it “Port-kerne", and indeed linked this name to the cove, suggesting that the natural harbour is the 'port' regardless of any settlement.

[2] Writing in 1833–4, Dr Frederick Trevan referred to the settlement as "Porth Karn Hun...now commonly called Port Gavern".

[7] In 1802 Warwick Guy, of the family that owned much of the Port,[2][3] leased land to build a fish cellar (a Cornish term for a building used to make and store fishing pots, nets, sails and equipment, and historically to process the catch),[8] one of four eventually constructed by the Guy family, who continued to live and work in the area for the next hundred years.

In 1833–4, Dr Trevan describes a "small unsafe cove where [the] principal business of parish [is] carried on chiefly in slate – 3000 to 4000 tons annually", with the Port then having five families of thirty five inhabitants total.

[3] In 1859, Murray's Handbook of Devon and Cornwall recorded that "the quarries present one of the most astonishing and animated scenes imaginable".

[2] During the Second World War, Port Gaverne hosted evacuees, and the fish cellars were converted into homes for the children, along with the clifftop Headlands Hotel.

[15] The Draft Neighbourhood Plan describes the local coastline as "particularly rugged, unique and spectacular", "recognised as a heritage coast, whilst inland the area remains extensively unspoilt with prevailing rural characteristics".

[16] The hamlet lies within the Pentire Point to Widemouth Section of the Cornwall Area of Outstanding Natural Beauty.

According to the AONB Management Plan, the local area has "a noticeable lack of tree cover due to the exposure to coastal winds".

[2] The steep hillsides enclosing the hamlet have forced developers to look at the flat, if exposed, land on the headland between Ports Gaverne and Isaac for new build.

[14] The population of the parish is declining, as younger groups (particularly families) move away in search of job opportunities and more affordable housing.

[20][21] A 2013 University of Plymouth study found that visitors had "clear destination images of Port Isaac prior to their visit", with the television series acting as "a display window for the village and surrounding area".