[1] Albury lies between Little Hadham to the south and Furneux Pelham to the north and includes the hamlets Albury End, Clapgate, Patmore Heath and Upwick Green.

[2] The 1894–1895 edition of The Comprehensive Gazetteer of England and Wales listed the hamlets: Albury End, Church End, Clapgate, Gravesend, Patmore Heath, and Upwich.

The name "Albury" derives from the Old English ald (old) and burh (fortification).

[5] To the northwest of the village stood Albury Hall, a three-storey manor house believed to have been re-built by MP John Calvert, around 1780 after an earlier house was demolished.

Calvert's son, also named John and also an MP, inherited it in 1808, and successive owners modified the house, the army requisitioned it during World War II, and it was demolished around 1950.