The name Colbury is derived from Middle English for "Cola's manor",[3] and near Colbury is an estate called Langley which was held by "Cola the Hunter" in the Domesday Book of 1086.

[4] The manor of Colbury was given to the Abbot of Beaulieu by Robert de Punchardon sometime in the 13th century.

[2] A grant of free warren in the manor was made in 1359–60 to the Abbot and convent of Beaulieu.

[2] Successive abbots remained in possession of the manor until the dissolution of the abbey in April 1538, when it passed to the Crown.

The village has a hall built in 1928 as a memorial to the First World War.