The village lies five miles (8 km) south-east of the nearby town of North Walsham.

The nearest railway station is at Worstead for the Bittern Line which runs between Sheringham, Cromer and Norwich.

The community is bordered to the north and east by the River Ant and dykes and to the south and west by tracks and hedgerows.

The first part was the present village centre and the area around Low Street which was a considerable-sized hamlet and this constituted the manor of Smallburgh.

White's Directory of 1845 states that Smallburgh had two shoemakers, blacksmith, surgeon, Jeremiah Hannant of the Crown Inn was a joiner and victualler, wheelwright, grocer, draper, school mistress and school, a shopkeeper, surgeon, another blacksmith and furrier, tailor, plumber and painter.

These include carpenters, bricklayers, plumber, shoemaker, grocer and draper, surgeon and registrar, blacksmith, a master of the workhouse, a mistress of the school, three public or beer houses, carrier, tailor, rector, book-keeper, many farmers and smallholders.

Other records show that the annual salaries bill for the officers and those who worked at the Union Workhouse was £293.

Smallburgh's parish church is called St Peter's and dates from the 13th century,[7] with the walls being raised in height and large windows being incorporated into the building in 1400.

A short distance east of the village across the A149 and on the banks of the River Ant can be found the remains of what was Smallburgh or Moy's drainage mill.

The sails of the mill powered a scoopwheel which was set on a shaft which extended beyond the wheel to allow for an auxiliary engine drive.

Today all that remains is an 8 ft (2.4 m) high stump of the mill tower although it is still working but has a Lister diesel engine to power it.

The mill was built in 1850 by millwrights England's of Ludham and stood four storeys and 30 feet (9.1 m) tall.

The configuration of the mill was of four double-shuttered patent sails, each with five bays of three shutters and one bay of four shutters, struck by rack and pinion via a chain pole that drove a 14-foot-diameter (4.3 m) by 9-inch-wide (230 mm) scoop wheel and a pair of under driven 3-foot-6-inch (1.07 m) French burr stones on the first floor.

The Crown Inn is situated on the eastern side of the A149 road that runs through the centre of the village.

It is of timber-framed construction and has a thatched roof hipped on the eastern side and gabled on the western end.