According to the United States Census Bureau, the village has a total area of 0.88 square miles (2.28 km2), all of it land.

Fairfield's main street is Soldier Road but is accessed by U.S. Route 20, a two-lane undivided highway connecting it to the west with Interstate 84 near Mountain Home, and to the east with State Highway 75 at Timmerman Junction in Blaine County.

Four miles (6.4 km) east of Fairfield, State Highway 46 begins and proceeds south to Gooding.

The Soldier Mountain ski area, established in 1948, is 12 miles (19 km) north of the village.

The Snowkiting area, known around the world as one of the top destinations for this extreme sport, is 16 miles (26 km) west of the village.

The Sun Valley ski resort is approximately 45 highway miles (70 km) northeast, up the Big Wood River Valley along State Highway 75, the Sawtooth Scenic Byway.

[10] The western end of the Magic Reservoir is 10 miles (16 km) east of Fairfield, and the Silver Creek fly fishing stream is about 30 miles (50 km) east of the village.

Fairfield experiences a continental climate (Köppen Dsb) with cold, snowy winters and hot, dry summers.