James Gardens consists of broad lawns, numerous flower beds, specimen plantings, rock gardens, nature trails, three large and four small ponds fed by a spring and connected by a stream, a carp pool, and a lawn bowling court.

Each year thousands of flowers and over 75,000 tulips are planted in the beds, whose designs and materials are changed annually in the rock gardens and under the well-pruned trees and shrubs.

In 1908, Fred James purchased the property from his wife's parents, Mr. and Mrs. Thomas Rumney, as well as the adjoining land from the Home Smith Company.

This purchase totalled 21 acres (8.5 ha) of hills, valleys, virgin forest, and bush meadow land on the west side of the Humber River in the area then part of Etobicoke.

The James family lived in a house situated on the top west corner of the estate at Edgehill Road.