The 21-year-old was ambitious and worked in the fishing village of San Pedro, initially as a store clerk, and later as a stagecoach driver on the line that connected the hamlet with the pueblo of Los Angeles, a town of less than 2,000 people 20 miles (30 km) to the north.

[3] Banning was elected to a one-year term on the Los Angeles Common Council, the governing body of that city, beginning May 10, 1858, and ending May 9, 1859.

By the 1860s, Banning stagecoach wagons were traveling to Salt Lake City, the Kern River gold fields, the new military installation at Yuma, Arizona, the Mormon settlement at San Bernardino, and in an arc around the Southern California region.

He also began expanding the harbor and docks at San Pedro from their beginnings as illegal exchange sites for mission contraband during the Spanish and Mexican eras, and made them efficient enterprises.

In the late 1850s Banning and a group of Southern California investors purchased 640 acres (2.6 km2) of land adjacent to San Pedro for port expansion.

Banning invested the profits from his trade networks into the development of a more sophisticated port complex and for the creation of roads, telegraphs, and other connections to Los Angeles.

Family life was relatively stable in the Banning household, and Phineas was a doting, if distant father to his three boys, who grew up around the expanding docks in San Pedro.

An astute businessman and a vocal patriot, Banning and fellow Californian politician Benjamin Wilson donated adjacent plots of land in Wilmington for a military base.

As a California state senator, he campaigned for greater transportation connections to the city of Los Angeles and the growing port, his personal project.

Realizing that Los Angeles would wither into nothingness if the company bypassed it, the city complied and Banning surrendered his hard-earned railroad.

[7] Banning's legacies lived on, and his dreams were realized with the federal approval of the Port of Los Angeles in the early 20th century, and the completion of a full breakwater in 1914, creating one of the world's busiest harbors.