[1][3] In 1843, Sargent relocated to Massachusetts, where he spent the winters driving a milk cart for his uncle near Quincy.

[5] In 1850, they arrived in Weaverville, California, in the San Joaquin Valley, and did business under the name of "Sargent Brothers".

In 1853, Sargent moved to Santa Clara County, California, and settled on the Los Angeles Ranch where he continued livestock raising.

[1][3] In 1856, Sargent purchased Rancho Juristac, a 4,540 acres (1,840 ha) property located five miles south of Gilroy, California.

[9] The Sargent Brothers acquired and developed sizable holdings in San Joaquin, Santa Clara, and Monterey counties.

[3] Sargent was involved in Republican Party politics, serving a term as a member of the California State Assembly.

He served as a director of the Santa Clara Valley Agricultural Society for ten years and was elected its president for one term in 1877.