Born in Springfield Township, New Jersey, his parents were Samuel and Mary Halsey (née Hutchings).

In 1840 Armstrong moved to Newark, New Jersey, where his father engaged in the leather business.

[1] In 1846, at age 19, he entered the firm which was called Samuel Halsey & Sons even after the death of his father.

[1] He was a member of the New Jersey General Assembly in 1861 and 1862, and was United States assessor of internal revenue,[1] from 1862 to 1866.

[1] Halsey was elected a Republican to the United States House of Representatives in 1866, serving from 1867 to 1869, being unsuccessful for reelection in 1868.