After having lived in Mexico for the previous two decades, Factor and other Black Seminoles joined the US Army as Seminole-Negro Indian Scouts in August 1870 and served in the Red River War.

On April 25, 1875, he was serving as a private by the Pecos River in Texas where, "[w]ith 3 other men, he participated in a charge against 25 hostiles while on a scouting patrol."

Two of the other men who took part in the charge, Isaac Payne and John Ward, both Black Seminoles, also received Medals of Honor.

[1] Following the fatal shooting of his former fellow scout and Medal of Honor recipient Adam Paine on New Years Day 1877, by another Medal of Honor recipient, Claron A. Windus, deputy sheriff of Brackettville, Texas, who was attempting to arrest Paine as a murder suspect, Factor deserted from the Army and fled back to Mexico.

[2] Factor died at age 78 or 79 and was buried at the Seminole Indian Scout Cemetery in Brackettville, Texas.