Stephens was born in about 1832 in Philadelphia, where his family had moved from Virginia the previous year to escape the white violence which arose following Nat Turner’s rebellion.
His father, William Stephens,[1] worked as a bootblack, waiter, and laborer, and became a lay preacher in the First African Baptist Church, a strongly abolitionist congregation active in the Underground Railroad.
[2] After the Civil War, he initially worked in conjunction with the Freedmen's Bureau educating newly freed slaves in Virginia.
During his time at sea during 1857–1858, Stephens was nearly enslaved in Charleston, South Carolina, building his hatred of slavery even higher than his earlier strong abolitionist views.
Like a great many northern African-Americans, once the civil war broke out Stephens was outraged and frustrated by the Federal government's initial unwillingness to allow them to fight against the South.