Undersized at 5 feet 6 inches (1.68 m) and 171 pounds (78 kg),[2][3] he spent his entire five-year NFL career with the Chargers from 1984 to 1988.

James was born in Albany, Georgia,[1] where he attended Dougherty High School, playing football, basketball, and running track.

Because of his small stature of 5 feet 6 inches (1.68 m) and 150 pounds (68 kg), Auburn University was the only major college to recruit him.

[4] James was Auburn's leading rusher with 561 yards in 1981, head coach Pat Dye's first year with the team.

[10] The total yardage was second at the time only to the 373 yards by Billy Cannon in 1961, and remains a Chargers franchise record.

He might have broken the record in an earlier game that season against the Cincinnati Bengals except for a Chargers penalty that cost him 89 yards of a 100-yard kickoff return.

He was waived during preseason in 1989, when the Chargers opted for a quicker running back, rookie Dana Brinson.

[14] The Kansas City Chiefs claimed James, planning to use him primarily as a wide receiver.