Pifer grew up in Ridgecrest, California, and played football at Sherman E. Burroughs High School.

[1] Pifer enrolled at Oregon State University and played for the freshman football team in 1963.

[5] His 1,095 rushing yards was an Oregon State single-season record,[6] and ranked fifth nationally among major college football players.

[10] At the end of the 1966 season, he received the W. J. Voit Memorial Trophy, edging Mel Farr and Gary Beban as the outstanding college football player on the Pacific Coast.

[14] After retiring from football, Pifer owned a McDonald's franchise in Morgantown, West Virginia, and later a restaurant in Springfield, Oregon.