Richardson was born in Usworth, County Durham.
His father was killed in 1885, while working in the Usworth Colliery.
He joined the Durham Miners' Association, and became branch secretary in 1898, and was elected to the county executive in 1912.
He served as General Secretary of the union from 1924 until his death in 1930 aged 57, and as treasurer of the Miners' Federation of Great Britain from 1921.
His brother Tom was also active in the party, and was the Member of Parliament for Whitehaven from 1910 to 1918.