The abstract group of generalized permutation matrices is the wreath product of F× and Sn.
Concretely, this means that it is the semidirect product of Δ(n, F) by the symmetric group Sn: where Sn acts by permuting coordinates and the diagonal matrices Δ(n, F) are isomorphic to the n-fold product (F×)n. To be precise, the generalized permutation matrices are a (faithful) linear representation of this abstract wreath product: a realization of the abstract group as a subgroup of matrices.
One can generalize further by allowing the entries to lie in a ring, rather than in a field.
In that case if the non-zero entries are required to be units in the ring, one again obtains a group.
This is an abuse of notation, since element of matrices being multiplied must allow multiplication and addition, but is suggestive notion for the (formally correct) abstract group