The phrase in both instances refers to the number of stitches per inch rather than the size of the finished article of clothing.
Sometimes the gauge is deliberately altered within a garment, usually by changing needle size; for example, smaller stitches are often made at the collar, sleeve cuffs, hemline ribbing, or pocket edges.
Larger needles produce a smaller gauge (fewer stitches per inch) and smaller needles produce a larger gauge (more stitches per inch).
To check one's gauge before starting a project, a sample of knitting (a swatch) is made, ideally in the stitch pattern used in the garment.
Making a swatch also helps familiarize the knitter with the stitch pattern and yarn, which will lead to a more uniform gauge in the final garment.