A pitcher is credited with starting the game if he throws the first pitch to the first opposing batter.
If a player is listed in the starting lineup as the team's pitcher, but is replaced before facing an opposing batter, the player is credited with a game pitched but not a game started; there have been instances in major league history in which a starting pitcher was removed before his first pitch due to an injury, perhaps suffered while batting or running the bases during the top half of the first inning.
The all-time leader for games started is Cy Young with 815 over a 22-year career.
[1] For position players, games started is also used to denote the number of times their names appear in a team's starting lineup during the season.
The statistic is also used in other sports including football and basketball.