The phrase "I know it when I see it" was used in 1964 by United States Supreme Court Justice Potter Stewart to describe his threshold test for obscenity in Jacobellis v.
Before Roth v. United States in 1957, common law rules stemming from the 1868 English case R v Hicklin have articulated that anything which "deprave[s] and corrupt[s] those whose minds are open to such immoral influences" was said to be obscene, and therefore banned.
Justice Potter Stewart, in his concurrence to the majority opinion, created the standard whereby all speech is protected except for "hard-core pornography".
As for what, exactly, constitutes hard-core pornography, Stewart said "I shall not today attempt further to define the kinds of material I understand to be embraced within that shorthand description, and perhaps I could never succeed in intelligibly doing so.
This was modified in Memoirs v. Massachusetts (1966), in which obscenity was defined as anything patently offensive, appealing to prurient interest, and of no redeeming social value.