A fictional book is a text created specifically for a work in an imaginary narrative that is referred to, depicted, or excerpted in a story, book, film, or other work of fiction, and which exists only in one or more such works.
For example, George Orwell's novel Nineteen Eighty-Four has excerpts from a book by Emmanuel Goldstein entitled The Theory and Practice of Oligarchical Collectivism which provides background on concepts explored in the novel (both the named author [Goldstein] and the text on collectivism are made up by Orwell).
An example of a fictional book that is part of the plot of another work (in addition to Nineteen Eighty-Four) is Philip K. Dick's The Man in the High Castle, in which resistance members circulate a banned book entitled The Grasshopper Lies Heavy.
An example of a fictional book linking a series is Encyclopedia Galactica, an imaginary set of encyclopedias created by Isaac Asimov and referred to in the novels in his Foundation Series.
An example of an author referring to a fictional book in a number of unconnected works is Jack Vance's quotes from an imaginary twelve-volume opus entitled Life by Unspiek, Baron Bodissey in Vance's novels (Bodissey is a fictional character created by Vance).