The story is narrated as a "found manuscript" penned by Karl Heinrich, Graf von Altberg-Ehrenstein, a lieutenant-commander in the Imperial German Navy during the days of World War I. Altberg begins by declaring that he has decided to document the events leading up to his untimely end in order to "set certain facts" before the public, aware that he will not survive to do so himself.

In the North Atlantic, after sinking a British freighter and its occupied lifeboats, the cruel and arrogant Altberg commands his U-boat to submerge, surfacing later to find the dead body of a seaman who died clinging to the exterior railing of the sub.

Afterward, it is unable to surface when its ballast tanks fail to repressurize, leaving it being pulled southward without resistance while slowly sinking deeper into the ocean; they never see the light of day again.

Certain of their fate, the two pass the time in their drifting vessel by sweeping the sub's powerful searchlight through the dark abyss, noting that dolphins follow them at depths and for lengths previously unheard-of.

Unable to soothe his insane companion, and unwilling to join him in suicide, Altberg agrees to operate the airlock, grateful to send Klenze to an assured death in the airless, crushing pressure of the deep.

In the end, he acknowledges that even with his mighty "German will", he is no longer able to resist the powerful visions and auditory hallucinations, nor his madness-inspired impulse to depart his U-boat and enter the temple, now impossibly illuminated by what seems to be a flickering altar flame.

An H. P. Lovecraft Encyclopedia judges "The Temple" to be "marred by crude satire on the protagonist's militarist and chauvinist sentiments", and by "an excess of supernaturalism, with many bizarre occurrences that do not seem to unify into a coherent whole.