Mardian's father, Samuel, was from the Armenian town of Hadjin in the Vilayet of Adana in the Ottoman Empire (present day Saimbeyli in Mediterranean Turkey).

From 1962, Mardian left his law practice to become vice president and chief legal officer of a savings and loan association.

In the intervening years, he served as chairman of Ronald Reagan's state advisory committee during his 1966 gubernatorial campaign in California.

Mardian was in charge of the Internal Security Division, which headed up the fight against the radical left, prosecuting draft dodgers.

He was entrusted to transfer to the White House the wiretap logs which had been discovered among J. Edgar Hoover's possessions in the Federal Bureau of Investigation after his death.

Mardian became involved in the Nixon administration's unorthodox campaigns early when he headed the federal prosecution of Pentagon Papers leaker Daniel Ellsberg in 1971.

Having learned of the arrest of the five men in the Watergate complex, Jeb Stuart Magruder testified that at John N. Mitchell's suggestion Mardian telephoned G. Gordon Liddy and told Liddy to contact Attorney General Kleindienst, with an order that James W. McCord, Jr. should be released before his identity was discovered.

When Jeb Stuart Magruder decided to cooperate with the prosecution on April 10, 1973, it became certain that Mardian would be indicted, although he had first to go before the Ervin Senate committee (July 19–20, 1973).

The special prosecutor declined to retry him; in 1997 his appeal lawyer Arnold Rochvarg wrote a book outlining the legal history of the case and arguing that Mardian was innocent.