Alfred was a nephew of American painter Eastman Johnson, who was a co-founder of the Metropolitan Museum of Art, New York City.

Alfred grew up with his family on his father's assignments in California, Maine, and in the District of Columbia, where he attended public schools.

Johnson served successively on Newark (C-1), Texas (1892), Detroit (C-10), Kearsarge (BB-5), and had duty in the Philippines during the insurrection aboard New York (ACR-2).

From 1907 until 1910, he served as an instructor in the Department of Mathematics at the Naval Academy, as part of a regular pattern of alternating sea and shore assignments.

In 1914, after several months in command of the Benham (DD-49), Johnson was given charge of fitting out Downes (DD-45) at the New York Shipbuilding Company, Camden, New Jersey.

For his services in command of the Conyngham, Johnson was awarded the Distinguished Service Medal with citation as follows: For exceptionally meritorious service in a duty of great responsibility as Commanding Officer of the USS Conyngham engaged in the important, exacting and hazardous duty of patrolling the waters infested with enemy submarines and mines, in escorting and protecting vitally important convoys of troops and supplies through these waters, and in offensive and defensive action, vigorously and unremittingly prosecuted against all forms of enemy naval activity; and especially for an attack upon an enemy submarine on October 19, 1917, during which attack Conyngham was directly over the submarine and dropped a depth bomb which is believed to have destroyed the submarine.Johnson fitted out and commanded the destroyer Kimberly (DD-80) early in 1918.

Upon his return to the United States later that year, he reported for duty as aide to the commandant, New York Navy Yard and Third Naval District.

Planes under his command made the first flight across the Caribbean to Panama and in June–July 1921 were engaged in bombing experiments with ex-German warships off the Virginia Capes.

While in this command, he developed the patrol plane squadrons by conducting their operations away from home bases for extended periods of time in distant theaters of activity, to increase their usefulness to the fleet.

While in command of the Atlantic Squadron during the winter of 1938–1939, Admiral Johnson collaborated with the Naval Research Laboratory scientists in conducting the first comprehensive radar experiments at sea.

He was advanced to the rank of Vice Admiral, in recognition of his being "specially commended for performance of duty in actual combat with the enemy during the World War".

His citation states: ... Vice Admiral Johnson rendered invaluable assistance to all members of the Board during the early stages of hostilities and, by his superb ability and tact, was in large measure responsible for securing rights and privileges vitally needed by our forces in the conduct of the war and for the maintenance of cordial relations between the United States and Latin American countries...Johnson retired on August 13, 1945, after forty-nine-and-a-half years of service.