In the 1880 Census Baker identified himself as a glass manufacturer, but he had also founded three businesses supplying coal, tugs, and lighters and cold storage facilities in Baltimore harbor.
Although the U.S. mercantile marine had been declining for decades in the face of British competition and high domestic operating costs, Baker’s ambition was to build a major American owned transatlantic steamship line in Baltimore.
Baker’s move to sell the line to his principal British competitor in the late 1890s led to the creation of J Pierpont Morgan's colossal International Mercantile Marine Company (IMM) in 1902 through the merger of the A.T.L.
But he had gained an international reputation as an authority on shipping and was consulted by the U.S. Government on the Panama Canal and was one of four experts appointed for a National Sub-Committee on Transportation Problems.
He lent vessels to carry grain to starving Russians and for use as hospital ships in time of war, and he gave large sums to a wide variety of worthy causes.