It imported broadcloths, dry goods, crockery, hardware, rum, and other products from the British West Indies.

In 1833, his father moved to Brooklyn, New York, with his family, to pursue work in the freight and commission business.

The firm established lines of sailing vessels for the ports of Savannah, Charleston, Mobile, and New Orleans.

[1][4] On July 15, 1847, his father died while on board the bark Nautilus, at the foot of Wall Street, when he was struck on the head by the main boom.

The firm entered the shipping trade, controlling lines of vessels to the East and West Indies, South America, and European ports.

On her trial trip from New York City, on April 2, 1867, James W. Elwell was on board along with a delegation of other prominent men.

[8] In 1886, the firm became the agent for the Fabre Line of freight and passenger steamers between Mediterranean ports and New York.

[9][10] Elwell lived at 70 Lefferts Place, Brooklyn, New York, for over forty years and had a conservatory for his garden and flowers on his property.

For twenty years he was a trustee of the American Congregational Union; vice president and trustee of the Seamen's Friend Society of the City of New York; Mission and Track Society of Brooklyn; Brooklyn Dispensary; New York Port Society; founder of the Helping Hand Night Mission, which he helped organize; the Home for Friendless Woman and Children of Brooklyn; trustee of Clinton Avenue Congregational Church; founder of the Fresh Air Fund now the Seaside Home; president of the board of council of the Mariners' Family Asylum of Staten Island and trustee of the Children's Aid Society and Industrial School of Brooklyn.

[14] In 1922, the SS West Pocasset was reactivated and allocated to James W. Elwell & Co. for sailing on its Spanish routes.

[15] In 1920, the business was incorporated under the state as James W. Elwell & Co., Inc. Howard E. Jones was elected president, Harvey G. Perine vice-president, Rodrique Joly, treasurer, and Robert W. Swanson, secretary.