James Madison Broom (1776 – January 15, 1850) was an American lawyer and politician from Wilmington, in New Castle County, Delaware.

He was a member of the Federalist Party, who served as a U. S. Representative from Delaware.

His first wife's name was Ann who died August 9, 1808, and he may have remarried Mary Lowber.

By his first wife he has two children, Elizabeth and Jacob, a Congressman from Pennsylvania.

He moved to Philadelphia, Pennsylvania, in 1819 and resumed the practice of law.