James Baldwin Brown (1820–1884) was a British Congregational minister.
He was the eldest son of James Baldwin Brown the elder (1785–1843), barrister and writer.
Brown was famous at the time for his liberal philosophical views, both in the pulpit and in the press, and the numerous religious works that he published.
He married a sister of the sculptor Henry Leifchild, and they were parents of Gerard Baldwin Brown, the art historian.
He was buried at West Norwood Cemetery, where his memorial was a large Celtic cross, listed Grade II.