Horse-drawn trains began operating on November 2, 1836; the horses were replaced by a newly arrived steam locomotive, Adrian No.

On May 9, 1846, the partially completed line was sold to the Michigan Southern Rail Road, which changed the planned western terminal to Chicago, using the charter of the Northern Indiana Railroad.

The Michigan Southern leased the Erie and Kalamazoo on August 1, 1849, giving it a branch to Toledo, and a connection to planned railroads to the east.

A more direct line was soon planned from Elkhart, east to Toledo, and the Northern Indiana Railroad was chartered in Ohio, on March 3, 1851.

The Franklin Canal Company was chartered on May 21, 1844, and built a railroad from Erie, Pennsylvania, southwest to the Ohio border.

The Buffalo and State Line Railroad was incorporated October 13, 1849, and opened January 1, 1852, from Dunkirk,New York, west to Pennsylvania.

On November 16, 1853, an agreement was made between the two railroads, which had been built at 6 ft (1,829 mm) broad gauge, to relay the rails at 4 ft 8+1⁄2 in (1,435 mm) standard gauge to match the Franklin Canal Company's railroad (see below) on the other side of Erie, and for the Buffalo and State Line to operate the Erie and Northeast.

The Northern Division opened from Cleveland, west to Sandusky, Ohio, on October 24, 1853, and the rest of the way to Toledo, on April 24, 1855.

On April 6, the Michigan Southern and Northern Indiana Railroad and Lake Shore merged to form the Lake Shore and Michigan Southern Railway, which absorbed the Buffalo and Erie Railroad on June 22, giving one company the whole route from Buffalo to Chicago.

The main route passed through Dunkirk; Erie; Ashtabula, Ohio; Cleveland; Toledo; Waterloo, Indiana; and South Bend.

In 1901, the railroad bought a new property in Collinwood for $2 million to build a much larger repair center that by the 1920s employed more than 2,000 people.

Around 1877, Cornelius Vanderbilt, and his New York Central and Hudson River Railroad, gained a majority of stock of the Lake Shore and Michigan Southern Railway.

Amtrak's New York City–Chicago Lake Shore Limited runs along the full route from Buffalo west.

A major branch of the LS&MS extended from Northeastern Ohio, to the coal and oil fields of northwestern Pennsylvania, terminating near Brookville.

This line included perhaps the most impressive engineering structures on the LS&MS, as well as the later NYC, with several large trestles, bridges, and tunnels, near Brookville, including a bridge-tunnel-bridge-tunnel-fill combination near Piney, Pennsylvania, and two magnificent trestles west of Brookville, near Corsica, Pennsylvania.

Many of the larger trestles were taken out in the late 2000s, reportedly on orders of the Pennsylvania Public Utility Commission (PUC), although the bridge across the Clarion River survived, as of 2015.