The idea for the Connection Machine grew out of doctoral research on alternatives to the traditional von Neumann architecture of computers by Danny Hillis at Massachusetts Institute of Technology (MIT) in the early 1980s.

Starting with CM-1, the machines were intended originally for applications in artificial intelligence (AI) and symbolic processing, but later versions found greater success in the field of computational science.

Danny Hillis and Sheryl Handler founded Thinking Machines Corporation (TMC) in Waltham, Massachusetts, in 1983, moving in 1984 to Cambridge, MA.

At TMC, Hillis assembled a team to develop what would become the CM-1 Connection Machine, a design for a massively parallel hypercube-based arrangement of thousands of microprocessors, springing from his PhD thesis work at MIT in Electrical Engineering and Computer Science (1985).

[3] The dissertation won the ACM Distinguished Dissertation prize in 1985,[4] and was presented as a monograph that overviewed the philosophy, architecture, and software for the first Connection Machine, including information on its data routing between central processing unit (CPU) nodes, its memory handling, and the programming language Lisp applied in the parallel machine.

Nobel Prize-winning physicist Richard Feynman had previously calculated that five buffers would be enough, using a differential equation involving the average number of 1 bits in an address.

[7][8] To improve its commercial viability, TMC launched the CM-2 in 1987, adding Weitek 3132 floating-point numeric coprocessors and more RAM to the system.

Many applications for the CM-2, however, were written in C*, a data-parallel superset of ANSI C. With the CM-5, announced in 1991, TMC switched from the CM-2's hypercubic architecture of simple processors to a new and different multiple instruction, multiple data (MIMD) architecture based on a fat tree network of reduced instruction set computing (RISC) SPARC processors.

[10] The physical form of the CM-1, CM-2, and CM-200 chassis was a cube-of-cubes, referencing the machine's internal 12-dimensional hypercube network, with the red light-emitting diodes (LEDs), by default indicating the processor status, visible through the doors of each cube.