Single program, multiple data
[7] It is also a prerequisite for research concepts such as active messages and distributed shared memory.In SPMD parallel execution, multiple autonomous processors simultaneously execute the same program at independent points, rather than in the lockstep that SIMD or SIMT imposes on different data.
On distributed memory computer architectures, SPMD implementations usually employ message passing programming.
For distributed memory environments, serial sections of the program can be implemented by identical computation of the serial section on all nodes rather than computing the result on one node and sending it to the others, if that improves performance by reducing communication overhead.
Nowadays, the programmer is isolated from the details of the message passing by standard interfaces, such as PVM and MPI.
SPMD on a shared memory machine can be implemented by standard processes (heavyweight) or threads (lightweight).
Current computers allow exploiting many parallel modes at the same time for maximum combined effect.
[7] The two respective groups of researchers were unaware of each other’s use of the term SPMD to independently describe different models of parallel programming.
In Auguin’s SPMD model, the same (parallel) task (“same program”) is executed on different (SIMD) processors (“operating in lock-step mode”[1] acting on a part (“slice”) of the data-vector.
Specifically, in their 1985 paper[2] (and similarly in[3][1]) is stated: “we consider the SPMD (Single Program, Multiple Data) operating mode.
Starting around the same timeframe (in late 1983 – early 1984), the SPMD term was proposed by Frederica Darema (at IBM at that time, and part of the RP3 group) to define a different SPMD computational model that she proposed,[6][5][4] as a programming model which in the intervening years has been applied to a wide range of general-purpose high-performance computers (including RP3 - the 512-processor IBM Research Parallel Processor Prototype) and has led to the current parallel computing standards.
