The Message Passing Interface (MPI) is a portable message-passing standard designed to function on parallel computing architectures.
[1] The MPI standard defines the syntax and semantics of library routines that are useful to a wide range of users writing portable message-passing programs in C, C++, and Fortran.
The message passing interface effort began in the summer of 1991 when a small group of researchers started discussions at a mountain retreat in Austria.
Out of that discussion came a Workshop on Standards for Message Passing in a Distributed Memory Environment, held on April 29–30, 1992 in Williamsburg, Virginia.
Jack Dongarra, Tony Hey, and David W. Walker put forward a preliminary draft proposal, "MPI1", in November 1992.
These meetings and the email discussion together constituted the MPI Forum, membership of which has been open to all members of the high-performance-computing community.
Most of the major vendors of concurrent computers were involved in the MPI effort, collaborating with researchers from universities, government laboratories, and industry.
MPI provides parallel hardware vendors with a clearly defined base set of routines that can be efficiently implemented.
MPI provides a simple-to-use portable interface for the basic user, yet one powerful enough to allow programmers to use the high-performance message passing operations available on advanced machines.
The message-passing paradigm is attractive because of wide portability and can be used in communication for distributed-memory and shared-memory multiprocessors, networks of workstations, and a combination of these elements.
MPI "is a message-passing application programmer interface, together with protocol and semantic specifications for how its features must behave in any implementation.
Most MPI implementations consist of a specific set of routines directly callable from C, C++, Fortran (i.e., an API) and any language able to interface with such libraries, including C#, Java or Python.
MPI-4.0 is a major update that introduces large-count versions of many routines, persistent collective operations, partitioned communications, and a new MPI initialization method.
It also adds application info assertions and improves error handling definitions, along with various smaller enhancements.
Typically, for maximum performance, each CPU (or core in a multi-core machine) will be assigned just a single process.
Point-to-point operations, as these are called, are particularly useful in patterned or irregular communication, for example, a data-parallel architecture in which each processor routinely swaps regions of data with specific other processors between calculation steps, or a master–slave architecture in which the master sends new task data to a slave whenever the prior task is completed.
MPI_Reduce is often useful at the start or end of a large distributed calculation, where each processor operates on a part of the data and then combines it into a result.
[16] Since the C language does not allow a type itself to be passed as a parameter, MPI predefines the constants MPI_INT, MPI_CHAR, MPI_DOUBLE to correspond with int, char, double, etc.
These types of call can often be useful for algorithms in which synchronization would be inconvenient (e.g. distributed matrix multiplication), or where it is desirable for tasks to be able to balance their load while other processors are operating on data.
The newly spawned set of MPI processes form a new MPI_COMM_WORLD intracommunicator but can communicate with the parent and the intercommunicator the function returns.
For example, an implementation of sparse matrix-vector multiplications using the MPI I/O library shows a general behavior of minor performance gain, but these results are inconclusive.
Due to its vast performance benefits, MPI-IO also became the underlying I/O layer for many state-of-the-art I/O libraries, such as HDF5 and Parallel NetCDF.
[25][26] Many other efforts are derivatives of MPICH, LAM, and other works, including, but not limited to, commercial implementations from HPE, Intel, Microsoft, and NEC.
Although Java does not have an official MPI binding, several groups attempt to bridge the two, with different degrees of success and compatibility.
Here Java sockets or specialized I/O interconnects like Myrinet can support messaging between MPJ Express processes.
The OCamlMPI Module[36] implements a large subset of MPI functions and is in active use in scientific computing.
An 11,000-line OCaml program was "MPI-ified" using the module, with an additional 500 lines of code and slight restructuring and ran with excellent results on up to 170 nodes in a supercomputer.
It is enough for MPI to provide an SPMD-style program with MPI_COMM_WORLD, its own rank, and the size of the world to allow algorithms to decide what to do.
The MPI Forum reconvened in 2007 to clarify some MPI-2 issues and explore developments for a possible MPI-3, which resulted in versions MPI-3.0 (September 2012)[47] and MPI-3.1 (June 2015).
Astrophysicist Jonathan Dursi wrote an opinion piece calling MPI obsolescent, pointing to newer technologies like the Chapel language, Unified Parallel C, Hadoop, Spark and Flink.