Message passing in computer clusters

All computer clusters, ranging from homemade Beowulfs to some of the fastest supercomputers in the world, rely on message passing to coordinate the activities of the many nodes they encompass.

As the number of nodes in a cluster increases, the rapid growth in the complexity of the communication subsystem makes message passing delays over the interconnect a serious performance issue in the execution of parallel programs.

Before a large computer cluster is assembled, a trace-based simulator can use a small number of nodes to help predict the performance of message passing on larger configurations.

[4][5] Historically, the two typical approaches to communication between cluster nodes have been PVM, the Parallel Virtual Machine and MPI, the Message Passing Interface.

It provides run-time environment for message-passing, task and resource management, and fault notification and must be directly installed on every cluster node.

The specification emerged in the early 1990 out of discussions between 40 organizations, the initial effort having been supported by ARPA and National Science Foundation.

However, given the high level nature of this profile, this type of information only provides a glimpse at the real behavior of the communication system.

[13][14][15] When a large scale, often supercomputer level, parallel system is being developed, it is essential to be able to experiment with multiple configurations and simulate performance.

[3] Systems such as BIGSIM provide these facilities by allowing the simulation of performance on various node topologies, message passing and scheduling strategies.

[4] At the analytical level, it is necessary to model the communication time T in term of a set of subcomponents such as the startup latency, the asymptotic bandwidth and the number of processors.

The communication overhead for MPI message passing can thus be simulated and better understood in the context of large-scale parallel job execution.

[18] Unlike MPI-Sim, BIGSIM is a trace-driven system that simulates based on the logs of executions saved in files by a separate emulator program.