It bridges the performance gap between the processing speed of the compute nodes and the Input/output (I/O) bandwidth of the storage systems.
With the deployment of burst buffers, processes can quickly write their data to a burst buffer after one round of computation, instead of writing to the slow hard disk based storage system, and immediately proceed to the next round of computation without waiting for the data to be moved to the back-end storage system;[2][3] the data are then asynchronously flushed from the burst buffer to the storage system during the next round of computation.
[4][5] In another common use case, scientific applications can stage their intermediate data in and out of burst buffer without interacting with the slower storage systems.
[7][8][9][10] It also comes with the demand for a scalable metadata management strategy to maintain a global namespace for data distributed across all the burst buffers.
Hence, several well-known commercialized software products have been developed to manage this type of burst buffer, such as DataWarp and Infinite Memory Engine.