Apache Arrow

It contains a standardized column-oriented memory format that is able to represent flat and hierarchical data for efficient analytic operations on modern CPU and GPU hardware.

[2][3][4][5][6] This reduces or eliminates factors that limit the feasibility of working with large sets of data, such as the cost, volatility, or physical constraints of dynamic random-access memory.

The project includes native software libraries written in C, C++, C#, Go, Java, JavaScript, Julia, MATLAB, Python, R, Ruby, and Rust.

Arrow allows for zero-copy reads and fast data access and interchange without serialization overhead between these languages and systems.

[11] The hardware resource engineering trade-offs for in-memory processing vary from those associated with on-disk storage.