Memory hierarchy

Since response time, complexity, and capacity are related, the levels may also be distinguished by their performance and controlling technologies.

[1] Memory hierarchy affects performance in computer architectural design, algorithm predictions, and lower level programming constructs involving locality of reference.

Designing for high performance requires considering the restrictions of the memory hierarchy, i.e. the size and capabilities of each component.

As a result, the CPU spends much of its time idling, waiting for memory I/O to complete.

Taking optimal advantage of the memory hierarchy requires the cooperation of programmers, hardware, and compilers (as well as underlying support from the operating system): Many programmers assume one level of memory.