Database scalability

[1] According to Marc Brooker: "a system is scalable in the range where marginal cost of additional workload is nearly constant."

One key software innovation in the late 1980s was to reduce update locking granularity from tables and disk blocks to individual rows.

This eliminated a critical scalability bottleneck, as coarser locks could delay access to rows even though they were not directly involved in a transaction.

This involved allowing multiple processors to handle database requests simultaneously, without blocking each other.

A much more significant change involved allowing distributed transactions to affect data stored on separate computers, using the two-phase commit protocol, establishing the shared-nothing architecture.

[5] Still later, Oracle introduced the shared-everything architecture, which provided full functionality on multi-server clusters.

[7] In the early twenty-first century, NoSQL systems gained favor over relational databases for some workloads.

It achieved near-linear scalability across multiple server farms, at the cost of features such as multi-row transactions and joins.

NewSQL systems attempt to combine NoSQL scalability with ACID transactions and SQL interfaces.

A related concept is elasticity, the ability of a system to transparently add and subtract capacity to meet changing workloads.

[11] Databases run on individual hardware ranging in capacity from smartwatches to supercomputers to multiple transparently reconfigurable server farms.

[3] Databases also scaled vertically to run on 64-bit microprocessors, multi-core CPUs, and large SMP multiprocessors, using multi-threaded implementations.

[13] A basic technique is to split large tables into multiple partitions based on ranges of values in a key field.

This scaling technique is particularly convenient for seldom or never-updated data, such as transaction history or tax tables.