Data stream management system

However, in contrast to a DBMS, a DSMS executes a continuous query that is not only performed once, but is permanently installed.

Since most DSMS are data-driven, a continuous query produces new results as long as new data arrive at the system.

This basic concept is similar to Complex event processing so that both technologies are partially coalescing.

One important feature of a DSMS is the possibility to handle potentially infinite and rapidly changing data streams by offering flexible processing at the same time, although there are only limited resources such as main memory.

There are different approaches to limit the amount of data in one pass, which can be divided into two classes.

The algorithms range from selecting random data points called sampling to summarization using histograms, wavelets or sketching.

Instead of memorizing each data point, the synopsis only holds the sum and the number of items.

However, most of them are based on SQL, such as the Continuous Query Language (CQL), StreamSQL and ESP.

A query plan is a directed graph where the nodes are operators and the edges describe the processing flow.

The logical query plan can be optimized, which strongly depends on the streaming model.

The basic concepts for optimizing continuous queries are equal to those from database systems.

However, it is possible to observe a data stream for a certain time to obtain some statistics.

The bottom of the graph (of the query plan) is connected to the incoming sources, which can be everything like connectors to sensors.