Apache Storm

Apache Storm is a distributed stream processing computation framework written predominantly in the Clojure programming language.

Originally created by Nathan Marz[2] and team at BackType,[3] the project was open sourced after being acquired by Twitter.

[4] It uses custom created "spouts" and "bolts" to define information sources and manipulations to allow batch, distributed processing of streaming data.

At a superficial level the general topology structure is similar to a MapReduce job, with the main difference being that data is processed in real time as opposed to in individual batches.

Additionally, Storm topologies run indefinitely until killed, while a MapReduce job DAG must eventually end.