In the last two years I was exploring the ideas of Storm (see https://storm.incubator.apache.org/), implementing a local topology Node.js. It’s time to practice more C # and TDD (Test-Driven Development), with:
https://github.com/ajlopez/SharpStorm
Current status of the project:
The idea is to implement Bolt, Spout and a TopologyBilder. See
https://storm.incubator.apache.org/about/simple-api.html
There are just three abstractions in Storm: spouts, bolts, and topologies. A spout is a source of streams in a computation. Typically a spout reads from a queueing broker such as Kestrel, RabbitMQ, or Kafka, but a spout can also generate its own stream or read from somewhere like the Twitter streaming API. Spout implementations already exist for most queueing systems.
A bolt processes any number of input streams and produces any number of new output streams. Most of the logic of a computation goes into bolts, such as functions, filters, streaming joins, streaming aggregations, talking to databases, and so on.
A topology is a network of spouts and bolts, with each edge in the network representing a bolt subscribing to the output stream of some other spout or bolt. A topology is an arbitrarily complex multi-stage stream computation. Topologies run indefinitely when deployed.
In the project are now defined interfaces, to have bolts and spouts, and topology builder to build a topology. For now, I want a local implementation (without distributed processing). Then, I hope to implement ACK algorithm that has original Storm, which lets you know when an incoming message was completely processed (including processes messages that occurred in the middle of the process).
Upcoming topics: discuss ISpout, iBOLT, and topology implementation.
Stay tuned
Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez
Angel \"Java\" Lopez on Blog 