Angel \”Java\” Lopez on Blog

July 29, 2014

SharpStorm Implementing Storm in C# (1) Initial Project

Filed under: .NET, C Sharp, Open Source Projects, SharpStorm, Storm — ajlopez @ 12:44 pm

In the last two years I was exploring the ideas of Storm (see https://storm.incubator.apache.org/), implementing a local topology Node.js. It’s time to practice more C # and TDD (Test-Driven Development), with:

https://github.com/ajlopez/SharpStorm

Current status of the project:

The idea is to implement Bolt, Spout and a TopologyBilder. See

https://storm.incubator.apache.org/about/simple-api.html

There are just three abstractions in Storm: spouts, bolts, and topologies. A spout is a source of streams in a computation. Typically a spout reads from a queueing broker such as Kestrel, RabbitMQ, or Kafka, but a spout can also generate its own stream or read from somewhere like the Twitter streaming API. Spout implementations already exist for most queueing systems.

A bolt processes any number of input streams and produces any number of new output streams. Most of the logic of a computation goes into bolts, such as functions, filters, streaming joins, streaming aggregations, talking to databases, and so on.

A topology is a network of spouts and bolts, with each edge in the network representing a bolt subscribing to the output stream of some other spout or bolt. A topology is an arbitrarily complex multi-stage stream computation. Topologies run indefinitely when deployed.

In the project are now defined interfaces, to have bolts and spouts, and topology builder to build a topology. For now, I want a local implementation (without distributed processing). Then, I hope to implement ACK algorithm that has original Storm, which lets you know when an incoming message was completely processed (including processes messages that occurred in the middle of the process).

Upcoming topics: discuss ISpout, iBOLT, and topology implementation.

Stay tuned

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

1 Comment »

  1. […] Storm has eased events processing at the Highly Concurrent Distributed Computing Level.SharpStorm seems to be the closest(partial) .NET implementation of Storm although one can opt to use Apache […]

    Pingback by A Technology(.NET) Architect’s Confession | Insight's Delight — July 13, 2015 @ 6:25 pm


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: