Angel \”Java\” Lopez on Blog

February 15, 2011

Azure: Fractal application

Filed under: .NET, Azure, Cloud Computing, Distributed Computing — ajlopez @ 10:21 am

In January, I reimplemented my Fractal application, now using Azure (my Azure-related posts). The idea is to calculate each sector of a fractal image, using the power of worker roles, store the result in blobs, and consume them from a WinForm application.

This is the solution:

The source code is in my AjCodeKatas Google project. The code is at:

http://code.google.com/p/ajcodekatas/source/browse/#svn/trunk/Azure/AzureFractak

If you are lazy to use SVN, this is the current frozen code: AzureFractal.zip.

The projects in the solution:

AzureFractal: the Azure cloud definition.

Fractal: it contains my original code from previous fractal applications. An independent library class.

Fractal.Azure: serialization utilities of fractal info, and a service facade to post that info to a Azure message queue.

AzureLibrary: utility classes I used in other Azure examples. They are evolving in each example.

FractalWorkerRole: the worker role that consumes messages indicating what sector (rectangle) of the Mandelbrot fractal to calculate.

Fractal.GUI: a client WinForm project that sends and receives message to/from the worker role, using Azure queues.

You should configure the solution to have a multiple startup:

The WinForm application sends a message to a queue, with the info about the fractal sector to calculate:

private void Calculate()
{
    Bitmap bitmap = new Bitmap(pcbFractal.Width, 
       pcbFractal.Height);
    pcbFractal.Image = bitmap;
    pcbFractal.Refresh();
    realWidth = realDelta * pcbFractal.Width;
    imgHeight = imgDelta * pcbFractal.Height;
    realMin = realCenter - realWidth / 2;
    imgMin = imgCenter - imgHeight / 2;
    int width = pcbFractal.Width;
    int height = pcbFractal.Height;
    Guid id = Guid.NewGuid();
    SectorInfo sectorinfo = new SectorInfo()
    {
        Id = id,
        FromX = 0,
        FromY = 0,
        Width = width,
        Height = height,
        RealMinimum = realMin,
        ImgMinimum = imgMin,
        Delta = realDelta,
        MaxIterations = colors.Length,
        MaxValue = 4
    };
    Calculator calculator = new Calculator();
    this.queue.AddMessage(
        SectorUtilities.FromSectorInfoToMessage(sectorinfo));
}

The worker role reads messages from the queue, and deserialize SectorInfo:

while (true)
{
    CloudQueueMessage msg = queue.GetMessage();
    if (msg != null)
    {
        Trace.WriteLine(string.Format("Processing {0}", msg.AsString));
        SectorInfo info = SectorUtilities.FromMessageToSectorInfo(msg);

If the sector is too big, new messages are generated:

if (info.Width > 100 || info.Height > 100)
{
    Trace.WriteLine("Splitting message...");
    for (int x = 0; x < info.Width; x += 100)
        for (int y = 0; y < info.Height; y += 100)
        {
            SectorInfo newinfo = info.Clone();
            newinfo.FromX = x + info.FromX;
            newinfo.FromY = y + info.FromY;
            newinfo.Width = Math.Min(100, info.Width - x);
            newinfo.Height = Math.Min(100, info.Height - y);
            CloudQueueMessage newmsg = 
              SectorUtilities.FromSectorInfoToMessage(newinfo);
            queue.AddMessage(newmsg);
        }
}

If the sector is small enough, then it is processed:

Trace.WriteLine("Processing message...");
Sector sector = calculator.CalculateSector(info);
string blobname = string.Format("{0}.{1}.{2}.{3}.{4}", 
info.Id, sector.FromX, sector.FromY, sector.Width, sector.Height);
CloudBlob blob = blobContainer.GetBlobReference(blobname);
MemoryStream stream = new MemoryStream();
BinaryWriter writer =new BinaryWriter(stream);
foreach (int value in sector.Values)
    writer.Write(value);
writer.Flush();
stream.Seek(0, SeekOrigin.Begin);
blob.UploadFromStream(stream);
stream.Close();
CloudQueueMessage outmsg = new CloudQueueMessage(blobname);
outqueue.AddMessage(outmsg);

A blob with the result is generated, and a message is sent to another queue to notify the client application.

The WinForm has a thread with a loop reading messages from the second queue:

string blobname = msg.AsString;
CloudBlob blob = this.blobContainer.GetBlobReference(blobname);
MemoryStream stream = new MemoryStream();
blob.DownloadToStream(stream);
blob.Delete();
this.inqueue.DeleteMessage(msg);
string[] parameters = blobname.Split('.');
Guid id = new Guid(parameters[0]);
int fromx = Int32.Parse(parameters[1]);
int fromy = Int32.Parse(parameters[2]);
int width = Int32.Parse(parameters[3]);
int height = Int32.Parse(parameters[4]);
int[] values = new int[width * height];
stream.Seek(0, SeekOrigin.Begin);
BinaryReader reader = new BinaryReader(stream);
for (int k = 0; k < values.Length; k++)
    values[k] = reader.ReadInt32();
stream.Close();
this.Invoke((Action<int,int,int,int,int[]>) ((x,y,h,w,v) 
=> 
this.DrawValues(x,y,h,w,v)), fromx, fromy, width, height, values);

Note the use of .Invoke to run the drawing of the image in the UI thread.

This is the WinForm app, after click on Calculate button. Note that the sectors are arriving:

There are some blob sectors that are still not arrived. You can drag the mouse to have a new sector:

You can change the colors, clicking on New Colors button:

This is a sample application, a “proof-of-concept”. Probably, you will get a better performance if you use a single machine. But the idea is that you can defer work to worker roles, specially if the work can be do in parallel (imagine a parallel render machine, for animations). If you run these an application in Azure, with many worker roles, the performance could be improved.

Next steps: implement a distributed web crawler, try distributed genetic algorithm, running in the Azure cloud.

Keep tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

1 Comment »

  1. Excellent goods from you, man. I’ve understand your stuff previous to and you are just extremely great. I actually like what you have acquired here, certainly like what you are saying and the way in which you say it. You make it entertaining and you still care for to keep it sensible. I can’t wait to read much more
    from you. This is really a tremendous web site.

    Comment by blogspot.com — July 23, 2013 @ 2:49 pm


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: