Angel \”Java\” Lopez on Blog

February 15, 2011

Azure: Fractal application

Filed under: .NET, Azure, Cloud Computing, Distributed Computing — ajlopez @ 10:21 am

In January, I reimplemented my Fractal application, now using Azure (my Azure-related posts). The idea is to calculate each sector of a fractal image, using the power of worker roles, store the result in blobs, and consume them from a WinForm application.

This is the solution:

The source code is in my AjCodeKatas Google project. The code is at:

http://code.google.com/p/ajcodekatas/source/browse/#svn/trunk/Azure/AzureFractak

If you are lazy to use SVN, this is the current frozen code: AzureFractal.zip.

The projects in the solution:

AzureFractal: the Azure cloud definition.

Fractal: it contains my original code from previous fractal applications. An independent library class.

Fractal.Azure: serialization utilities of fractal info, and a service facade to post that info to a Azure message queue.

AzureLibrary: utility classes I used in other Azure examples. They are evolving in each example.

FractalWorkerRole: the worker role that consumes messages indicating what sector (rectangle) of the Mandelbrot fractal to calculate.

Fractal.GUI: a client WinForm project that sends and receives message to/from the worker role, using Azure queues.

You should configure the solution to have a multiple startup:

The WinForm application sends a message to a queue, with the info about the fractal sector to calculate:

private void Calculate()
{
    Bitmap bitmap = new Bitmap(pcbFractal.Width, 
       pcbFractal.Height);
    pcbFractal.Image = bitmap;
    pcbFractal.Refresh();
    realWidth = realDelta * pcbFractal.Width;
    imgHeight = imgDelta * pcbFractal.Height;
    realMin = realCenter - realWidth / 2;
    imgMin = imgCenter - imgHeight / 2;
    int width = pcbFractal.Width;
    int height = pcbFractal.Height;
    Guid id = Guid.NewGuid();
    SectorInfo sectorinfo = new SectorInfo()
    {
        Id = id,
        FromX = 0,
        FromY = 0,
        Width = width,
        Height = height,
        RealMinimum = realMin,
        ImgMinimum = imgMin,
        Delta = realDelta,
        MaxIterations = colors.Length,
        MaxValue = 4
    };
    Calculator calculator = new Calculator();
    this.queue.AddMessage(
        SectorUtilities.FromSectorInfoToMessage(sectorinfo));
}

The worker role reads messages from the queue, and deserialize SectorInfo:

while (true)
{
    CloudQueueMessage msg = queue.GetMessage();
    if (msg != null)
    {
        Trace.WriteLine(string.Format("Processing {0}", msg.AsString));
        SectorInfo info = SectorUtilities.FromMessageToSectorInfo(msg);

If the sector is too big, new messages are generated:

if (info.Width > 100 || info.Height > 100)
{
    Trace.WriteLine("Splitting message...");
    for (int x = 0; x < info.Width; x += 100)
        for (int y = 0; y < info.Height; y += 100)
        {
            SectorInfo newinfo = info.Clone();
            newinfo.FromX = x + info.FromX;
            newinfo.FromY = y + info.FromY;
            newinfo.Width = Math.Min(100, info.Width - x);
            newinfo.Height = Math.Min(100, info.Height - y);
            CloudQueueMessage newmsg = 
              SectorUtilities.FromSectorInfoToMessage(newinfo);
            queue.AddMessage(newmsg);
        }
}

If the sector is small enough, then it is processed:

Trace.WriteLine("Processing message...");
Sector sector = calculator.CalculateSector(info);
string blobname = string.Format("{0}.{1}.{2}.{3}.{4}", 
info.Id, sector.FromX, sector.FromY, sector.Width, sector.Height);
CloudBlob blob = blobContainer.GetBlobReference(blobname);
MemoryStream stream = new MemoryStream();
BinaryWriter writer =new BinaryWriter(stream);
foreach (int value in sector.Values)
    writer.Write(value);
writer.Flush();
stream.Seek(0, SeekOrigin.Begin);
blob.UploadFromStream(stream);
stream.Close();
CloudQueueMessage outmsg = new CloudQueueMessage(blobname);
outqueue.AddMessage(outmsg);

A blob with the result is generated, and a message is sent to another queue to notify the client application.

The WinForm has a thread with a loop reading messages from the second queue:

string blobname = msg.AsString;
CloudBlob blob = this.blobContainer.GetBlobReference(blobname);
MemoryStream stream = new MemoryStream();
blob.DownloadToStream(stream);
blob.Delete();
this.inqueue.DeleteMessage(msg);
string[] parameters = blobname.Split('.');
Guid id = new Guid(parameters[0]);
int fromx = Int32.Parse(parameters[1]);
int fromy = Int32.Parse(parameters[2]);
int width = Int32.Parse(parameters[3]);
int height = Int32.Parse(parameters[4]);
int[] values = new int[width * height];
stream.Seek(0, SeekOrigin.Begin);
BinaryReader reader = new BinaryReader(stream);
for (int k = 0; k < values.Length; k++)
    values[k] = reader.ReadInt32();
stream.Close();
this.Invoke((Action<int,int,int,int,int[]>) ((x,y,h,w,v) 
=> 
this.DrawValues(x,y,h,w,v)), fromx, fromy, width, height, values);

Note the use of .Invoke to run the drawing of the image in the UI thread.

This is the WinForm app, after click on Calculate button. Note that the sectors are arriving:

There are some blob sectors that are still not arrived. You can drag the mouse to have a new sector:

You can change the colors, clicking on New Colors button:

This is a sample application, a “proof-of-concept”. Probably, you will get a better performance if you use a single machine. But the idea is that you can defer work to worker roles, specially if the work can be do in parallel (imagine a parallel render machine, for animations). If you run these an application in Azure, with many worker roles, the performance could be improved.

Next steps: implement a distributed web crawler, try distributed genetic algorithm, running in the Azure cloud.

Keep tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

Theme: Shocking Blue Green. Get a free blog at WordPress.com

Follow

Get every new post delivered to your Inbox.

Join 67 other followers