Category Archives: AjMessages

Introducing AjProcessor (Part 1)

Last month, I was working in AjProcessor code, as part of my code katas Google project:

The idea is the evolution of some exploratory coding with AjMessages and other examples. I want to have an application, based in message passing, that can be run in a grid of heterogeneus machines. The application could 
be partitioned in steps, and each step could run in the same host machine, or on a another machine. The deployment in machines should be transparent to the writing of the application code.

Some of those goals were reached with AjMessages, but this time, I want a more clear kick off, based on the lesson learnt of my previous attempts.

First, I want to write down some basic ideas, to explain the motivation of the initial code written for AjProcessor. The basic idea, is to have message processors. A message processor is a code that receives a message, and process it.


The message has a payload (an arbitrary object, string, anything), and additional properties, with key/value.

Another essential brick in this lego-like application, is the element that publish messages. That is, there is a publisher:


An outgoing message could be received by any other code. The same message can be processed by more than one processor:


It’s like the publish/subscribe pattern. Another pattern to take into account, is a router-like component. Depending on the message (property, content), it could be send to different targets.


Frequently, a component will implement both roles, message processor and message publisher. In order to call plain old .NET objects, it would be nice to have a processor that receives a message, take some part of the message (i.e. the payload), and send it as a parameter to one its methods. The return value could feed a new message.


The components can be arranged as a chain, implementing a pipeline to process a message:


A more complex arrangement could receive a message, and forward it to different pipelines, according to some property or content in incoming message.


(this concept could be mapped to an application in AjMessage, but without the idea of distributed processing). A more interesting idea is to run that kind of pipeline routers in many machines


AjProcessor infrastructure is in charge of the serialization/routing/deserialization of messages between host machine. It could be WCF or anything else. The basic idea is to have a pluggable transport.

Well, these are the seed ideas behind the project. In an upcoming post, I’ll explain some of the current code (only few interfaces and classes, now).

Angel “Java” Lopez

Distributed Web Crawler using AjMessages

Last January, I update my AjMessages project to support communication using DSS/CCR and Windows Communication Foundation. AjMessages is a program that can run in a distributed way, sending asynchronous messages from one logical node to another. The logical nodes can be hosted in one or more physical machines. More info about AjMessages at:

AjMessages- a message processor
Distributed Applications with AjMessages using DSS/CCR

You can download the source code from

I had the idea of write a distributed message processor. Years ago, I wrote my first attempt, named AjServer

Hacia el AjServer (Spanish Post)

After that project, I met Fabriq ideas and implementation, where there are logical message handler, that can be distributed in a transparent way, via configuration:

In 2007, I wrote the first version of AjMessages, as a proof of concept: I could reproduce most of the Fabriq ideas, trying to raise some abstraction, in order to have a little more flexibility. Fabriq was more SOA-oriented. AjMessages is oriented to arbitrary message processing, to have a grid-like application.

This is the current AjMessages solution:

There is core project, and two transport projects: one for WCF support, and another to DSS/CCR (Microsoft Robotics technology). If you don’t have DSS/CCR, you can remove the related projects. The core AjMessages is free of dependencies on such technologies. AjMessages.SampleApp contains simple handler: a decrement handler, that takes a message with an integer, and produces a new message with a new decremented integer. AjMessages.WebCrawler implements the message and handlers that make a distributed web crawler application. You can run it in only one machine or in many hosts.

Each AjMessage message has:

Body: an arbitrary payload
Headers: additional information, key-value.
Action: describing the target of the message, using pattern Application/Node/Action

(more detail in the mentioned post AjMessages- a message processor)

Let’s explore the distributed web crawler example. This application is composed by nodes, that can be orchestrated to visit a link, and gets from that initial page all the links and related pages, up to a level. Logically, it can be described as:

The first messages is send to the node Controller, action Dispatch. It contains the initial page to visit. The message is enriched, and passed to node Controller, action Resolve. It’s in charge to keep a list of visited page, and to control the deep of processing. If the link is aproved, then a message is send to node Downloader, action Download. The content of the page is downloaded, added to the message, and forwarded to node Harvester, action Harvest. This action analyze the content of the page, and emits many message, one for each link found in  it. The recipient of the new links is the node Resolver, action Resolve. So, the process continues up to visit all pages (with a limit on the deep of processing).

This application can be described using an XML configuration file, example:

<?xml version="1.0" encoding="utf-8" ?> <AjMessages> <Application Name="WebCrawler"> <Node Name="Dispatcher"> <Handler Name="DispatcherHandler" Type="AjMessages.WebCrawler.Handlers.Dispatcher, AjMessages.WebCrawler"/> <Action Name="Dispatch" Handler="DispatcherHandler"/> </Node> <Node Name="Harvester"> <Handler Name="HarvesterHandler" Type="AjMessages.WebCrawler.Handlers.Harvester, AjMessages.WebCrawler"/> <Action Name="Harvest" Handler="HarvesterHandler"/> </Node> <Node Name="Downloader"> <Handler Name="DownloaderHandler" Type="AjMessages.WebCrawler.Handlers.Downloader, AjMessages.WebCrawler"/> <Action Name="Download" Handler="DownloaderHandler"/> </Node> <Node Name="Controller"> <Handler Name="ResolverHandler" Type="AjMessages.WebCrawler.Handlers.Resolver, AjMessages.WebCrawler"/> <Action Name="Resolve" Handler="ResolverHandler"/> </Node> </Application> </AjMessages>

An application is composed by nodes. Each node can be viewed as a “logical class”. Each node can handle Actions, that are the targets of messages. An action can be composed by one or more steps, of message handler. This is the extensibility point of the application: you must provided the steps, the message handler, and write a configuration file to orchestrate the message processing.

But one thing is the application, and another its distribution over physical machines. You can have two hosts, and install different logical nodes in each one:

In this diagram, the Dispatcher and Resolver are in one host, and the Downloader and Harvester in the other one. But you can put harversters in each host, or in twenty machines. It’s up to you how to distribute the load of the work. When a message is sent to a target, AjMessage forwards it to an appropiate host, that have a node capable of attending the message action.

The distribution of logical nodes into physical host is defined via configuration, an example:

<?xml version="1.0" encoding="utf-8" ?> <AjMessages> <Host Name="Server1" Address="http://localhost:50002/AjMessages" Activate="true"> <Application Name="AjMessages"> <Node Name="Administration"/> </Application> <Application Name="WebCrawler"> <Node Name="Controller"/> <Node Name="Dispatcher"/> <Node Name="Harvester"/> <Node Name="Downloader"/> </Application> </Host> <Host Name="Server2" Address="http://localhost:50003/AjMessages"> <Application Name="AjMessages"> <Node Name="Administration"/> </Application> <Application Name="WebCrawler"> <Node Name="Dispatcher"/> <Node Name="Harvester"/> <Node Name="Downloader"/> </Application> </Host> </AjMessages>

Running the Web Crawler example

You can test the program, launching two host, in the same machine. Run the console program AjMessage.Console. Then enter the command


This command launches a second host. Now, you are ready to configure the two hosts. In the first one, enter:

load ConfigurationServer1.xml

This file loads the initial AjMessage application, and define a WCF endpoint to listen messages. Then, enter:

load ConfigurationWebCrawler.xml

This command loads the Web Crawler node definitions (no deploy info yet). The third command define the distribution in hosts:

load ConfigurationWebCrawlerNode1.xml

Now, back to second host console. Enter the corresponding commands:

load ConfigurationServer2.xml
load ConfigurationWebCrawler.xml
load ConfigurationWebCrawlerNode2.xml

You are ready to launch the first web crawling. Go to first console, and enter:

send WebCrawler/Dispatcher/Dispatch

(that is my Spanish non technical blog… 😉

The web crawling begins to work, in both consoles. A typical view on first console:

Next steps

I should fix some problems in DSS/CCR tranport (I could run a simpler app, but not the web crawler). Two points to implements:

– Remote configuration of nodes
– Distribution of message handlers bits to remote nodes
– One configuration for all the nodes (now, each node is configured separately)

I began to experiment with a more abstract way of processing message, that should implement uses cases like:

– A simple hello world
– Decrement example, as in AjMessages sample app (see AjMessage.SampleApp)
– Distributed web crawler
– Enterprise Service Bus-like app

My first steps at:

But the project is still in its infancy. Keep tuned!

Angel “Java” Lopez

Distributed applications with AjMessages using DSS/CCR

Well, it’s only a project. The sample application is minimal. But it’s the evolution of my previous work with AjMessages via Windows Communication Foundation:

AjMessages- a message processor

Last year, I wrote that example, and another one that uses DSS/CCR. Only the first was published, now, I’m reviewing my DSS/CCR implementation, mounted on Microsoft Robotics technology. Currently, you can download the code from CodePlex at:

Both examples were based on Fabriq project ideas, trasposed to new technologies. Recently, Arvindra Sehmi posted about a new version of Fabriq, named Fabriq4Dss, that is using DSS/CCR in a more powerful ways. He prepared a presentation for JAOO, read more at:

JAOO 2008 Presentation

Keep tuned!

The solution

It’s composed by three projects:

The first one is a core implementation, that is independent of the transport to use for distributed messages:

The second project contains a simple message handler to use in the demo. The third project is the DSS service component to use as a host node:


For a more detailed description of application, message, hosts, read my previous post:

AjMessages- a message processor

The application

A configuration XML file define the applications to run. An application has:

  • Nodes: a collection of actions to process.
  • Handlers: objects to process a message. A handler can be a composite of other handlers.
  • Actions: named entry points to a handler

An example application definition (from AjMessages.SampleApp\Configurations\Server1.xml):

<?xml version="1.0" encoding="utf-8" ?> <AjMessages> <Application Name="AjMessages"> <Node Name="Administration"> <Handler Name="ConfigureHandler" Type="AjMessages.ConfigureHandler, AjMessages"/> <Handler Name="PrintHandler" Type="AjMessages.PrintHandler, AjMessages"/> <Handler Name="PrintMessageHandler" Type="AjMessages.PrintMessageHandler, AjMessages"/> <Handler Name="ConfigurePipeline"> <Handler Name="PrintHandler"> <Property Name="Text" Value="Reconfiguring server..."/> </Handler> <Handler Name="ConfigureHandler"/> </Handler> <Action Name="Configure" Handler="ConfigurePipeline"/> </Node> </Application> <Application Name="App1"> <Node Name="Node1"> <Handler Name="PrintMessageHandler" Type="AjMessages.PrintMessageHandler, AjMessages"/> <Handler Name="PostHandler" Type="AjMessages.PostHandler, AjMessages"/> <Handler Name="DecrementHandler" Type="AjMessages.SampleApp.Handlers.DecrementHandler, AjMessages.SampleApp"/> <Handler Name="Pipeline1"> <Handler Name="DecrementHandler"/> <Handler Name="PostHandler"> <Property Name="Action" Value="App1/Node1/Process"/> </Handler> </Handler> <Action Name="Process" Handler="Pipeline1"/> </Node> <Node Name="Node2"/> </Application> <Host Name="Server1" Activate="true"> <Application Name="AjMessages"> <Node Name="Administration"/> </Application> <Application Name="App1"> <Node Name="Node1"/> <Node Name="Node2"/> </Application> </Host> <Host Name="Server2" Address="http://localhost:50002/ajmessages/node"> <Application Name="AjMessages"> <Node Name="Administration"/> </Application> <Application Name="App1"> <Node Name="Node1"/> <Node Name="Node2"/> </Application> </Host> </AjMessages>

Running the example

Load the solution in Visual Studio 2008 (you must have Microsoft Robotics Developer Studio, a CTP version of this year, and you must configure the output path of the projects to point to your MRDS directory). Running the DSS project lunch a DSS Host that shows a controller window:

The host is running on ports 50000/50001. Configure the host using the AjMessages.SampleApp\Configurations\Server1.xml.

You can launch a second host from DSS prompt:

bin/dsshost /p:50002 /t:50003

A second form controller appears:

Use the other config file AjMessages.SampleApp\Configurations\Server2.xml and port 50002.

You can send a message to the sample app. The message is only an integer, that is decremented in each node. The decrement node is installed in the two servers, so the message travels from one to another server.

Next steps

I want to support an arbitrary object, serialized in DSS, and uses WCF as a transport, too. The previous post describes the WCF implementation.

Angel “Java” Lopez

Messages everywhere

Recently, I wrote a post about Message Passing Interface:

Message Passing Interface, CCR, DSS, and Pure MPI.NET

I used to pass message between agents in my example of a web crawler:

Distributed Agents using DSS/VPL

The passing of messages between components, agents, objects, is a feature that deserves more study. I guess we could write new kinds of applications, using one-way message passing, so we could abandon the call-a-method current way of doing things. Let’s explore first, some message passing applications (not only one-way style).

In the Wikipedia entry about Message Passing:

we can read Alan Kay opinion:

Alan Kay has suggested that he placed too much emphasis on objects themselves and not enough on the messages being sent between them. Message passing enables extreme late binding in systems.

If you develop systems with Smalltalk, Self or alikes, you’ll notice that the message is a first citizen, in many implementations, a full object, not only a way of call a method.

There is another place for use message passing. For 25 years, QNX operating systems uses the message passing paradigm to run a real time kernel.

I found this interview at Dr. Dobb’s to Sebastien Marineau-Mes, VP of engeneering at QNX:

Real Time OS in Today s World

Sebastien talks about the use of QNX in current market, and the challenge that multi-core machine could create on legacy code.

Remember: all the QNX kernel is based on message passing, although its messages are not one-way, and the passing is optimized to not incurr in loss of performance (more details, see QNX at Wikipedia). I see that many of these challenges and opportunities could be translated to the use, not only to multi-core, but to “multi-machines” in a grid. There many forces that are conspiring to bring these topics to current arena:

– We have a better understanding of agents, message passing and other ideas

– Normal hardware is cheap

– Each year, there are more application with higher level needs of scalabity (the user base will be the whole Internet in the near future, for non-trivial app)

– Many application must interact with other application in the same enterprise or in the net, and messages are an effective way to do that.

– In order to face that challenges, we must begin to abandon n-tier-only apps, to a cloud, grid or “something alike” schema.

I could imagine languages and technologies, based on message passing features. That is one of the reasons I’ve been involved exploring simples and crazy ideas with AjMessages, AjAgents, and Distributed Software Services during last months. I hope to write more about these subjects:

– Another example of distributed agents using DSS/VPL

– AjMessages ported to DSS (I had an implementation last year, but I never published it, I published the WCF only version)

– One-way messages implemented as primitives in AjTalk (Smalltalk-like interpreter)

– Deferred/Concurrent/LazyEvaluation/SomethingAlike implemented in AjBasic (using CCR?)

– Blog about a better-finished app running in a grid of DSS hosts (my team was working hard on this, last months).

Incidentally, you can read more about use cases applied in the real world using CCR/DDS, in Arvindra Sehmi’s post:

CCR/DSS Use Cases in the Enterprise

So many ideas…. only one life…. Should I begin to parallized myself? ajlopez-in-a-grid…. 😉

Angel “Java” Lopez

Agents in a Grid

Recently, I was involved in the development of an application that runs on a grid of diskless machines. I was an exciting project, and now, after the rush of shipping it, I want to write down some ideas to explore.

In this post, I use the term “agent”, that is a bit fuzzy. I won’t define precisely the concept, I want to use as a base term to refine in a future. By now, let explore some basic (naive?) ideas to better understanding of the problems related to agents and grid applications. Yes, some ideas presented here could be viewed as naive, but I feel that this is a needed exercise, to grasp the key concepts and problems to be solved in these kind of applications. At the end of this article, I’ll present some implementation suggestions.

I described some applications to run in a grid in my previous post:

Grid Computing Programming

Agent concepts

In this post, agent is a piece of software, with behaviour and state. It runs inside an agent host, a host application that provides the base services so an agent can “live” and run. An agent will be represented as in this drawing:

Agent communication patterns

There is a lot of literature about agents communication, from simple techniques to elaborated ones as contracts, bids, and more. We can have agents with beliefs, desires and intention. In this post, an agent is simpler: only an object with state, that can send messages. It can receive stimula from other agents and from the host environment.

The simplest communication pattern is composed of an agent sending a message to another agent:

Some notes:

– The sender agent knows the receiver agent. That is, some form of identity must be implemented. The message is not send to anyone: the sender intention is to send the message to a determined receiver.

– The message carries data, and it must be understood by the receiver, possibly invoking one of its methods.

– The sender doen’t wait for a response. It’s not interested in a return message or data.

– The agents could reside in different physical machines in the grid, in a transparent way.

During its life an agent can send many messages to differents target agents, each one is a known agent for the sender:

Sometimes, the sender agent will receive a message from the receiver agents, notifying some work done, or informing some processed data. Depending on the application, the response message could carry information to identify the original message:

In this case, the sender must be prepared to receive the response in an asynchronous way. This could be an interesting problem to solve: an agent can send many messages, and it would be better if it can still run without receiving all the responses in time. For example, in an board game application, one agent could delegate some tree branch exploring to other agents, and, after some time, it should be able to take a decision, without all the information.

Clouds in the grid sky

Now, another case: an agent could be interested in send a message, without forward it to a determined receptor. Instead, it could be send to a cloud of agents, so anyone interested in the message could have the opportunity of processing it.

(Ok, I’m a bit odd creative in my drawings… 😉

This feature could be implemented using these strategies:

– An agent could send a message to a blackboard system.

– An agent could send a message to host application, indicating a topic (like in a message queue), so any subscribed agent will receive the message. An variant: only some subscriptors receive the message, depending on application settings.

– An agent could send a message targeting some service provider definition. A service provider is an agent that declares, at the beginning of its life, its capabilities and service that is capable to provide. 

As example, we could take a chess/board game application (I prefer the game of go). An agent in such application can send a message reclaiming to solve some attack position. In a blackboard system, it will publish the request in the blackboard. In a topic system, it deposits the message on topic “attackers”. In a service provider strategy, it sends the message to one or many AttackService provider.

As in the previous pattern, an agent can receive an asynchronous response, now from “the cloud”:


Agent Duplication

An agent has behaviour and state. If the agent must fork its jobs, it could take the way of duplicating itself:

It’s a bit strange action, but it could be useful, depending on application at hand.

Agents and the Grid

Each agent can be hosted in a grid node. The message sending mechanism must be able to send a remote message to another node. The host application keeps a list of agents by identity, and knows which one is local or remote. An acid test: a grid application with agents must be able to run all in one machine, or in grid, without changing the code or algorithm.


As in another grid implementations (discussed in my previous post), a central server is in charge of distributing the tasks between the grid nodes. That server could receive tasks from outside application or even, from other grid. “Grid as a Service” is a new buzzword to apply.

Moving agent

An agent can initiate its activities in one node. But at point, it can decide to continue the work in another node (the host application in that node could take that decision, too). Then, its state should be send from one node to another.

Note that the agent behaviour (perhaps implemented in compiled code) doesn’t travel.

Injecting behaviour

The behaviour of each kind of agent could be expressed in compiled code (.jar files in Java, assemblies in .NET). Other alternatives are possible: the behaviour could be specified in a specific scripting agent language.

If the behaviour is expressed in compiled form, one or more servers have the responsability of storing and distributing that components:

Implementation ideas

Most of these ideas can be implemented in any appropiate language/technology, like Java and .NET, that supports multi threading, remote invocation, message serialization, etc…

Last months I was working on my pet projects AjMessages and AjAgents, more info at previous posts:

AjMessages: a message processor

Agents using Concurrency and Coordination Runtime (CCR)

(I have a new project, named AjGrid, not published yet). For this post, I guess that AjAgents could be an appropiate implementation. AjMessages now has remote support, but it’s more oriented to a pipeline processing: it’s difficult to implement in such disposition some of the above ideas.

I must add some features to AjAgents (actually, AjAgents is a local, proof of concept implementation):

Configuration: Load and creation of agents in runtime, from a configuration file, or on the fly, in the middle of a run.

Remote assembly: So a grid node can be injected with new agents.

Agent identification: To identify an agent in a unique way (an UID could be a solution).

Message transport: Windows Communication Foundation is a candidate.

But another approach is to take Decentralized System Services (DSS) from Microsoft Robotics Studio. An agent can be implemented as a DSS Service, running in a DSS Host. The intermachine communication would be implemented using DSSP as protocol.

Stay tuned, to see some code in the coming weeks.

Angel “Java” Lopez

AjMessages: a message processor

Following ideas from Fabriq project, I wrote last week an example of a server application that can receive and process OneWay Windows Communication Foundation (WCF) messages. The server can host applications, and each application is composed by nodes, that have actions. The example code can be download from

The message

The message to exchange is a WCF message (the original Fabriq was written before the WCF era). The main information that AjMessages uses is the Headers.Action property. This string has the name of the application, node, and action to invoke.

The action

It’s a string with format:


An application is a set of nodes. A server can host some or all nodes of an application. But the same node can run in different servers. In this way, the internal process of the server can forward a message to other server: the action is not a physical address, but a logical one. Each server knows the address and nodes in other servers, and then, it can decide to forward a message to another server.

The application

A configuration XML file define the applications to run. An application has:

  • Nodes: a collection of actions to process.
  • Handlers: objects to process a message. A handler can be a composite of other handlers.
  • Actions: named entry points to a handler

An example application definition:

  <Application Name="App1"> 
    <Node Name="Node1"> 
      <Handler Name="PrintMessageHandler" 
Type="AjMessages.PrintMessageHandler, AjMessages"/> 
      <Handler Name="PostHandler" 
Type="AjMessages.PostHandler, AjMessages"/> 
      <Handler Name="DecrementHandler" 
Type="AjMessages.Console01.DecrementHandler, AjMessages.Console01"/> 
      <Handler Name="Pipeline1"> 
        <Handler Name="DecrementHandler"/> 
        <Handler Name="PostHandler"> 
          <Property Name="Action" 
      <Action Name="Process" 
    <Node Name="Node2"/> 


The handler

A handler must implements an interface:

    public interface IHandler 
        void Process(Context ctx); 

The context has a property Message, that points to the WCF message in process, and some other methods to use: a Server property that reference to the current server object, and a Post method, to send a new message. The idea is to put in the context all the needed external functionality to the handler, so this is decoupled of the server and message in process.

This is a departure of the original implementation in Fabriq, where the handler receives a message, not a context.

The host

A host is a logical unit that runs inside a server. A server is informed, via configuration, the hosts that are available in other servers, and which ones are local. An example:

  <Host Name="Host1" 

<Application Name="AjMessages"> 
      <Node Name="Administration"/> 
    <Application Name="App1"> 
      <Node Name="Node1"/> 
      <Node Name="Node2"/> 

Dynamic configuration

One of the key features implemented is a handler that reconfigure in runtime the running server. It receives a new XML configuration, and merges it with the current information. Then, using these handler, an application can send a message to a AjMessages server, and load, stop, update hosts, applications and nodes.

The solution

The class library AjMessages project contains the implementation of the example. The other projects are manual test to run that shows how to use the example.

Inside Configuration folder there are classes defined to load configuration data from an XML structure.

Handlers folder contains some predefined handlers to use, an important one is the ConfigurationHandler that permits to reconfigure dynamically the server while it’s running.

Ports folder has implementations of endpoints to receive and send message from an to other AjMessages servers.

PipelineHandler.cs is the composite handler, that contain other handlers.

LocalHost.cs is the implementation of a host running in the current server. It mantains a list of active nodes.

RemoteHost.cs mantains the info associated to a host running in other servers. Using this information, the local server knows how to forward a message, inspecting the remote hosts that have an active node.

Running the example

There is a console project AjMessages.Console01 that can be used as a manual test.

You can run the example, and enter the commands:

load ConfigurationServer1.xml
load ConfigurationHost1.xml
load ConfigurationHost2.xml
send App1/Node1/Process 20

The first command creates the server, using the configuration file. The second and third command reconfigure the server to install and run two hosts, with its applications and nodes. The last command send an integer message using App1/Node1/Process as action.

After that, you enter:


These launch another server console program. In the first one, enter:

load ConfigurationServer1.xml
load ConfigurationHost1.xml
load ConfigurationRemoteHost2.xml

and in the other server, enter:

load ConfigurationServer2.xml
load ConfigurationHost2.xml
load ConfigurationRemoteHost1.xml

In anyone, enter:

send App1/Node1/Process 20

and you’ll both servers exchanging messages.

In any moment, you can enter:


to obtain a list of commands to use.

Next steps

In a future installment of the example, I’ll rewrite the application to use a custom Message, capable of be transported via WCF, or DSS (Decentralized System Services), so the example could be hosted in different technologies. I must refactor some classes, for example, Server, that has many responsabilities.

Comments, suggestions, welcome!

Angel “Java” Lopez