Angel “Java” Lopez on Blog

Recently, I wrote a post about Message Passing Interface:

Message Passing Interface, CCR, DSS, and Pure MPI.NET

I used to pass message between agents in my example of a web crawler:

Distributed Agents using DSS/VPL

The passing of messages between components, agents, objects, is a feature that deserves more study. I guess we could write new kinds of applications, using one-way message passing, so we could abandon the call-a-method current way of doing things. Let’s explore first, some message passing applications (not only one-way style).

In the Wikipedia entry about Message Passing:

http://en.wikipedia.org/wiki/Message_passing

we can read Alan Kay opinion:

Alan Kay has suggested that he placed too much emphasis on objects themselves and not enough on the messages being sent between them. Message passing enables extreme late binding in systems.

If you develop systems with Smalltalk, Self or alikes, you’ll notice that the message is a first citizen, in many implementations, a full object, not only a way of call a method.

There is another place for use message passing. For 25 years, QNX operating systems uses the message passing paradigm to run a real time kernel.

I found this interview at Dr. Dobb’s to Sebastien Marineau-Mes, VP of engeneering at QNX:

Real Time OS in Today s World

Sebastien talks about the use of QNX in current market, and the challenge that multi-core machine could create on legacy code.

Remember: all the QNX kernel is based on message passing, although its messages are not one-way, and the passing is optimized to not incurr in loss of performance (more details, see QNX at Wikipedia). I see that many of these challenges and opportunities could be translated to the use, not only to multi-core, but to “multi-machines” in a grid. There many forces that are conspiring to bring these topics to current arena:

- We have a better understanding of agents, message passing and other ideas

- Normal hardware is cheap

- Each year, there are more application with higher level needs of scalabity (the user base will be the whole Internet in the near future, for non-trivial app)

- Many application must interact with other application in the same enterprise or in the net, and messages are an effective way to do that.

- In order to face that challenges, we must begin to abandon n-tier-only apps, to a cloud, grid or “something alike” schema.

I could imagine languages and technologies, based on message passing features. That is one of the reasons I’ve been involved exploring simples and crazy ideas with AjMessages, AjAgents, and Distributed Software Services during last months. I hope to write more about these subjects:

- Another example of distributed agents using DSS/VPL

- AjMessages ported to DSS (I had an implementation last year, but I never published it, I published the WCF only version)

- One-way messages implemented as primitives in AjTalk (Smalltalk-like interpreter)

- Deferred/Concurrent/LazyEvaluation/SomethingAlike implemented in AjBasic (using CCR?)

- Blog about a better-finished app running in a grid of DSS hosts (my team was working hard on this, last months).

Incidentally, you can read more about use cases applied in the real world using CCR/DDS, in Arvindra Sehmi’s post:

CCR/DSS Use Cases in the Enterprise

So many ideas…. only one life…. Should I begin to parallized myself? ajlopez-in-a-grid….

Angel “Java” Lopez
http://www.ajlopez.com/en

Recently, I was involved in the development of an application that runs on a grid of diskless machines. I was an exciting project, and now, after the rush of shipping it, I want to write down some ideas to explore.

In this post, I use the term “agent”, that is a bit fuzzy. I won’t define precisely the concept, I want to use as a base term to refine in a future. By now, let explore some basic (naive?) ideas to better understanding of the problems related to agents and grid applications. Yes, some ideas presented here could be viewed as naive, but I feel that this is a needed exercise, to grasp the key concepts and problems to be solved in these kind of applications. At the end of this article, I’ll present some implementation suggestions.

I described some applications to run in a grid in my previous post:

Grid Computing Programming

Agent concepts

In this post, agent is a piece of software, with behaviour and state. It runs inside an agent host, a host application that provides the base services so an agent can “live” and run. An agent will be represented as in this drawing:

Agent communication patterns

There is a lot of literature about agents communication, from simple techniques to elaborated ones as contracts, bids, and more. We can have agents with beliefs, desires and intention. In this post, an agent is simpler: only an object with state, that can send messages. It can receive stimula from other agents and from the host environment.

The simplest communication pattern is composed of an agent sending a message to another agent:

Some notes:

- The sender agent knows the receiver agent. That is, some form of identity must be implemented. The message is not send to anyone: the sender intention is to send the message to a determined receiver.

- The message carries data, and it must be understood by the receiver, possibly invoking one of its methods.

- The sender doen’t wait for a response. It’s not interested in a return message or data.

- The agents could reside in different physical machines in the grid, in a transparent way.

During its life an agent can send many messages to differents target agents, each one is a known agent for the sender:

Sometimes, the sender agent will receive a message from the receiver agents, notifying some work done, or informing some processed data. Depending on the application, the response message could carry information to identify the original message:

In this case, the sender must be prepared to receive the response in an asynchronous way. This could be an interesting problem to solve: an agent can send many messages, and it would be better if it can still run without receiving all the responses in time. For example, in an board game application, one agent could delegate some tree branch exploring to other agents, and, after some time, it should be able to take a decision, without all the information.

Clouds in the grid sky

Now, another case: an agent could be interested in send a message, without forward it to a determined receptor. Instead, it could be send to a cloud of agents, so anyone interested in the message could have the opportunity of processing it.

(Ok, I’m a bit odd creative in my drawings…

This feature could be implemented using these strategies:

- An agent could send a message to a blackboard system.

- An agent could send a message to host application, indicating a topic (like in a message queue), so any subscribed agent will receive the message. An variant: only some subscriptors receive the message, depending on application settings.

- An agent could send a message targeting some service provider definition. A service provider is an agent that declares, at the beginning of its life, its capabilities and service that is capable to provide. 

As example, we could take a chess/board game application (I prefer the game of go). An agent in such application can send a message reclaiming to solve some attack position. In a blackboard system, it will publish the request in the blackboard. In a topic system, it deposits the message on topic “attackers”. In a service provider strategy, it sends the message to one or many AttackService provider.

As in the previous pattern, an agent can receive an asynchronous response, now from “the cloud”:

 

Agent Duplication

An agent has behaviour and state. If the agent must fork its jobs, it could take the way of duplicating itself:

It’s a bit strange action, but it could be useful, depending on application at hand.

Agents and the Grid

Each agent can be hosted in a grid node. The message sending mechanism must be able to send a remote message to another node. The host application keeps a list of agents by identity, and knows which one is local or remote. An acid test: a grid application with agents must be able to run all in one machine, or in grid, without changing the code or algorithm.

 

As in another grid implementations (discussed in my previous post), a central server is in charge of distributing the tasks between the grid nodes. That server could receive tasks from outside application or even, from other grid. “Grid as a Service” is a new buzzword to apply.

Moving agent

An agent can initiate its activities in one node. But at point, it can decide to continue the work in another node (the host application in that node could take that decision, too). Then, its state should be send from one node to another.

Note that the agent behaviour (perhaps implemented in compiled code) doesn’t travel.

Injecting behaviour

The behaviour of each kind of agent could be expressed in compiled code (.jar files in Java, assemblies in .NET). Other alternatives are possible: the behaviour could be specified in a specific scripting agent language.

If the behaviour is expressed in compiled form, one or more servers have the responsability of storing and distributing that components:

Implementation ideas

Most of these ideas can be implemented in any appropiate language/technology, like Java and .NET, that supports multi threading, remote invocation, message serialization, etc…

Last months I was working on my pet projects AjMessages and AjAgents, more info at previous posts:

AjMessages: a message processor

Agents using Concurrency and Coordination Runtime (CCR)

(I have a new project, named AjGrid, not published yet). For this post, I guess that AjAgents could be an appropiate implementation. AjMessages now has remote support, but it’s more oriented to a pipeline processing: it’s difficult to implement in such disposition some of the above ideas.

I must add some features to AjAgents (actually, AjAgents is a local, proof of concept implementation):

- Configuration: Load and creation of agents in runtime, from a configuration file, or on the fly, in the middle of a run.

- Remote assembly: So a grid node can be injected with new agents.

- Agent identification: To identify an agent in a unique way (an UID could be a solution).

- Message transport: Windows Communication Foundation is a candidate.

But another approach is to take Decentralized System Services (DSS) from Microsoft Robotics Studio. An agent can be implemented as a DSS Service, running in a DSS Host. The intermachine communication would be implemented using DSSP as protocol.

Stay tuned, to see some code in the coming weeks.

Angel “Java” Lopez
http://www.ajlopez.com/en

Following ideas from Fabriq project, I wrote last week an example of a server application that can receive and process OneWay Windows Communication Foundation (WCF) messages. The server can host applications, and each application is composed by nodes, that have actions. The example code can be download from AjMessagesWCF-0.1.1.zip

The message

The message to exchange is a WCF message (the original Fabriq was written before the WCF era). The main information that AjMessages uses is the Headers.Action property. This string has the name of the application, node, and action to invoke.

The action

It’s a string with format:

<application>/<node>/<action>

An application is a set of nodes. A server can host some or all nodes of an application. But the same node can run in different servers. In this way, the internal process of the server can forward a message to other server: the action is not a physical address, but a logical one. Each server knows the address and nodes in other servers, and then, it can decide to forward a message to another server.

The application

A configuration XML file define the applications to run. An application has:

• Nodes: a collection of actions to process.
• Handlers: objects to process a message. A handler can be a composite of other handlers.
• Actions: named entry points to a handler

An example application definition:

  <Application Name="App1">
    <Node Name="Node1">
      <Handler Name="PrintMessageHandler"
Type="AjMessages.PrintMessageHandler, AjMessages"/>
      <Handler Name="PostHandler"
Type="AjMessages.PostHandler, AjMessages"/>
      <Handler Name="DecrementHandler"
Type="AjMessages.Console01.DecrementHandler, AjMessages.Console01"/>
      <Handler Name="Pipeline1">
        <Handler Name="DecrementHandler"/>
        <Handler Name="PostHandler">
          <Property Name="Action"
Value="App2/Node1/Process"/>
        </Handler>
      </Handler>
      <Action Name="Process"
Handler="Pipeline1"/>
    </Node>
    <Node Name="Node2"/>
  </Application>  



 

The handler

A handler must implements an interface:

    public interface IHandler
    {
        void Process(Context ctx);
    }

The context has a property Message, that points to the WCF message in process, and some other methods to use: a Server property that reference to the current server object, and a Post method, to send a new message. The idea is to put in the context all the needed external functionality to the handler, so this is decoupled of the server and message in process.

This is a departure of the original implementation in Fabriq, where the handler receives a message, not a context.

The host

A host is a logical unit that runs inside a server. A server is informed, via configuration, the hosts that are available in other servers, and which ones are local. An example:

  <Host Name="Host1"
    Address=http://localhost:50000/AjMessages
    Activate="true">
      

<Application Name="AjMessages">
      <Node Name="Administration"/>
    </Application>
    <Application Name="App1">
      <Node Name="Node1"/>
      <Node Name="Node2"/>
    </Application>
  </Host>

Dynamic configuration

One of the key features implemented is a handler that reconfigure in runtime the running server. It receives a new XML configuration, and merges it with the current information. Then, using these handler, an application can send a message to a AjMessages server, and load, stop, update hosts, applications and nodes.

The solution

The class library AjMessages project contains the implementation of the example. The other projects are manual test to run that shows how to use the example.

Inside Configuration folder there are classes defined to load configuration data from an XML structure.

Handlers folder contains some predefined handlers to use, an important one is the ConfigurationHandler that permits to reconfigure dynamically the server while it’s running.

Ports folder has implementations of endpoints to receive and send message from an to other AjMessages servers.

PipelineHandler.cs is the composite handler, that contain other handlers.

LocalHost.cs is the implementation of a host running in the current server. It mantains a list of active nodes.

RemoteHost.cs mantains the info associated to a host running in other servers. Using this information, the local server knows how to forward a message, inspecting the remote hosts that have an active node.

Running the example

There is a console project AjMessages.Console01 that can be used as a manual test.

You can run the example, and enter the commands:

load ConfigurationServer1.xml
load ConfigurationHost1.xml
load ConfigurationHost2.xml
send App1/Node1/Process 20

The first command creates the server, using the configuration file. The second and third command reconfigure the server to install and run two hosts, with its applications and nodes. The last command send an integer message using App1/Node1/Process as action.

After that, you enter:

unload
fork

These launch another server console program. In the first one, enter:

load ConfigurationServer1.xml
load ConfigurationHost1.xml
load ConfigurationRemoteHost2.xml

and in the other server, enter:

load ConfigurationServer2.xml
load ConfigurationHost2.xml
load ConfigurationRemoteHost1.xml

In anyone, enter:

send App1/Node1/Process 20

and you’ll both servers exchanging messages.

In any moment, you can enter:

help

to obtain a list of commands to use.

Next steps

In a future installment of the example, I’ll rewrite the application to use a custom Message, capable of be transported via WCF, or DSS (Decentralized System Services), so the example could be hosted in different technologies. I must refactor some classes, for example, Server, that has many responsabilities.

Comments, suggestions, welcome!

Angel “Java” Lopez
http://www.ajlopez.com/en