Archive for the 'Software Development' Category

AjTalk: a Smalltalk-like interpreter

Weeks ago, I was working on my open source project AjTalk, an Smalltalk-like interpreter, written in .NET (using C#), and now, I want to present the status of that work. For years, I was interested in Smalltalk development, altought only in my free time, never as a professional developer. It’s not the first time I wrote such kind of interpreter (my first one was written in the eighties, and it was very simple: no garbage collector, only text commands), but this time I feel it could become a very complete implementation.

Current version is minimal, but it is taken form. The idea is to have dynamic objects, like in Smalltalk, and, at some point, add prototypes a la Self. The objects and the interpreter would access full .NET framework and other libraries, as I did in my AjBasic interpreter (included as part of AjGenesis code generation project).

The very initial version is now published at Google Code:

http://code.google.com/p/ajtalk/

The solution

I’m using Professional VS 2005. There are four projects in the solution.

AjTalk is the main project, a class library containing the core of the system.

AjTalk.Test01 and AjTalk.Test02 are console applications, only for manual tests.

AjTalk.Tests contains the unit tests, written using NUnit framework 2.2.8.

Most of the core system consists of interfaces, defining the base behaviors, and classes, implementing such interfaces.

The object

The base is to have an interface to represent any object:

using System; namespace AjTalk { public interface IObject { IClass Class { get; } object this[int n] { get; set;} object SendMessage(string msgname, object [] args); } }

I could make a message an object (of type Message), but for now, the message is only a name, and an array of arguments.

The index this[int n] access the intance variables. Each value in AjTalk can point to any .NET object, not only the ones that implements IObject. In this way, I can manage int, long, String, DataSet, from other IObject objects.

The class

I’m implementing a simple IClass interface, without distinguing between class, behaviours, and other classes, as in the classic Smalltalk. I’ll separate these classes in a future version. For now, I’m managing only a interface:

using System; using System.Collections; using System.Collections.Generic; namespace AjTalk { public interface IClass : IObject { IClass SuperClass { get; } string Name { get; } void DefineClassMethod(IMethod method); void DefineInstanceMethod(IMethod method); void DefineClassVariable(string varname); void DefineInstanceVariable(string varname); IObject NewObject(); IMethod GetClassMethod(string mthname); IMethod GetInstanceMethod(string mthname); int GetClassVariableOffset(string varname); int GetInstanceVariableOffset(string varname); } }

There is a dictionary of class and instance methods, and lists of class and instance variable names. It doesn’t have support for indexed variables, yet. IClass interface is now implemented in a BaseClass class.

The method

There is an interface implemented in Method class:

using System; namespace AjTalk { public interface IMethod { string Name { get; } IClass Class { get; } object Execute(IObject receiver, object [] args); } }

The concrete Method class, has an Execute method:

public object Execute(IObject receiver, object[] args) { return (new ExecutionBlock(receiver,receiver,this,args)).Execute(); }

Execution blocks have local variables and arguments. The Execute of an execution block takes the instructions (bytecodes) from “compiled” methods, and then it executes them. Here, I took a departure from Smalltalk ideas: the execution block is not an AjTalk object. In this way, I could run this interpreter without implementing a lot of base classes. I have to research the advantages and problems that this decision could have in the overall design and implementation.

The bytecodes

I must think about using a tree of objects (as in Interpreter pattern) instead of bytecodes. But in this version, bytecodes are used. These are the basic instruction that my “virtual machine” understands and executes step by step.

There is a bytecode list (an enumeration)

namespace AjTalk { public enum ByteCode : byte { Nop = 0, GetVariable = 1, SetVariable = 2, GetArgument = 3, SetArgument = 4, GetConstant = 5, GetLocal = 6, SetLocal = 7, GetClassVariable = 8, SetClassVariable = 9, GetSelf = 20, GetClass = 21, GetSuperClass = 22, NewObject = 23, Pop = 24, ReturnSub = 25, ReturnPop = 26, Add = 40, Substract = 41, Multiply = 42, Divide = 43, Send = 50 } }

The bytecodes contained in a method, are interpreted and executed by the execution block. An excerpt of that code:

 

while (ip < method.ByteCodes.Length) { ByteCode bc = (ByteCode) method.ByteCodes[ip]; Byte arg; switch (bc) { case ByteCode.ReturnSub: return null; case ByteCode.ReturnPop: return Top; case ByteCode.GetConstant: ip++; arg = method.ByteCodes[ip]; Push(method.GetConstant(arg)); break; case ByteCode.GetArgument: ip++; arg = method.ByteCodes[ip]; Push(arguments[arg]); break; ....

Test everywhere

The initial code was written in VB.NET. Last year I began to rewrote the original source code to C#. Then, this year I switched to “TDD mind”, so, late but sure, I added NUnit tests:

Bootstraping

I plan to use a text file, with an ad-hoc format, to inject the definitions of the initial classes and objects. Now, in the AjTest.Test02, there is an example of such format in Definitions.txt file:

class Point
variables x y

method
x ^x.

method
y ^y.

class Rectangle
variables point1 point2
class Square Rectangle

Next steps

There is a lot of work to do:

- Complete the hierarchy of base classes (Behaviour, Class, ….)

- More byte codes

- Support of local variables in methods

- Standard file text format

- Access to native .NET objects

- Use from .NET applications

- Define the classes and methods for a minimal implementation

- Serialization/deserialization of memory image

- Support for adding variables to a class with created instances (this is a tough problem)

- Support for become:

- And much more….

But I’m confident on the shape the project is taken. I’m applying “baby steps”, to improve the base code and functionality.

Angel “Java” Lopez
http://www.ajlopez.com/en

The early history of Smalltalk

Today I wanted to comment a text, in my opinion, a “must be read” by all those of us dedicated to software development. It is a writing of Alan Kay, about the early history of Smalltalk:

Smallhistory.pdf

It is an excellent text to read, and that has influences simply beyond Smalltalk. It is a history of how some ideas were arising, within the American community of software development. How the programming with objects was arising, and how the form to interact with the present computers was taking form.

Read, for example, how Kay detects  some germinal ideas (data along with behaviour) in developments of the Air Force, that today we would see very remote of the OOP. Read on his contact with Lisp, and Seymour Papert. Read on the internal problems of Xerox, the competition with DEC, how some ideas were almost generated by chance. Read on Simula, Euler, IPL predecessor of Lisp. Read on the Sketchpad of Sutherland (who I found by first time in some historical revision of the Scientific American). It is a delicious and enlighten reading, at least for me.

Kay has been having an idea for years, that I share: the idea that the machine, and computing in general, must serve to us to expand our human capacities. Excellent idea. It is what of some form also today we are reaching using Internet. Today, branches of the knowledge and human actions, have been leveraged by software, the hardware and everything what it has happened in our profession in the last decades.

Years ago, people guessed that the space trips were going to revolutionize human history. That has still not happened. But of some form subproduct of the cold war and the space race, the development of the computing science (we remember its modern beginnings in the second war, and the appearance of the cybernetics impelled by military subjects) and of Internet, is what it has caused a change, that is reaching to great part of the humanity.

To read the history of Kay is indispensable to be understanding what it has happened. Somebody that has said ” the best form to dominate the future, is inventing it” (approximated phrase, reads the text, to see where it arose exactly).

Angel ” Java” Lopez
http://www.ajlopez.com/en

Messages everywhere

Recently, I wrote a post about Message Passing Interface:

Message Passing Interface, CCR, DSS, and Pure MPI.NET

I used to pass message between agents in my example of a web crawler:

Distributed Agents using DSS/VPL

The passing of messages between components, agents, objects, is a feature that deserves more study. I guess we could write new kinds of applications, using one-way message passing, so we could abandon the call-a-method current way of doing things. Let’s explore first, some message passing applications (not only one-way style).

In the Wikipedia entry about Message Passing:

http://en.wikipedia.org/wiki/Message_passing

we can read Alan Kay opinion:

Alan Kay has suggested that he placed too much emphasis on objects themselves and not enough on the messages being sent between them. Message passing enables extreme late binding in systems.

If you develop systems with Smalltalk, Self or alikes, you’ll notice that the message is a first citizen, in many implementations, a full object, not only a way of call a method.

There is another place for use message passing. For 25 years, QNX operating systems uses the message passing paradigm to run a real time kernel.

I found this interview at Dr. Dobb’s to Sebastien Marineau-Mes, VP of engeneering at QNX:

Real Time OS in Today s World

Sebastien talks about the use of QNX in current market, and the challenge that multi-core machine could create on legacy code.

Remember: all the QNX kernel is based on message passing, although its messages are not one-way, and the passing is optimized to not incurr in loss of performance (more details, see QNX at Wikipedia). I see that many of these challenges and opportunities could be translated to the use, not only to multi-core, but to “multi-machines” in a grid. There many forces that are conspiring to bring these topics to current arena:

- We have a better understanding of agents, message passing and other ideas

- Normal hardware is cheap

- Each year, there are more application with higher level needs of scalabity (the user base will be the whole Internet in the near future, for non-trivial app)

- Many application must interact with other application in the same enterprise or in the net, and messages are an effective way to do that.

- In order to face that challenges, we must begin to abandon n-tier-only apps, to a cloud, grid or “something alike” schema.

I could imagine languages and technologies, based on message passing features. That is one of the reasons I’ve been involved exploring simples and crazy ideas with AjMessages, AjAgents, and Distributed Software Services during last months. I hope to write more about these subjects:

- Another example of distributed agents using DSS/VPL

- AjMessages ported to DSS (I had an implementation last year, but I never published it, I published the WCF only version)

- One-way messages implemented as primitives in AjTalk (Smalltalk-like interpreter)

- Deferred/Concurrent/LazyEvaluation/SomethingAlike implemented in AjBasic (using CCR?)

- Blog about a better-finished app running in a grid of DSS hosts (my team was working hard on this, last months).

Incidentally, you can read more about use cases applied in the real world using CCR/DDS, in Arvindra Sehmi’s post:

CCR/DSS Use Cases in the Enterprise

So many ideas…. only one life…. Should I begin to parallized myself? ajlopez-in-a-grid…. ;-)

Angel “Java” Lopez
http://www.ajlopez.com/en

The new Microsoft baby: Live Mesh

These week, the Redmond giant announced its new creation, Live Mesh (a tech preview):

https://www.mesh.com/Welcome/Welcome.aspx

Thanks to Arvindra Sehmi post:

 http://blogs.msdn.com/asehmi/archive/2008/04/24/introducing-live-mesh.aspx

we now have a list of comments about this new Microsoft baby. Thise new redmonian inception is not easy to describe briefly, but it deserves to take a shot. It is an incursion from Microsoft into Web 2.0 and its own creation, Software plus Service world.

(for bloggers, note the use of links with screenshots, great idea Arvindra!)

This video explains the high level architecture (thanks to Matias Woloski for this link)

http://channel9.msdn.com/Showpost.aspx?postid=399577

In that list of articles, I found a good introduction by the ineffable Mary Jo Foley

Ten things to know about Microsoft’s Live Mesh

Read the point one:

1. The definition. As has become the norm with so many of its Software + Services products and strategies, Microsoft isn’t the best at coming up with a succinct Live Mesh definition. The closest I found (in a Live Mesh reviewer’s guide) was this: “Live Mesh is a ’software-plus-services’ platform and experience from Microsoft that enables PCs and other devices to ‘come alive’ by making them aware of each other through the Internet, enabling individuals and organizations to manage, access, and share their files and applications seamlessly on the Web and across their world of devices.” If I were in charge of defining Live Mesh, I think I’d go with “a Software + Services platform for synchronization and collaboration.”

Here, I detect the reborn of some ideas envisioned by Bill Joy for Jini, a technology that was born too early:

The Jini Technology Vision

Jini technology redefines the concept of a client. Instead of a fixed set of “local” devices, Jini technology supplies the Java client with a federation of remote “plug and play” devices in a dynamic configuration (the Jini Federation) that is personalized for each client. With Jini technology, the network truly becomes the client computer!

All this new move from MS could be the visible result of some of Ray Ozzie’s ideas, expressed years ago in his noteworthy memo:

The Internet Services Disruption

I guess Microsoft is moving more and more to the Internet (remember its Yahoo affaire). That’s a company that embraced the Andy Grove’s motto: only the paranoid survive. There are betting on RIA, Silverlight, web 2.0, rich client… Any number on roulette is covered, even the double zero… ;-)

One note: it’s amusing that, after all this effort, Microsoft still reveals its new tools only to US residents. Google, instead, pushed its innovations as beta or whatever, but for all the world at once. Hey, toc, toc, McFly…. some MS VP reading here? ;-)

Angel “EverMeshed” Lopez
http://www.ajlopez.com/en

Code Generation as a Service with AjGenesis

I’m back! I want to write in this post about the new features I’ve added to my code generation project AjGenesis. The main points are:

- A new interactive way to use it, a web based application

- Examples with that new interface, to generate C#, VB.Net, Java and PHP applications, all from the same abstract model

These features were published last January at release 0.5 in CodePlex

Using AjGenesis

Years ago, I began to work in this project. Those days, the only way to execute and use it were the console, with commands. Later, I adopted NAnt, and NAntGui, as interfaces to launch the system and execute tasks and transformations. The project can be used as a set of .DLLs, referenced from custom application. Carlos Marcelo Santos has published posts (in Spanish) about how to use AjGenesis from NAnt

Cómo generar código con AjGenesis sirviéndonos de NAnt
Cómo generar código con AjGenesis sirviéndonos de NAnt - Parte II

Last year, 2007, Jonathan Cisneros wrote a great Windows interface, named AjGenesis Studio:

AjGenesis Studio: una IDE para AjGenesis
AjGenesis Studio at CodePlex

As an exercise at the end of year 2007, I wrote a new project in the original solution, and I published last January inside the last release 0.5 at CodePlex.

It’s a new web project, that I called, inspired by the work of Jonathan, AjGenesis Web Studio

The main point is, with this project, to show that the functionality from the AjGenesis core can be used and exposed in our applications, this time from a web interface, using ASP.NET. With this approach, we can put a server in our internal network or at Internet, and create, edit, upload and download user defined models, generate source code, and download the results. We could call this, Code Generation as a Service.

Let’s look the project, its functions and some inners.

The project

It’s a web project, named AjGenesis.WebStudio.

To launch it, load the AjGenesis.sln solution (located at src directory inside the version 0.5 from CodePlex) with Visual Studio 2005, and build the complete solution.

(If you have errors in the AjGenesis.WebStudio project, probably you must add a reference to a .DLL that manages the file compressions. Add as reference to the project the file src\Libraries\Ionic.Utils.Zip.dll. This library generates .zip files, and expand them).

In the web solution, I used Master Pages, and a default Theme. It doesn’t manage a database, only files and directories. We’ll see below that the project has a mini wiki, to write help pages that can be defined by the user.

The look and feel is simple, but you can change the theme and master page to obtain a better result.

The system can generate code in the fly, and download it as a .zip file. You can use the classic AjGenesis system, as I explained in past post, but now, we can use it from any browser. You don’t need .NET in your client machine, and you can work from other operating systems.

After the build of the solution, click right mouse button on the file \Default.aspx in project AjGenesis.WebStudio, and choose View in browser…:

This is preferred to launch the project in debug mode: if you debug the project, it will execute the code generation task in very slow way.

Working Directory

The system uses a working directory, with a structure that resembles the examples at AjGenesisExamples3.zip (to download from examples at CodePlex). The default working directory is AjGenesis.WebStudioWorkingDirectory:

Some directories:

Projects: There is a directory here for each project model.
Templates: Containing the template files to process during the code generation phase
Build: The code generated code resides in this directory.

The working directory can be changed using the left menu option Working Directory and then, choosing the option Change Working Directory:

Generating Hello World

As I used to comment in my speeches, the acid test of all code generation system is to help us to model and produce the most simple and universal software system, the reknowed Hello World example. There is an explanation on how to do this with classic AjGenesis in my post

Code Generation with AjGenesis- A Hello World application

In AjGenesis Web Studio, select the left menu option Generate. This is the result page

There are two dropdown list. We’ll see how they are filled. At the first list, there are the defined projects:

Choose HelloWorld. Then, go to technology list, showing:

In this project, only Java and Net20 are the defined technologies. Other projects have more technologies. Choose Java and the press the Generate button. After a while, the page shows a result.

How this works?

First point, the system manage a working directory, the default is AjGenesis.WebStudioWorkDirectory, a syster directory of the web project. There is a Projects project, cointaining more directory, each one is a project for the system.

The directory AjGenesis.WebStudioWorkDirectory\Projects\HelloWorld has a Project.xml file that describes the free model we use in this case:

<Project> <Name>HelloWorld</Name> <Description>HelloWorld Example</Description> <GenerateTask>GenerateHelloWorld</GenerateTask> <Message>Hello World</Message> </Project>

Inside the directory project, there is a Technologies directory. Each XML file that is found there it’s managed as a technology model in the system. Again, as I explained in my posts on AjGenesis , this file has a free model, you can put here anything to process from code generation process. The concepts of project and technology is not mandatory, but AjGenesis Web Studio takes them as special directories, and read them to obtain the list of project and associated technologies.

The technology Java.xml file contains:

<Technology> <Name>Java</Name> </Technology>

and Net20.xml contains:

<Technology> <Name>Net20</Name> </Technology>

Nothing special. The value Name inside Technology drives what steps to execute during code generation phase.

After press the generate button, the system loads the project and technology model in memory. Then, it executes the file GenerateCode.ajg located in directory Tasks. But if the project file contains a  GenerateTask tag, as in this project, that file is executed instead the default one. This is the content of Tasks\GenerateHelloWorld.ajg:

 

PrintLine "Generating HelloWorld..." if not Project.BuildDir then Project.BuildDir = "${WorkingDir}Build/${Project.Name}" end if if not Project.Title then Project.Title = Project.Name end if if not Project.Version then Project.Version = "1.0.*" end if if not Project.Language then Project.Language = "en" end if if not Project.SystemName then Project.SystemName = Project.Name end if PrintLine "Creating Directory ${Project.BuildDir}" FileManager.CreateDirectory("${Project.BuildDir}") Include "Tasks\BuildHelloWorld${Technology.Name}.ajg"

This file is written in an scripting language, a pillar of AjGenesis, affectously named AjBasic. The Project variable is the representation in memory of the tag <Project> loaded from the Project.xml file, described above. In memory, it is as an object, with properties. It’s not an static object: you can add properties in runtime, as Project.Version in the above text. If in an if sentence we asks the value of  Project.Version and this doesn’t exist, it’s not an error, the result is simply Nothing and that value is treated as false for boolean evaluations (like in other languages, notably PHP).

Finally, the last line includes an additional file to process dinamically. The string contants in AjBasic have expression expansion: all between ${ and } is evaluated. Using the value of Technology.Name (obtained from loading in memory the XML technology file), it executes BuildHelloWorldJava.ajg or BuildHelloWorldNet20.ajg. The first one contains the steps to generate a .java file:

<# PrintLine "Generating Solution ${Project.Name}" if not Project.Title then Project.Title = Project.Name end if if not Project.Version then Project.Version = "1.0.*" end if PrintLine "Creating Directories" FileManager.CreateDirectory("${Project.BuildDir}") FileManager.CreateDirectory("${Project.BuildDir}\${Technology.Name}") TransformerManager.Transform("Templates\HelloWorld\ClassJava.tpl", "${Project.BuildDir}\${Technology.Name}\HelloWorld.java", Environment) #>

This code uses inner AjGenesis utilities that create directories and expand a template file. ClassJava.tpl contains:

// // Automatically generated by AjGenesis // http://www.ajlopez.com/ajgenesis // public class HelloWorld { public static void main(String[] args) { System.out.println("${Project.Message}"); } }

All this produces the directory Build\HelloWorld\Java:

If we choose the Net20 technology, more files would be generated:

This is a more complete result. It has a VS 2005 solution file, two project directories, one in VB.NET and the other in C#, with source code and project files:

To see the generated code, its files and directories, go to left menu option Builds:

There is a directory for each project. You can download a complete directory, as a .zip file. We can create a file, download a directory, upload and expand a .zip file, and consult a help page. We can read, edit, and delete each file, in this case readme.txt.

Go to the View at HelloWorld directory:

You can browse directories, upload and download files. You can move a directory, or make a copy, to experiment without loosing the original content.

Generating C# and VB.NET solutions

After this simple example, let’s go for more. For more information about the next examples and their models, read:

Application Generation using AjGenesis

Go again to left menu Generate option. Now, we choose a project, AjSecondExample. Its technology list is more complete:

Choose CSharp2 and generate. This process creates a complete solution inside Builds\AjSecondExample\CSharp2. There is an Sql directory with DDL script to create a MS SQL Server database. Inside the Src directory, you’ll find the solution, composed of a set of projects:

This projects follows a layered archictecture, that you can modify from the templates. They use a data access layer, based on my AjFramework project, but you can change to access Enterprise Library or your own data services.

You must create the database using the generated scripts, and modify the connection string in the web.config to match your environment. The application can be compiled and executed:

The administration page shows:

Back to generate page, execute the VbNet2 technology, obtaining a directory with the same structure, now with Visual Basic.NET code:

All these depend of the specified technology inside the files at Projects\AjSecondExample\Technologies. The content of VbNet2.xml is:

<Technology> <Programming> <Dialect>VbNet2</Dialect> </Programming> <Database> <Dialect>MsSql</Dialect> <Name>AjSecondExample</Name> <Username>sa</Username> <Prefix>ajse_</Prefix> <Host>(local)</Host> </Database> </Technology>

and CSharp2.xml:

<Technology> <Programming> <Dialect>CSharp2</Dialect> </Programming> <Database> <Dialect>MsSql</Dialect> <Name>AjSecondExample</Name> <Username>sa</Username> <Prefix>ajse_</Prefix> <Host>(local)</Host> </Database> </Technology>

These data are used during the code generation process.

Note: inside Tasks\BuildTechnology.ajg file, you’ll find a new variable, injected from the web project:

if not Project.BuildDir then Project.BuildDir = "${WorkingDir}Build/${Project.Name}/${Technology.Programming.Dialect}" end if

This is the WorkingDir variable that points to the working directory we selected in AjGenesis Web Studio.

Genarating Java and JSP applications

Many projects (AjFirstExample, AjSecondExample, AjTest….) can be generated in Java with JavaServer Pages. A build.xml file is generated to be modified and used to compile the generated code using the ant utility (you can use this file from an IDE, like Eclipse or NetBeans).

The target build  makes a .war deploy file. You can browse and modify the build.xml file to install the .war in a Tomcat web container.

The generated code is composed of .jsp files (web pages in Java) and Java code to compile that implements a service layer and a data access layer, exchanging simple JavaBeans between layers. The DDL script, in these examples, is a MySql one.

If you build and deploy the application, and install the database in a MySql server, you can see it working fromn a Tomcat server:

Generating application using Domain-Driven Design ideas and Hibernate, NHibernate

Many example projects (AjFirstExample, AjSecondExample, AjTest…) have defined technologies CSharp2DDDNh, JavaDDDHb, VbNet2DDDNh, and even VbNet2Nh. The DDD ones implement some concepts from Domain-Driven Desigg, as Application, Domain and Infrastructure layer, but the templates could be improved. The *Nh projects generate code for entities using NHibernate mapping. The *Hb projects use Hibernate.

An example of NHibernate generated mapping:

 

<?xml version="1.0" encoding="utf-8" ?> <hibernate-mapping xmlns="urn:nhibernate-mapping-2.0"> <class name="AjFirstExample.Domain.Customer, AjFirstExample.Domain" table="ajfe_customers"> <id name="Id" column="Id" type="Int32" unsaved-value="0"> <generator class="native"/> </id> <property column="Name" type="String" name="Name" length="255"/> <property column="Address" type="String" name="Address" length="255"/> <property column="Notes" type="String" name="Notes" length="255"/> </class> </hibernate-mapping>

These solution can be built as the previous examples. You’ll see better generator templates for NHibernate in the AjOmar example at CodePlex. Post (in Spanish) explaining the process at

Generando código para NHibernate (Parte 1)
Generando código para NHibernate (Parte 2)
Generando código para NHibernate (Parte 3)

Generating PHP applications

If you choose PHP technology, web page with PHP code are generated, using MySql as database. The generated code can be copied to a directory in a web site that supports PHP, Apache, or IIS. AjTest example running in a IIS with PHP4:

Other directories

In the working directory, there are other directories:

SourceCode: Containing some source code that is copied into the solution during code generation.

Libraries: Additional .dlls, to copy and use in the generated applications.

The larger subdirectory  is Templates:

In previous posts I described in detail the creation and use of a template file. This is the content of a template that generate a C# entity:

<# rem Entity Generator rem for C Sharp include "Templates/CSharp2/CSharpFunctions.tpl" message "Processing Entity ${Entity.Name}" include "Templates/CSharp2/Prologue.tpl" #> /* * Project ${Project.Name} * ${Project.Description} * Entity ${Entity.Name} * ${Entity.Description} * */ using System; namespace ${Project.SystemName}.Entities { public class ${Entity.Name} { // Private Fields <# for each Property in Entity.Properties message "Processing Field ${Property.Name}" #> private ${CSharpType(Property)} ${CSharpFieldName(Property)}; <# end for #> // Default Constructor public ${Entity.Name}() { } // Public Properties <# for each Property in Entity.Properties message "Processing Property ${Property.Name}" #> public ${CSharpType(Property)} ${Property.Name} { get { return ${CSharpFieldName(Property)}; } set { ${CSharpFieldName(Property)} = value; } } <# end for #> } }

Help pages

I’m testing some ideas in this web project. There are code to create “wiki” pages, that can be used as help pages in the system.

You can edit this kind of pages:

There is a way to insert links to other wiki pages and to external ones. The page data is represented as a .NET object in memory, and that object is serialized in an XML file in Pages application directory. There is no database to store page: each one is an object, serialized in a .xml file.

Conclusions

I hope this system helps to a better understanding of AjGenesis project features and potential. With a more user friendly interface, the project can be used by more people. We can generate code from command line, or using NAnt utility, or AjGenesis Studio.

I ever remark a point: all these are examples, AjGenesis is not limited to generate only this kind application. You are in charge: you can define the model that matches your project requirements, using any technology you choose. You can codify DDD ideas in a different and better way. You can generate Hibernate/NHibernate mappins using other patterns and mapping idioms. You can use other ORM utilities, and generate code for them. You can generate PHP5 pages that use Prado, instead of plain PHP4. You can generate stored procedures for Oracle, instead of my examples that use MS SQL Server. Or write better ones. You can generate WinForms code, instead of ASP.NET page. You can generate JavaServer Faces pages, views, beans, and abandon JavaServer Pages. You can generate the code or text you want: e.g., unit tests, or documentation pages. I recommend that the generated code resembles the code you could generate manually.

I invite anyone that uses this system, to post his/her work, or leave comments here. You can participate in the Spanish list:

http://groups.google.com/group/codegeneration?hl=es

Next steps

Back to develop, I have some ideas:

- Improve the templates to generate more technologies, as VS 2008 projects, solutions, templates for Struts 1/2, Spring, better mapping for Hibernate, NHibernate, and code generation for the new ASP.NET MVC.

- Write posts about these templates

- Publish the new examples

- Write down a useful documentation (now, it’s minimal: you can found more details in this posts)

Enjoy the project!

Angel “Java” Lopez
http://www.ajlopez.com/en

QCon London 2008

I’ve received an email, from Floyd Marinescu, announcing this “mega” conference. They’ll cover all-about-software-development: agile, DSLs, SOA, SaaS, Cloud computing (I guess grid computing related), .NET, Java, Ruby, Rich Client, Scrum, XP, F#, REST…. It will be the DisneyWorld of all software developers! ;-)

Here’s the announce:

You are receiving this email because you opted-in to receive notifications related to QCon London, from the QCon London website. QCon London is taking place in just 8 weeks, March 12-14 and the conference editorial is nearing final. Since our last update in December, a number of important changes have been made to the conference:

  • Google, Yahoo!, Amazon, Salesforce.com now presenting in the Cloud Computing Track
  • MySpace.com, eBay, Magus confirmed in the Architectures you’ve always wondered about Track
  • XP founder Kent Beck, and Analysis Patterns & Refactoring Author Martin Fowler keynoting
  • Banking Architectures Track adds sessions from Merrill, Betfair, Credit Suisse
  • GoF Patterns Author & Eclipse Architect Erich Gamma presenting
  • and much more…

QCon is an enterprise software development conference for team leads, architects, and project managers covering Architecture & Design, Java, .NET, Ruby, SOA, Agile methodologies and other timely topics such as DSLs, cloud computing, and bank/finance architectures. Last year’s QCon London had almost 500 registrants, and was covered very heavily in blog space see QCon London blogger coverage & key take aways).  

The track themes for QCon London & track hosts are as follows:
Architectures You’ve Always Wondered About -  Hosted by Redmonk’s James Governor
The last 2 QCon’s featured: Amazon, Linked-In, Yahoo!, eBay, Second Life, and Orbitz architecture case studies
Domain Specific Languages in Practice - Hosted by ThoughtWorks’ Neal Ford
Takes DSLs to the next level by practical applications and tools that are useful today.
Effective Design
Translate a set of abstract ideas into working and functional software in an effective way.
Architectures in Agile - Hosted by POSA author Kevlin Henney
How to integrate the trade-offs related to aspects such as performance, security, scalability and modifiability into an agile process
Banking: Complex high volume/low latency architectures  - Hosted by John Davies & Alexis Richardson
The latest innovations as well as time-proven best practices that architects of banking & finance systems need to know.
The Cloud as the New Middleware Platform - Hosted by EAI Patterns Author Gregor Hohpe
Is the internet becoming the computer?
Implementing Scrum and XP- Hosted by Patterns Almanac author Linda Rising
Leading practitioners will present and explain how Scrum and XP are implemented in the most effective way.
Java Emerging Technologies - Hosted by Java Concurrency Author/spec lead Brian Goetz
The previous QCon covered: JRuby, Grails, Server-side OSGi, DSL development, Batch Processing
.NET in the Enterprise - Hosted by Matt Deacon
.NET has brought Microsoft’s platforms into many business-critical applications, back-office, and server-side solutions.
Programming Languages of Tomorrow
Erlang, F#,, Intentional and Scala. How can we best leverage them in our next software project?
SOA, REST and the Web - Hosted by InfoQ’s Lead SOA Editor Stefan Tilkov
REST & SOA, Internet Scale Integration, REST & WS Myths
Browser & Emerging Rich Client Technologies - Hosted by InfoQ lead RIA/Java editor Scott Delap
Silverlight vs. JavaFX/Consumer JRE vs. Adobe Flex/AIR vs. Google’s Ajax RIA stack
The Rise of Ruby
Learn how to best take advantage of what Ruby has to offer
The first two QCon were well received, below are some comments from bloggers who attended our most recent QCon:

  • David Forbes - Exhilirated after gorging on brain candy this week, I have a moment to reflect on what just happened. QCon was the right place to be. I can’t imagine where else I would have rather been. If I had the week to do again, I would probably march right down to the Westin…again.
  • Denis Bregeon - I was very happy with it. Most of the talks tickled my imagination and that is the primary thing I was looking for. Many others gave me details on more technical subjects that I wanted to learn about.
  • Srini Penchikala - I was at the QCon conference in San Francisco last week. It was a great experience to be there. I learned a lot not only from the presentation speakers and panelists but also from the attendees who came from different countries (England, Syria, Australia to name a few) and companies.
  • Alex Olaru - Great conference: excellent speakers, very relevant topics, just enough product pushing without it becoming annoying. All in all a conference I would highly recommend to any architect or project manager.
  • Ola Bini - Last week I attended QCon San Francisco, a conference organized by InfoQ and Trifork (the company behind JAOO). It must admit that I was very positively surprised. I had expected it to be good, but I was blown away by the quality of most presentations.
  • Martin van Vliet - All in all, this was a good conference and more than enough reason to look forward to the next QCon, next year in London.
  • Pete Lacey - A wonderful conference made better by being able to meet many people face to face for the first time…

See also past QCon/JAOO talks available online on InfoQ:

Registration for the 3 day conference is 1180£ until Feb 22nd, and 1220£ after March 11th. The conference will be held at the Queen Elizabeth II Center, like last year.  Join us at QCon London!

Angel “Java” Lopez
http://www.ajlopez.com/en

Lessons learnt from an agile year

Recently, my team shipped a product, working at Southworks, after hard work. There were interesting and intense days, and now, I want to write down some lessons learnt from that project, and from others works, during 2007 (and part of 2006).

Architecture decisions must be taken by only one person. This is a lesson that not all my coworkers share, but for me, it’s clear that too many hands on that decisions, could produce a suboptimal result. The member that decides on architecture must hear the other members opinions, actively asking them about problems to resolve, but the final cut on architecture must be made by only one person. I’m skeptic on emergent design, at least on some kind of projects that involves new challenges.

You must be aware of the status of the whole project. As a team member, you must not forget the full picture. It’s your responsability (and the responsability of each member) to keep an updated vision of the advance (or lack of advance) of each committed final deliverable, not only yours, but the other members too.

Don’t hesitate to rise a red flag. At any moment, if you feel that something is out of track, or if you see a potential problem, you must rise hands, and communicate clearly and firmly that problem to the rest of the team. If they were a problem, don’t forget call Houston.

Pursue visibility of what you are doing, and demand the same from the rest of the team.

Practice your communication skills. Give presentations about the advance of the project, lessons learnt, to your peers. This activity increase the team understanding of the project. If you understand something, you can explain it. This is a test of your grasp of the project ideas.

Write down vision and mission for the project from the very beginning. You can have an internal and external mission and vision, but your team must write down those items, and must have a clear understanding of them.

Minimize risk. You must resolve the riskiest part of the project, or be early aware of where risks could rise, and manage to minimize its dangereous influence. Write down proof of concept apps, or take time to spike on ways of doing things, but don’t hesitate to assign resources on those risk management activities.

Every deliverable must be important to you. If you are a coder, don’t forget documentation. If you are a writer, don’t miss the graphic appearance. The process should produce a whole product. You and your team are in charge of the full final result.

Don’t live with broken window. It’s a concept I learnt from “The pragmatic programmer” book. It means that you must not leave open little holes, incomplete work, hand-wired implementation. You must proud of the final product. Any quick hack must be avoided, as it usually spread its influence, as a rotten apple. More on the concept at:

Don’t live with broken window, a conversation with Andy Hunt and Dave Thomas

Get the right people in the team. A team must be composed of different personalities, with complementary skills. If they all were coders, nothing of sustained value could be produced.

Write down a glossary. This is a task that is frequently forgotten. If you don’t have a glossary, explaining the main concepts involved in the project, your team won’t grasp the key ideas and problems that compose the solution to build. The lack of glossary could lead to miscommunication and confusion, inside the team, and in its relation to customer and stakeholders. 

Draw a conceptual map of the whole project, and discuss it with the team. Use an initial meeting to talk about the full model, vision of the project.

Write down a clear list of the final deliverables to produce. Specify a clear contract with the customer, setting his/her acceptance criteria of those deliverables.

Talk the customer language. This could be related to the glossary item. You must understand and use the customer language. You must frequently communicate with her/him, and take account of his/her necessities and problems to resolve.

I didn’t follow all these recommendation all the time, and then, I suffered the consequences of not adhere to this list.

I can think of a few more lessons learnt, but the above list is good enough for now.

Angel “Java” Lopez
http://www.ajlopez.com/en

Genesis Studio: an IDE for AjGenesis

Thanks to the excellent work of  Jonathan Cisneros, now we have an IDE to use with AjGenesis. Developed with CSharp2, it edits projects, model and template files, even with syntax color!!.

It’s an application, ready to use, without rough edges. It will evolve: Jonathan has some ideas for extensions, to complete its program.

You can download Genesis Studio from the group Code Generation page:

Code Generation Google Groups

or you can try this link. In that group there are many files to download, with new templates, models and examples. Genesis Studio is at:

It comes with source code, in a Visual Studio 2005 solution. It’s compilation is easy, it have all the required libraries:

With this application we can edit a project directory, its entities, model, technologies. Remeber, AjGenesis has a free model that you can define for each project. Genesis Studio can read and edit template ant task files.

Some options:

When you choose a project directory, a project explorer appears, showing folders and content:

XML, templates and task files have sintax color support:

 

My thanks to Jonathan Cisneros, for this useful application.

Code generation is technique that has power uses. In an technology change environment, with complex application, it is important to delegate the menial and repetitive tasks to the software itself.

Angel “Java” Lopez
http://www.ajlopez.com/en

Grid Computing Programming

In previous posts, I described my projects AjMessages and AjAgents, giving some source code to play with:

AjMessages: a message processor

Agents using Concurrency and Coordination Runtime (CCR)

Thanks my experience in the venerable (and old) project AjServer (see Hacia el AjServer (Spanish)), I could write the core of AjMessage in a day, from scratch. It was a funny day, of hard coding. Since then, I spent many hours adapting the code to use a custom message, instead of a WCF message, and writing the capability of use pluggable input and output channels, so it can achieve transport independence, to some degree.

One of the features of AjMessage is to communicate many program instances, that could be executing in different machines. They can send message with dynamic configuration, so we can distribute tasks to those machines at runtime. I must still resolve the assembly remote distribution, some ideas to explore at the end of this post. The same action (action is the minimal logical step to execute) can be attended by different machines. AjAgents project points to use distributed agents, but for now, it’s only a local application. I envision that any agent could be running in any machine, in a transparent way. I think that the use of agents, or arbitrary tasks, could be a more flexible way of distribution, in contrast to message passing as in AjMessages. An agent is capable of lunch many subtasks, assign them to other agents in an asynchronous way; it can send partial results to partner agents, and it can dialog and negotiate with many other agents in a more organized way.

Let explore some ideas for AjMessages project. It could be distributed using a server machine that controls the other instances of the system:

These scenario has similiarities with the concept of grid computing. I want to enumerate some of these scenarios, how we can use the system to use it in a grid of nodes and servers. “Grid Computing”, as many technological “buzzword”, has a wide scope, but let try to define it.

According to the excellent article from IBM people:

New to Grid Computing

Grid Computing can use a pool of server, storage systems, and networs, as a unique big system, in such way, that we can manage all those resources in the execution of a task. For the user, or for the application, the grid appears as a whole system.

In the case of AjMessage, following Fabriq ideas, these behaviour can be obtained because the servers execute one or more applications, distributing each action, in a transparent way.

The grid computing concept allows us to use more processing power, without the need of expensive hardware or sofware, using load balancing and task distribution on common machines. The scalability is reached via scale out: more machine in the grid, better results. Depending on the system that organize the distribution, we can add more nodes, obtain more throughput, without touching the application logic.

I imagine a set of machines, composing a grid, and exposing this set to use by users. I think that Grid as a Service is a term that can be coined to describe that arrangement.

Applications

Back to the main topic: what use cases, scenarios, could we imagine to use a grid?

There is a tentative list:

- Genetic Algorithm Processing: A problem could not have a clear solution. Its complexity could grow in exponential form, and then, it can be intractable using conventional approaches. Using genetic algorithms, the program can test many partial solutions, and using change and selection, it can discover better solutions. This work can be parallelized, being an ideal task to run in a grid. I’m collecting some candidates at:

http://del.icio.us/ajlopez/geneticalgorithms

I’m impressed by the results of http://www.darwinathome.org, although I think most of those results are not emergent, but they are consecuences of the selected fitness function.

- Tree Search: In many artificial intelligence problems is needed to explore branches in a search tree. One case is the analysis of play move in a game. It can be extended to business decisions and planning. A grid can help in the decision calculus of the next move in a computer go program, one of the hardest problems in artificial intelligence game programming.

- Web Crawler: the task that explores a site, gets its page contents, analyzes them, detects links, and continues the exploration to other linked pages, is one that can be distributed in many nodes in a grid. While a node gets a page content, other generates tasks for other nodes, as indexing the retrieved content, and retrieve new pages in process.

- Batch Processing: A network of nodes can process a great amount of information, if this info can be splitted in parts. The job could be to trasform data from a database table, to log analysis, to statistic generation. If the input is divisible, each part can be send to different nodes. An example: a node could process January data, meanwhile other ones process the other months. ETL processing in general is another example.

- Email List Distribution: A typical case. A company that offers email list distribution needs to receive, process and resend an incoming message to a list of recipients. The email could need some personalization process. Then, the incoming email could be derived to one or more nodes in the grid, to further process.

- Message Processing: In the actual SOA world, an application receives tons of XML message. Each one needs control, transformation, and content routing. In a grid system, each message is derived to a node. When more throughput is needed, more nodes are added to the grid.

- Workflow Execution: As in the previous example, this is more a scale out distributed task, rather than a grid specific one. A workflow can be designed, and each step can be assigned to a node or set of nodes. For example, in a SaaS application, the steps to make a new tenant provisioning can be executed in a grid. .

- Map Reduce: It’s a programming model to process big data sets. A Map function is specified to process an input key/value pair, commonly many pairs. There is other Reduce function to apply to all intermediate key/value pairs that share the same key. A function Map can receive a document to process, generates word/document pair, and the Recude function take those pairs with the same word, to make a list of documents that contains that word. For a more detailed explanation, see the Google Labs paper about MapReduce: Simplified Data Processing on Large Clusters.

- Biology and Genetic Software Applications: I’m interested in science in general, and in biology in particular. I guessed that there are applications where a grid can be applied, and I think I’m right (recently, I reviewed the course material of Introducción a la Biología Molecular para Programadores given by Sebastian Bassi and his partners). It’s interesting to found that there are implementations like BLAST that can be ported to a grid. See one such approach in the case studies of Digipede.

- Rendering and Image Processing: Many of the rendering, lightning, making of realistic images can be run in parallel.

- Animation Creation: Even if an image cannot be processed in parallel, sometimes we can lunch different tasks, one for each image, in order to produce an animation. A grid can be used to scale out this heavy processing.

- Media Processing: Video compression, key frame detection, scene change detection, can be partitioned to be processed with a grid.

- Simulations: A wide subject. There are systems where it’s not clear what output would be produced given an input. A set of input data set must be processed. Then, each input data set could be given to a node or nodes in the grid. With more nodes, the simulation can produce more results.

Software and languages

Point of view change: a grid can be exposed using web service. An interface can be defined to send tasks to the grid, tasks that can be written by grid programmers using an special SDK or framework. What kind of software can be send to a grid? Some options:

- Complete Assemblies, invoking some (predeterminated or not) methods.

- Scripting Language Programs, running in a “sandbox” interpreter, in order to control the security and health of the node.

- Agents, consisting in assemblies or code to run in an agent virtual machine.

- Grid Domain Specific Languages, designed to take advantage of the grid computing concept.

Such grid can be offered as a service to other service (even other grids). The concept of Grid as a Service emerges. The rent of its power, service level control, health monitoring, and more, are applications to consider in the future for these scenarios.

Links and resources

I’ll write in more detail about grid computing. For now, you can read the mentioned IBM article:

New to Grid Computing

There is an interesting open source implementation in Java:

GridGain

(The drawing at the beginning of this post was “inspired” in one from GridGain; but in my version, the nodes can communicate each other, using the location independence of each action in AjMessage).

I’ve collected links about grid computing in my del.icio.us account (del.icio.us is addictive):

http://del.icio.us/ajlopez/gridcomputing

For this post, I’ve pay attention to

http://www.gridgain.com
http://www.digipede.net
http://www.gridgistics.net/

Digipede implementation is very interesing. They distributed assemblies. There is a server that receive tasks, distributes them into the grid nodes, where the Digipede agents are running. The system keeps a database with the launched pending, and terminated tasks. It expose a control web interface. A user applicaction can communicate with the Digipede server, using a dedicated web service.

GridGain has a feature: “gridifying” a Java method, using an annotation: interesting idea to explore.

Some crazy ideas

I would need medication, but there is a list of crazy ideas to implement:

- Code Generation in a Grid: To generate code, using my project AjGenesis or anything else, is used to execute a list of steps. Not all of these steps must be executed in order: most of them could be launched in parallel, ideally in a grid. A code generation engine can consists of agents, mini expert systems, specialized on completing the model, making transformation, taking decisions, and more, in order to generate a system. A grid can host all these pieces.

- Computer Go in a Grid: I mentioned above, related to tree search. There is some work, gridifying GNUGo. For me, it’s a super interesing topic. Again, a community of agents, running distributed in a grid, can achieve more results than a common approach. The game of Go is not like chess: no game program could beat a professional human, yet. It merits more creative aproximations to the problem. More about the Computer Go at:

http://del.icio.us/ajlopez/computergo
Computer Go

- “Gridified” Programming Language: I have ideas to extends AjBasic with CCR or something similar, or to implement something more oriented to functional programming, where some operator (list processing, others) could be easily gridifiable. It would be interesting to write such language: its programs could run in a sole machine, but, transparently, could be distributed to multiple nodes on a grid. AjG# is coming… ;-)

Conclusion

As you see, grid computing is a great topic. I want to thanks here to Gabriel Szlechtman: he suggested many of the enumerated scenarios.

Any other applications, implementations, to comment?

Angel “Java” Lopez
http://www.ajlopez.com/en

A post a day keeps the doctor away

During last half year, I was writing dozens of post, inspired by a mindset: a post a day keeps the doctor way.

You can see the complete list of posts at my spanish non technical blog:

A post a day keeps the doctor away

I took inspiration from Gianpalo Carraro’s post:

1 day 1 blog: as simple as that

Gianpaolo is one of the Microsoft software architect: he writes a lot about Software as a Service. He describes himself as a dot com refugee: he survived the Internet bubble at the beginning of the century.

Some weeks ago, I met Gianpaolo, here, at Buenos Aires. He was visiting one of my customers, Southworks. He is a kindly man, that answered every question asked by a bunch of software developers (including me), during a private presentation.

His career includes works at Bell Laboratories, Internet companies, and now, the giant of Redmont.

Back to topic. Not all my post were relevant: but I think that writing them trained my communication skills, including this “non-standard” English dialect, known as Anglish, that is, “Angel’s English”…. ;-)

I began to write more in English, so, stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com/en

Next Page »