My Week

Once or twice a year, I post about my week activities. Here is the plan for this week:

Some points:

Tuesday’s Philosophy course is with Laura Klein, more info at: Más cursos de filosofia

Thursday’s Philosophy course is about what is the human being, or anthropological philosophy, more info at: De nuevo, curso de filosofía

I teach about software development at Club de Programadores.

No courses at Microsoft User Group in Argentina, this week.

Southworks is a customer where I working in a team, revamping Fabriq software.

Go is a great game, more info at http://www.gobase.org. I’m going to play in the Asociación Argentina de Go next saturday.

Angel “Java” Lopez
http://www.ajlopez.com/en

Preserving Code in AjGenesis

One of the issues to resolve when we use a code generation engine, is how to add our own code to a generated source file, without loss it in the next generation phase. Sometimes, we generate code only as a one-off activity, and modify the code without worry about a new generation, but in other situations, we use code generation many times. In this case, it would be great to have the capability of preserving manually added code.

During the past month, I added that functionality to my open source code generation engine AjGenesis. Thanks to the spanish group grupo de generación de código, and all its members that tested the new feature.

You can use it since version 0.5:

AjGenesis 0.5

to download from CodePlex.

An example

In that version, under directory examples\HelloPreserve, there is an example of preserving code.

The sample is generated from command line, entering

make

that invokes a .bat file containing:

..\..\bin\AjGenesis.Console Model.xml Build.ajg

This command loads a model of a  Hello World application, from Model.xml:

<Project Name="HelloWorld">
<Messages>
<Message>Hello</Message>
<Message>World</Message>
<Message>Again</Message>
</Messages>
</Project>

(You can read a detailed description of this kind of simple model in a previos post:

Code Generation with AjGenesis: A Hello World Application

A more advanced example at

Application Generation using AjGenesis

)

The make command invokes a small program written in an interpreter, AjBasic. The file is Build.ajg:

PrintLine "Generating HelloWorld" 

TransformerManager.Transform("ModuleVb.tpl", "HelloWorld.vb", Environment, "PRESERVE") 

There is something new: the third parameter in Transformer.Transform. Transformer is a helper object, that is implanted in the variable environment of AjGenesis, that is accesible from AjBasic programs. Using this object we can invoke the template engine. It has a method Transform with two parameters: the file nam of the template to process, and the name of the file to generate. In this new variant, there is a third parameter, that is used as a mark to delimitate a code preserving block inside the template.

This is the example template, ModuleVb.tpl:

'
' Project ${Project.Name}
' Automatically generated by AjGenesis
' http://www.ajlopez.com/ajgenesis
'

' PRESERVE comment start

' PRESERVE comment end

Module Module1 

    Sub Main()
        System.Console.WriteLine( _
<#
    n = 0
    for each msg in Project.Messages
        if n then
            Print "& "
        end if
        n = n + 1
#>
        "${msg}" _
<#
    end for
#>
        )
    End Sub 

' PRESERVE code start

' PRESERVE code end

End Module 

All in this code that contains the mark PRESERVE (that is, the third parameter passed to Transform method), until other line containing the same mark, it will be preserve during code generation.

NOTE: the algorithm matches the lines using the full content of the first line in the preserving block. That is, if the template contains a line:

‘ PRESERVE comment start

this is the line that is searched inside the original code, in order to preserve the block. The blocks to preserver are identified by the FULL CONTENT of the line that initiates each block.

Later, if you modified the generated code, adding text to that blocks, that text is preserved:

'
' Project HelloWorld
' Automatically generated by AjGenesis
' http://www.ajlopez.com/ajgenesis
'

' PRESERVE comment start

' This is my comment two

' PRESERVE comment end

Module Module1 

    Sub Main()
        System.Console.WriteLine( _
        "Hello" _
&         "World" _
&         "Again" _
        )
    End Sub 

' PRESERVE code start

    Sub MyMethod()
        Console.WriteLine("My own inmortal method")
    End Sub 

' PRESERVE code end

End Module 

Some points

Let’s review some points in this process. Why we must indicate a parameter as mark? Why not simply use a predefined mark? First, to maintain compatibility with previous templates. But, more important: you decide what is the mark to use to identify code to preserve. You can use different marks in each project or type of generated text file. You are in charge.

If you want to explore how this feature is implemented, you can examine the file Utilities\FileMerger.vb in project AjGenesis.GenericTransformer. In short, that code apply the transform to the template, generates a temporary file, and merge it with the previous file (I use a temp file, so, in case of any errors during the transform, you can read the generated code so far). Then, it merge the new file with the old one. The preserving code are determined by the supplied mark and the full content of its first line.

I hope you find this feature useful. I’m sure that we can apply our knowledge about application building in a smart way: delegating repetitive and menial tasks to software.

Angel “Java” Lopez
http://www.ajlopez.com/en

Application Generation using AjGenesis

These days, I wrote new examples for my open source project AjGenesis

http://www.ajlopez.com/ajgenesis
http://www.codeplex.com/ajgenesis

Using AjGenesis, we can produce, starting from our own model, applications for ASP.NET 1.x/2.x, JSP, using SQL Server in .NET, with ADO.NET or NHibernate, or MySql with Java and Hibernate. The sample generated applications use a layered architecture (but remember: these are sample applications, you can write and use your own models, tasks, and templates). Other option: it can generate layers following ideas from Eric Evans’ Domain-Driven Design.

I’m still writing those examples, but now, you can download the current version from

AjGenesisExamples3New.zip

The examples require the use of AjGenesis last release:

AjGenesis Version 0.4.3

but you can try the current version under development:

AjGenesis Version 0.5

In this post, I want to comment how AjGenesis works and some of its inner structure, using one of those examples. Although I will concentrate in producing VB.NET code, you’ll find that the example is able to produce C Sharp and Java/JSP solutions, as well.

First, let’s review first some points about the project. The big picture:

AjGenesis Version 0.4.3 is written using VB.NET V1.x, but version 0.5 is the current version under development, and it was rebuilt using .NET 2.0. The project is open source, and has a BSD-like license, that allows to use it in any project what you want: you only need to comply with the license (in short, put some reference to the original project). It is possible to be used like a library, invoked from your own project, or it is possible to be invoked from command line, or even it can be used from using NAnt (this tool gives you a better organization of the code generation tasks).

There are several samples in the page of the project (inside the project zip source file and in additional files). They generate code from a model, invoking tasks and processing templates. The samples generate code PHP, Java, JSP, VB.NET, C#, ASP.NET, and even scripts of procedure and data base stored. I want to emphasize two points:

- The model to use is totally definible by you. It’s a free model, it’s not a fixed one. You can model what you want to model.

- The tasks and templates to apply are totally programmable and controlable. You are in charge. 

That’s make a difference from other generators. We can create our own model, and its own templates and tasks, to generate any text artifacts. Other systems start from the data base, and they only generate a group of predefined text files (as POJOs, plain old java objects, or DAOs, Data Objects Access). But with AjGenesis you can generate any text file you need.

In order to better understandind the free model concept, read a previous post:

Generating Code Hello World with AjGenesis

In that post, the initial steps are described, using a free model, totally oriented to the domain to represent: a typical Hello World application, implemented in different technologies.

Creating an application

Let’s do something more complete in this post. Suppose we need to create a simple solution simple, with two tables, in a MS SQL Server databse, source code in VB.NET 2,0, web interface, layer of services, layer of data, business components and business entities a la Microsoft. We want to generate the solution, the projects, scripts for database creation, and stored procedures. This example is included in the examples AjGenesisExamples3New.zip. First step: to write the model.

The project

In a directory of projects of the examples that accompany this article, there is a Projects/AjFirstExample directory.

In that directory it is the Project.xml file that contains the model.

<Project>
    <Name>AjFirstExample</Name>
    <Description>First Example
    using AjGenesis</Description>
    <Prefix>AjFE</Prefix>
    <Domain>com.ajlopez</Domain>
    <CompanyName>ajlopez</CompanyName>
    <Model>
        <Entities>
            <Entity
            Source="Entities/Customer.xml"/>
            <Entity
            Source="Entities/Supplier.xml"/>
        </Entities>
    </Model>
</Project>

Remember: the model is free. Here we define the templates that we are going to use. The model contains two simple entities: customers and suppliers.

The entities

The XML file is not terribly long: AjGenesis allows that any node of the model is specified apart in a file. This is a criterion that I have used to define how the model is written: the resulting XML does not have to hurt at sight, must be understandable and abarcable in a reading.

In the Project.xml, that feature is used in the case of the entities, with the Source attribute. Let us examine an entity, written in Entities/Customer.xml:

<Entity>
    <Name>Customer</Name>
    <Description>Customer Entity</Description>
    <SetName>Customers</SetName>
    <Descriptor>Customer</Descriptor>
    <SetDescriptor>Customers</SetDescriptor>
    <SqlTable>customers</SqlTable> 

    <Properties> 

        <Property>
            <Name>Id</Name>
            <Type>Id</Type>
        </Property> 

        <Property>
            <Name>Name</Name>
            <Type>Text</Type>
            <SqlType>varchar(200)</SqlType>
        </Property> 

        <Property>
            <Name>Address</Name>
            <Type>Text</Type>
            <SqlType>text</SqlType>
        </Property> 

        <Property>
            <Name>Notes</Name>
            <Type>Text</Type>
            <SqlType>text</SqlType>
        </Property> 

    </Properties>
</Entity> 

There are attributes of the entities, like its name and description, in plural and singular. This data serve to name them in the resulting pages, or within the code. The properties are the fields to maintain in each entity..

Aside from the entities, in another directory, Technologies, specifies the dependent model of the technology, like VbNet2:

The templates

The templates to use are at the Templates/VbNet2 directory:

They are the templates for generation of code VB.Net 2.0. Also we will find groups for C# 1/2, Vb.NET 1,2, Java (although I’m thinking to drop templates for .NET 1.x: I don’t see any reason to maintain these). There are templates to use Nhibernate, Hibernate, JSP, MySql, and concepts of Domain-Driven Design, too. Let’s review a template as an example, the one that generates the organization in Visual BASIC, EntityVb.tpl:

<# 

message "Generating Entity ${Entity.Name}" 

include    "Templates/VbNet2/VbFunctions.tpl" 

include    "Templates/VbNet2/Prologue.tpl"
#> 

'
'    Project ${Project.Name}
'        ${Project.Description}
'    Entity    ${Entity.Name}
'        ${Entity.Description}
'    
'

Public Class ${Entity.Name} 

'    Private Fields

<#
for each Property in Entity.Properties
    message    "Procesando Campo ${Property.Name}"
#>
    Private m${Property.Name} as ${VbType(Property)}
<#
end for
#> 

'    Default Constructor

    Public Sub New()
    End Sub 

'    Public Properties

<#
for each Property in Entity.Properties
    message    "Procesando Propiedad ${Property.Name}"
#>
    Public Property ${Property.Name}() as ${VbType(Property)}
        Get
            Return m${Property.Name}
        End Get
        Set(ByVal Value As ${VbType(Property)})
            m${Property.Name} = Value
        End Set
    End Property
<#
end for
#> 

End Class 

Like before, control structures are used. XML is the serialized format of the model. During the code generation process, the model is loaded in memory, ready to be accesible via dynamic variables.

The steps

We have more files to generate: from the pages ASPX, and their associated code, the projects of facade on watch, organizations, access to data, the file of solution, and more. In order to automate this generation, the example has several files of tasks, in the Tasks directory, where the steps are described to execute. There are two great tasks: the steps to execute independently of the chosen technology, like completing the model, reviewing it, and the employees of the technology, like generating such file JSP or ASPX, depending on if we want Java or .NET.

The task of completing the model is in charge of Tasks\BuildProject.ajg, that begins with:

'
' Build Project
'    Complete the Project Data
'    Project must be loaded in global variable Project
'

PrintLine "Completing Project ${Project.Name}" 

include "Templates/EntityFunctions.tpl"
include "Templates/Utilities.tpl" 

if not Project.Title then
    Project.Title = Project.Name
end if 

if not Project.Version then
    Project.Version = "1.0.*"
end if 

if not Project.SystemName then
    Project.SystemName = Project.Name
end if 

The template includes some auxiliary functions, and then, it begins to complete the model that resides in the Project variable. For example: if the project lacks Project.Title the program set Project.Name as Project.Title. The program continues:

for each Entity in Project.Model.Entities
    PrintLine "Entity " + Entity.Name 

    for each Property in Entity.Properties
        PrintLine "Property " & Property.Name
        if Property.Type="Id" and not Property.SqlType then
            Property.SqlType="int"
        end if
        if Property.SqlType and not Property.SqlColumn then
            Property.SqlColumn = Property.Name
        end if
        if Property.Type="Id" and not Entity.IdProperty then
            Entity.IdProperty = Property
        end if
        if Property.Reference then
...

After that process, the technology tasks are executed. The example use Tasks\BuildVbNet2.ajg, here a fragment:

<# 

include "Templates/Utilities.tpl"
include "Templates/VbNet2/UtilitiesVb.tpl" 

message "Creating Directories..." 

FileManager.CreateDirectory(Project.BuildDir)
FileManager.CreateDirectory("${Project.BuildDir}/Sql")
FileManager.CreateDirectory("${Project.BuildDir}/Src/${Project.Name}.Entities")
FileManager.CreateDirectory("${Project.BuildDir}/Src/${Project.Name}.Entities/My Project")
FileManager.CreateDirectory("${Project.BuildDir}/Src/${Project.Name}.Data")
FileManager.CreateDirectory("${Project.BuildDir}/Src/${Project.Name}.Data/My Project")
FileManager.CreateDirectory("${Project.BuildDir}/Src/${Project.Name}.Services")
FileManager.CreateDirectory("${Project.BuildDir}/Src/${Project.Name}.Services/My Project")
FileManager.CreateDirectory("${Project.BuildDir}/Src/${Project.Name}.Business")
FileManager.CreateDirectory("${Project.BuildDir}/Src/${Project.Name}.Business/My Project")
FileManager.CreateDirectory("${Project.BuildDir}/Src/${Project.Name}.WebClient")
FileManager.CreateDirectory("${Project.BuildDir}/Src/${Project.Name}.WebClient/App_Themes")
FileManager.CreateDirectory("${Project.BuildDir}/Src/${Project.Name}.WebClient/App_Themes/Default")
FileManager.CreateDirectory("${Project.BuildDir}/Src/${Project.Name}.WebClient/Admin")
FileManager.CreateDirectory("${Project.BuildDir}/Src/${Project.Name}.WebClient/Controls")
FileManager.CreateDirectory("${Project.BuildDir}/Src/${Project.Name}.WebClient/MasterPages")
FileManager.CreateDirectory("${Project.BuildDir}/Src/${Project.Name}.WebServices")
FileManager.CreateDirectory("${Project.BuildDir}/Src/${Project.Name}.RemoteServices") 

message "Defining Solution and Projects..." 

...

In this fragment, the directories necessary are created to lodge the solution. The name of the directory is extracted of the model from Project.BuildDir. Note that ${ } in a string is used to expand the inside expression into the string value.

Technology Model

Under the directory Project\AjFirstExample\Technologies they are some models that describes the technology parameters to use:

 

Let’s examine VbNet2.xml:

<Technology>
    <Programming>
        <Dialect>VbNet2</Dialect>
    </Programming>
    <Database>
        <Dialect>MsSql</Dialect>
        <Name>AjFirstExample</Name>
        <Username>sa</Username>
        <Prefix>ajfe_</Prefix>
        <Host>(local)</Host>
    </Database>
</Technology>

These data is used during the template generation phase. It indicates the language, and the database to use.

Generating the solution

We could send the tasks from the command line, but we have a .build file for Nant, one for each technology to generate. We execute the tasks build, buildsql, and deploysql of AjFirstExampleVbNet2.build with the command line:

nant -buildfile:AjFirstExampleVbNet2.build build buildsql

(You must adjust the line

<property name=”ajgenesis.dir” value=”…./AjGenesis-0.5″/>

to reflect your AjGenesis installation directory)

Note the following lines in the build file:

    <target name="loadtasks" description="loads AjGenesis tasks">
        <loadtasks assembly="${nanttasks.dir}/AjGenesis.NAnt.dll" />
    </target> 

    <target name="init" depends="loadtasks" description="init the AjGenesis model, create build directory">
        <mkdir dir="${build.dir}"/>
        <loadmodel model="${project.dir}/Project.xml"/>
        <loadmodel model="${project.dir}/Technologies/${technology}.xml"/>
        <executetask task="${tasks.dir}/BuildProject.ajg"/>
        <executetask task="${tasks.dir}/BuildTechnology.ajg"/>
    </target> 

These tasks load an AjGenesis NAnt tasks, load the models, and execute the tasks.

In the Build/AjFirstExample/VbNet2/Sql directory they are left scripts of creation of the base and stored procedures. And in the directory Src brother, surprise! We’ll have the generated solution:

There were generated several projects, into a solution. We can load the solution into the Visual Studio 2005:

Using another NAnt file, AjFirstExampleCSharp2.build, we can generate the same solution in CSharp:

You will find other projects and examples of .build files, that generate solutions using NHibernate, Hibernate use, JSP, and concepts of Domain-Driven Desing a la Eric Evans.

Reflections

Sure, everything cannot be generated automatically. It is important to remember always that fact. But in the day to day, we recognize that we have amount of repetitive text, tasks that we can well delegate to software.

A key point: the model in AjGenesis is free. The presented examples are only examples: we can general the model that we want, and to write the templates we need. It is important to write the templates so that the generated code is similar to what we had manually generated. If we don’t feel comfortable with the generated code, if it does not have our style, our experience, we’ll end up generating something that we do not understand.

Another thinking: the model must be independent of the technology. In the final example, we have seen how, from the same model, we can generate solutions for VB.NET, CSharp and other technologies.

Software can help us to generate software. More: it MUST help us. Our experience counts: what we learned to make applications, we can overturn it in this species of expert system, code generation tools. In the future, I hope to be able to incorporate to the project, in the templates and tasks, more decision making: as well as we gain experience in writing of applications, we can incorporate our accumulated knowledge on patterns, architecture and styles of programming.

And as it is an open source project, AjGenesis allows us extend it, to our taste and necessity.

Suggestions, use case stories, comments are welcome. You can write as comments to this post, or write directly to me. Thanks to all that tested and adopted this project, and helped me to write it.

Angel “Java” Lopez
http://www.ajlopez.com/en