Category Archives: AjGenesis

New Month’s Resolutions: February 2014

Review of my January’s Resolutions:

– Start to implement Akka-like actor model in C# [complete] see repo
– Start to implement Scala interpreter in JavaScript [complete] see repo
– Work on AjErl [complete] see repo
– Work on Mass (I have many ideas to implement as module pattern and variable scope access) [pending]
– Work on DylanSharp [complete] see repo
– Start an implementation of ML (JavaScript? C#?) [pending]

I also worked on:

– Improving Scala interpreter in C# [complete] see repo
– Adding dot notation to Lisp interpreter in C# [complete] see repo
– Improving Ruby interpreter in JavaScript [complete] see repo
– Improving Clojure-like interpreter in C# [complete] see repo
– First templates, tasks generating Sinatra application in AjGenesis for Node [complete] see repo

My new month’s resolutions:

– Complete distributed messaging in AjErl
– Complete dot notation in AjLisp
– Improve ClojSharp
– Work on ScaScript
– Work on ScalaSharp
– Add variable scope to Mass
– Complete first version Aktores actor model in C#
– More code generation tasks, templates, models in AjGenesis for Node, generating Express, Meteor, Sinatra applications

More fun is coming

Keep tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

New Month’s Resolutions: September 2013

My past month’s resolutions status:

– Update SimpleProlog [complete] see repo
– Update Mass [pending]
– Update AjTalkJs [complete] see repo
– Update AjTalk [complete] see repo
– Update SimpleLisp [pending]
– SimpleLisp web server sample [pending] 
– New Google Hangout using Ruby and TDD [complete] see post 
– New Google Hangout using C# and TDD [complete] see post

Additional items:

– Another Google Hangout using C# and TDD [complete] see post
– Google Hangout using JavaScript and TDD [complete] see post
– Google Hangout about Exploring MongoDB [complete] see post
– Google Hangout about Code generation with AjGenesis for Ruby [complete] see post
– Update AjGenesis for Ruby [complete] see repo
– Update AjGenesis for Node.js [complete] see repo
– New AjGenesisNode module for Entities [complete] see repo
– Start new AjGenesisNode module for Express [complete] see repo
– New SimpleUnit node module for testing [complete] see repo
– Start ScalaSharp, Scala-like interpreter in C# [complete] see repo
– Improve left recursion in GrammGen [complete] see repo
– Update SimpleMongo [complete] see repo
– Give a talk about code generation with AjGenesis for Ruby [complete] see talks

and many node modules refactoring, using SimpleUnit

This month’s resolutions:

– Update SimpleForth
– Give a talk about SimpleMongo, SharpMongo
– Update SimpleMongo
– Update SharpMongo
– Update PageJs
– Update ScalaSharp
– Update JPyScript
– Update AjTalkJs
– Update Mass scopes
– Start Web Server example with AjTalkJs
– Update AjTalk
– Update AjLispJs
– Web Server example with AjLispJs
– Prepare a talk about Scala

Keep tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

AjGenesis Code Generation: News and Ideas

Past week, I moved my open source code generation project AjGenesis (.NET version) from Codeplex to GitHub. The Codeplex site has a new compiled version (with source code and examples), AjGenesis 0.6 (Some time ago, GitHub removed the hosting of binaries downloads) (my posts about AjGenesis).

That new version is only the compiled one from Codeplex trunk. It has minor fixes and improvements, and it was used in real projects. Now, it’s time to work on the next version. Some ideas to implement:

– Refactor test, to use Visual Studio tools

– Refactor AjGenesis Web Studio, to ASP.MVC 3, using C#, Razor, and Bootstrap 2.0

– New console main program, renamed to ajgen (instead of AjGenesis.Console, a long name for this times).

– Support for commands in repositories. That is, there will new names to use with console program, like:

ajgen generate <parameters>

ajgen newentity <parameters>

that is

ajgen <verb> <parameters>

The verbs will be defined in repositories, and everyone can write the commands (at first, usual AjGenesis tasks). In this way, I could support code generation in a easy way: instead of specifying explicit task files and models, they could be located in predefined locations. In the previous versions of AjGenesis, I was reluctant to use such kind of search for tasks, but after reviewing other code generation tools integrated in frameworks (like Ruby on Rails, Django, Express) I want to explore this path.

The usual

ajgen <task or model files>

will be supported, too, without changes.

Long term goals:

– JSON support for models

– AjSharp/AjBasic as scripting languages for AjGenesis (now, AjBasic is internally coupled to the tool)

Don’t forget they are new versions of AjGenesis: AjGenesisNode implemented using Node.js and AjGenesisRuby using Ruby.

Keep tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez



					

AjGenesis Code Generation in .NET, Javascript and Ruby, All the Gates are Open

Usually, I get up early in the morning. Today, I got up a bit earlier, with some ideas about how to implement a simple template engine for my personal project, AjGenesis in Ruby. Then, I put my hands at work, and 3 hours later I had a solution, built using TDD (I made frequent commits, so you can see the progress of development in the GitHub Log).

The result at my GitHub account:

https://github.com/ajlopez/AjGenesisRb

Initially, I planned to use one of the template engines already available in Ruby, like Haml. But I decided to write my own implementation. Past week, I wrote AjGenesis in Javascript/NodeJs, see:

https://github.com/ajlopez/AjGenesisNode

In that project, I separated the template engine in another package:

https://github.com/ajlopez/SimpleTpl

In this way, I designed the template format to be similar to the original templates in classic AjGenesis.

I reuse the Javascript template engine in another new project (code kata of the past weekend):

https://github.com/ajlopez/SimpleMvc

a simple MVC implementation over NodeJs/Express.

So, now you have three projects:

Classical AjGenesis code generation, implemented in .NET (see examples folder)
AjGenesis in Javascript/Node, with SimpleTpl template engine, with interpolated Javascript expressions, and embedded Javascript commands (see samples folder)
AjGenesis in Ruby, with a similar internal template engine, with interpolated Ruby expressions, and embedded Ruby commands (see samples folder)

AjGenesis in Javascript was published to npm (node package manager) as beta.

AjGenesis in Ruby is still not a gem: work in progress.

REMEMBER: all these projects GENERATES the text artifacts YOU WANT to generate. There are not limited to a technology, platform or programming languages.

It’s really interesting to see how simple ideas can be reimplemented in dynamic languages. Next steps: more samples, ruby gem, node package, and web site implementing code generation as a service (ASP.NET implementation commented in my old post). (see AjGenesis: Basis of Its Implementation).

Pending implementation languages: Python, maybe Java.

(Original photo from: http://commons.wikimedia.org/wiki/File:Table_Rock_Dam_during_April_2008_White_River_Flood.jpg)

Keep tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

AjGenesis: Basis of its Implementation

I dedicated a lot of time to the development of AjGenesis, my open source code generation engine (from a free model). It’s time to write down what I consider the key points of its implementation, the parts that makes AjGenesis what it is.

First: the use of a free model on memory, that it’s managed like a common object in a programming language. You don’t need to traverse XML path or something else. You can visit and manipulated one or mode models  like any other dynamic object. I didn’t adopt a type model or a model with a schema: the model is free, you can put on each model what you want to have in it. The model is deserialized from XML or from text files, or it could be injected for your program: AjGenesis can be hosted as a library in your own application.

Second: the use of a dynamic, simple language, to execute simple or complex task, with access to rich class library. In AjGenesis, I use AjBasic, a interpreter that can access .NET. It can use, create and change dynamic objects, like the models.

Third: the use of templates, with expressions and commands (i.e. loops) written using the dynamic language.

(Some early alternatives were: use XML with schemas for models, and XSLT for transformations. I rejected them because I felt they had shortcomings that could affect the clear development of the system).

See:
Building an Application with AjGenesis
Code Generation with AjGenesis in a Real Project
Models for Code Generation in AjGenesis
AjBasic: An Open Source Basic-alike Interpreter
Generating Code Hello World with AjGenesis

Having these three pillars, I used AjGenesis for years, in different technologies, languages and framework (from Java to .NET, from JSP to ASP.NET to ASP.NET MVC, from PHP4 to PHP5 to .NET4 to .NET5, DDL for MySql, MS SQL Server database, and one user is generating code for Ruby and Rails!). Notably, I’m generating text artifacts for technologies that didn’t exist at the beginning of the project, with minimal changes to AjGenesis itself. And it can run in Linux and Mac OSX thanks to the Mono project (see Running AjGenesis Using Mono and Ubuntu).

Ideas to implement: the ability of inject the source of the model (you could write your own serializer); you could choose the dynamic language to use (I plan to add AjSharp as dynamic language for tasks and templates).

But if I have the three key points I enumerated above, I could implement AjGenesis. Then, as a proof of concept, I started to implement the same ideas under Ruby: dynamic objects, that can be loaded from YAML (or XML, or other formats), the use of ruby for task programming, and its template using ERB (or, maybe, other template engines).

You can check my progress at: https://github.com/ajlopez/AjGenesisRb

Keep tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

Raise the Level of Abstraction (Part 1) Introduction

This century I wrote many posts about Code Generation. It’s a topic interested me since the eighties: how to make programs that write programs. But it’s not only code generation: it’s code generation from a model. That’s the key point to emphasize. Now, I want to write about the basis of such pragmatic solution to some of the problems in software development. My main assertion:

Code Generation From Model is an exercise of Raise the Level of Abstraction

The history of programming is a history of “raising the level of abstraction”. Instead to manipulate switch in the front panel of the first computers, we write programs using a language that is an abstraction: we use “int a”, as an abstraction of a word in memory. We write “foo(bar)” as an abstraction of save state in a stack, and jump to a memory address: we don’t write BALR (Branch and Link Register) instructions in IBM mainframe assembly language. Every day we use code generation: from the general/specific domain language our compilers generate the machine code, the only code that counts for our computers.

In this new post series I want to explore and write about these ideas:

– We improve our ideas and skills, but the technology is in constant change
– Abstract Model vs. Technology Models
– Separate problem from solutions
– Distill and reuse the developer knowledge
– Code generation FROM A MODEL as an exercise of abstraction
– Then, code generation from a model has a beneficial side-effect: shed light on the separation of what is important, from what is a technicality
– Code generation can (and must) coexist with human coding

Many of these topics were mentioned in my AjGenesis posts, but it’s time to present them as a integrated whole. The title of this post series “Raise the Level of Abstraction” was born in a technical talk at one of my customers, years ago. It’s good to find the same phrase as a “leit motiv” for Code Generation 2011:

http://www.codegeneration.net/cg2011/index.php

I remember an older phrase from the same site: “code generation for the pragmatic engineer”. Code generation from a model is not a silver bullet: is another bullet, you can use in the right context.

(Note: I found the images at top using Google Image search terms: “two apples”, “two oranges”, “two sheep” and…..  “two person” ;-)… no Microsoft promotion intended)

Keep tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

Code Generation and Artificial Intelligence

In late seventies and early eighties (past century ;-)) I was learning a lot about programming, programming languages and computer science in general. They were the days of COBOL, BCPL, IBM/360 assembler languages, JCL, punched cards, Pick Operating Systems, Pick Basic, ALGOL/W, Fortran, PL/I, and C (in CP/M, DOS, Unix/Xenix world). One of my preferred topics (then and now) was Artificial Intelligence, a broad term. Those days were the golden age of expert systems. I was writing a lot of C code, then, as a lazy programmer, I started to write my own C utilities to generate more C programs ;-). Since then, I have an idea:

Code Generation should be an Artificial Intelligence application

In the late eighties, early nineties, the amount of code to write to have a professional application diminished (new IDEs, wizards). But that was a transitory state: the complexity imposed by market, new customer requirements, online application, concurrency, new technologies and languages (Windows, GUIs…) all conspired to the increase of effort that should be employed to create any non-trivial application. I embraced Java at mid-90s, and .NET at 2001. Large class libraries, new tips and tricks, architectural styles, patterns, JSP, JSF, MVC, ASP.NET, Javascript, some PHP, distributed computing, physical tiers… The software development arena became more complex. And now, we have mobile, Android, Windows Phone 7, ASP.NET MVC, new JDKs, JVMs, new dynamic languages… I could continue this enumeration for ever, no break here 😉

So, at the begin of the century, I decided to start my open source project for code generation, AjGenesis. First, this is the current process I use in my code generation projects:

(Ok, I’m a developer, not a designer ;-))

The starting models (left) are free: you can use any model you want, it’s not fixed. This is a key decision: I want YOU have the freedom to start whit whatever you want as a base for your code generation. But it should be a MODEL. It’s not only code generation: it’s CODE GENERATION FROM A MODEL(S). Usually, I have an abstract model, like:

<Project>
	<Name>AjApp</Name>
	<Description>Building an Application using AjGenesis</Description>
	<Model>
		<Entities>
			<Entity Source="Entities/Customer.xml"/>
			<Entity Source="Entities/Supplier.xml"/>
		</Entities>
	</Model>
</Project>

and one or more technology model, like:

<Technology>
	<Programming>
		<Dialect>CSharp4NhMvc</Dialect>
	</Programming>
	<Database>
		<Dialect>MsSql</Dialect>
		<Name>AjTestNh</Name>
		<Host>.\SQLEXPRESS</Host>
	</Database>
	<NHibernate>
		<Dialect>NHibernate.Dialect.MsSql2000Dialect</Dialect>
	</NHibernate>
</Technology>

(more info in my AjGenesis posts). The models can be written using XML or text. Or you can use your own models (see Models for Code Generation in AjGenesis). The text artifacts could be Java, C#, VB.NET, Ruby, whatever you want. Notably: my AjGenesis examples started generating PHP, and JSP 1.x, and they evolved WITHOUT CHANGING THE TOOL to generate PHP (new versions), Java, JSP 2.x, Hibernate, NHibernate, ASP.NET 1.x, ASP.NET 2.x, ASP.NET MVC… It can be adapted to any text-based technology. To have such flexibility, it has no GUI IDE association. It’s has Spartan interface, but I liked it.

Since inception, I adopted a general dynamic language, AjBasic, to support tasks and templates: I didn’t want a transformation engine, from model to code via templates. I want to have the power of a GENERAL language. Why? TO SUPPORT EXTENSIBILITY, beyond template expansion. And, surprise! What kind of extensibility? Anyone, but you know…. yes, right! Artificial Intelligence!

My assertion:

Code Generation from a Model can be implemented as an Expert System, and then, it’s a clear target for Artificial Intelligence

Graphically:

You can view software developers as experts. We can distill their expertise in expert systems and add them to code generation process. Some examples:

– An expert system can use a rule engine to make decisions: macro or mini ones. Mini-one: “This web report can have lot of rows, I need pagination, filtering, and ordering….” And then, some part of the expert systems recommends: “Yes! We could use JQuery and JQuery.UI, client-side filtering”… and other part (yes, my expert systems is multi-mind ;-)) says “No! We need server-side filtering, then we need such queries and services, and Ajax endpoints and … “ Got the picture?

A micro-decision I employed last year in a real project: one kind of entity, in an abstract model, was marked as “massive” meaning: “we have lot of instances of Entity X”. Then, when in the UI we needed to select one Entity X, instead of a dropdown, the code generation engine put a search box, with auto-complete, with Ajax service at server.

I could use, for the abstract model, some base ontology and make assumptions and decisions. Example:

– Every entity could be: a person, a company, a place, a physical good, an event, a document. Then, the expert system could infer: a place has an address. A physical good should be located in a place. A document X should be needed to represent the move of a physical good from place A to place B. A person can be an employee. Every employee has social security… etc…. etc.. Then, we need a report of the X events, ordered by date, grouped by location, etc… etc… This document has a money total, it should be interesting to have a total grouped by any party (customer? company? person?) involved in the document. And by month? Yes, we need a pie chart … and so on… 😉

I could add a rule engine, to make some of these kind of decisions, with rules like “if entity X is massive and the UI is web and JQuery is an available option then … “. Or rules like: “if entity X inherits from entity Y and we need persistence, we adopt table-per-hierarchy (or table-per-concrete class) ORM strategy, and then we generate…. database tables and …. ORM mapping code or configuration”. Or: “this entity X is a document having a date, we add an index on that date/time column in Y table”. So many decisions can be made in this way. Actually, in AjGenesis, most of these decisions are in the tasks and technology models.

In the long term, code generation from a model involves:

– This is the starting point: abstract models, technical models (restrictions: “we need all this in ASP.NET MVC 3”)

– This is the expected deliverable (a partial working system)

– And these are the distilled expert knowledge we got from our/others experience.  And with all these, we can go from start to end. using the accumulated programming skills, ORM configuration, dependency injection libraries, UI tips and tricks, recurrent patterns, database creation knowledge, etc..

Software development and architecture is related to:

– Know what you need

– Know what are the problems to resolve (persistence, concurrency, security, traceability, UI access, etc…)

– Know what are the available solutions (database, NoSql, language, class libraries, frameworks, UI technology, patterns…)

Code generation from a model has a (great, IMO) benefit: it put clear the existence, separation and relations of the above points.

Other way to explore: the use of expert agents. What does “expert agent” mean in my jargon? My vision: you can have modules, agents that can be added, removed without much effort, that can resolves problems. Then, someone can develop a JSF expert, that knows what JavaBeans to define for implement a given interface in a use case. Then, if in the future the rule engine or the initial restrictions say: “Now, we need JSF 17.2, or ASP.NET MVC 6.5 or the new SuperDuperOpenSourceUIThatIsTheNextBigThingHere”, we can adopt new solutions for the similar problems, distilling the new knowledge and putting it in a new expert agent.

Ok, all these are the reasons for an (upcoming?) AjRules ;-). And the justification for AjLisp (see here) and some AjProlog to be updated (see it inside AjCodeKatas).

Related posts (in Spanish):

Models, decisions, Artificial Intelligence and AjGenesis

About Code Generation

Keep tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez