Angel \”Java\” Lopez on Blog

May 25, 2013

Mass Programming Language (4) Lexer and Parser

Filed under: .NET, C Sharp, Mass, Open Source Projects, Programming Languages — ajlopez @ 10:38 am

Previous Post

In the Mass programming language implementation, I have an enumeration and a class:

A token represents a word of code to be processed. Lexer is in charge of separate the code in words/tokens. And Parse takes that token stream and returns expressions and commands:

Lexer constructor receives a string:

public Lexer(string text)
{
    this.text = text;
}

This string is processed to be separated into tokens. Notice the lexer distinguish between operators (like +) and separators (like parenthesis). It take into account the end of line as a token, too (in other programming language, like C, the end of line is simply a blank space). The main method of Lexer is NextToken that returns the next token in code text. In some situations, it is needed to save a consumend token, so there are methods like PushToken and its variants.

Internally, Parser manages a Lexer. You can get the next command calling ParseCommand method, and the next expression using ParseExpression. When the text is exhausted, those methods return null.

I should modify Lexer to consume a text stream, instead of a string, so it could process an input, like console.

I should think about unifying commands and expressions, a la Ruby where all is an expression.

Keep tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

May 5, 2013

Mass Programming Language (3) Commands

Filed under: .NET, C Sharp, Mass, Open Source Projects, Programming Languages — ajlopez @ 6:16 pm

Previous Post
Next Post

Today let’s review command implementation in Mass (see repo). In the class library project, I have a dedicated namespace and folder for commands:

There are commands for if, while, for, for each, etc… Every command implements the ICommand interface:

public interface ICommand
{
    object Execute(Context context);
}

See that is very similar to IExpression. But I wanted to keep a separation between commands and expressions, at least for this first implementation, in order to have a clear separation of basis concerns.

A typical example of command is WhileCommand, partial code:

public class WhileCommand : ICommand
{
    private static int hashcode = typeof(WhileCommand).GetHashCode();

    private IExpression condition;
    private ICommand command;

    public WhileCommand(IExpression condition, ICommand command)
    {
        this.condition = condition;
        this.command = command;
    }

    public object Execute(Context context)
    {
        for (object value = this.condition.Evaluate(context); 
            value != null && !false.Equals(value);
            value = this.condition.Evaluate(context))
        {
            this.command.Execute(context);
            if (context.HasContinue())
                context.ClearContinue();
            if (context.HasBreak())
            {
                context.ClearBreak();
                break;
            }
        }

        return null;
    }
}

In Mass, every null or false value is false. All other value is true. I should refactor the code to have a central method IsFalse to be invoked in the above While code and in other commands, like IfCommand.

Another sample of command is ForEachCommand, partial code:

public class ForEachCommand : ICommand
{
    private string name;
    private IExpression expression;
    private ICommand command;

    public ForEachCommand(string name, IExpression expression, ICommand command)
    {
        this.name = name;
        this.expression = expression;
        this.command = command;
    }

    public object Execute(Context context)
    {
        var values = (IEnumerable)this.expression.Evaluate(context);

        foreach (var value in values)
        {
            context.Set(this.name, value);
            this.command.Execute(context);
            if (context.HasContinue())
                context.ClearContinue();
            if (context.HasBreak())
            {
                context.ClearBreak();
                break;
            }
        }

        return null;
    }
}

See that for Mass every IEnumerable value can be used as the element provider for ForCommand. I added some methods to the current context to signal the present of a break or continue (in the current version, I added return treatment, too).

And, as usual, all this code was developed using TDD flow: you can read repo commit history.

Next posts: Lexer, Parser, Mass samples, Mass scripting, using Mass from our .NET programs.

Keep tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

May 3, 2013

New Month’s Resolutions: May 2013

Filed under: NodeJs, Open Source Projects — ajlopez @ 4:18 pm

Review of my April’s resolutions:

- Update Mass [complete] see repo
- Start NodeAima [complete] see repo
- Give a one-day course on Node.js [complete]
- Update AjGenesis [pending]
- Update AjGenesisRuby [pending]
- Update SimpleGo [pending]
- Update RubySharp [complete] see repo 
- Update ClojSharp [pending]

Additionally, I worked on:

- Update AjFabriqNode [complete] see repo
- Update NodeSamples [complete] see repo (see Distributed Fractal)
- Update SimpleStorm [complete] see repo
- Update SimpleBus [complete] see repo
- Update SimpleBroadcast [complete] see repo
- Update SimpleGA [complete] see repo

All this work is related to a talk I gave one week ago, about Distributed Applications and Node.js

May’s resolutions:

- Update Mass
- Update RubySharp
- Give a talk, Intro to Ruby
- Update AjGenesis
- Update ClojSharp
- New posts about Mass
- New posts about Distributed Applications and Node.js

More fun and work.

Keep tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

April 30, 2013

Mass Programming Language (2) First Expressions

Filed under: .NET, Programming Languages, C Sharp, Open Source Projects, Mass — ajlopez @ 9:31 am

Previous Post
Next Post

Before using Mass language in samples (see repo), I want to visit some implementation points. First, a news: now there is a solution (at (en https://github.com/ajlopez/Mass/blob/master/Src/Mass.sln) that can be compiled with Visual Studio C# Express.

The Mass solution has a class library project. There is a namespace dedicated to expressions:

An expression implements IExpression:

  
public interface IExpression
{
	object Evaluate(Context context);
}

A Context keeps a key/value dictionary, to save the current variables:

  
public void Set(string name, object value)
{
	this.values[name] = value;
}

public object Get(string name)
{
	if (this.values.ContainsKey(name))
		return this.values[name];

	if (this.parent != null)
		return this.parent.Get(name);

	return null;
}

(notice that it supports nested context: each context can have a parent).

A simple expression is ConstantExpression, that returns a constant, without using the received context. A constant is any .NET value/object:

  
public class ConstantExpression : IExpression
{
	// ...

	private object value;

	public ConstantExpression(object value)
	{
		this.value = value;
	}

	public object Evaluate(Context context)
	{
		return this.value;
	}

	// ...
}

Another expression is the one that evaluates a variable name in the current context:

  
public class NameExpression : IExpression
{
	// ...

	private string name;

	public NameExpression(string name)
	{
		this.name = name;
	}

	public object Evaluate(Context context)
	{
		return context.Get(this.name);
	}

	// ...
}

The context is important: it could be the context of the current function, of a closure, of current object, of current module, etc… The same name could refer defers differente associated variables, as in other programming languages.

As usual, all these classes (expressions, Context, …) were written using the flow of TDD. You can check the test project and the repo commits history were there is evidence of that flow.

Next posts: some additional expressions, commands, using Mass for scripting, using Mass from our .NET programs.

Kee tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

April 25, 2013

Mass Programming Language (1) Inception

Filed under: .NET, C Sharp, Mass, Open Source Projects, Programming Languages — ajlopez @ 9:09 am

Next Post

Three weeks ago, I was working on the implementation of an interpreted language, written in C#. The new language is called Mass (dedicated to  @MArtinSaliaS):

https://github.com/ajlopez/Mass

The current solution has three projects: a class library, its tests, and a console program, mass.exe, that launches Mass programs

You can run a hello.ms:

mass hello.ms

The classic Hello world source code:

println("Hello, world")

An example with classes and objects

class Person
	define initialize(firstname, lastname)
		self.firstname = firstname
		self.lastname = lastname
	end
	
	define getName()
		return self.lastname + ", " + self.firstname
	end
end

adam = new Person("Adam", "TheFirst")

println(adam.getName())

An example with access to .NET types and objects:

dirinfo = new System.IO.DirectoryInfo(".")

for fileinfo in dirinfo.GetFiles()
	println(fileinfo.Name)
end

The idea is  to have a dynamic language that leverages an underlying language provided with a rich class library and ecosystem, like in AjSharp and other of my projects. Before Mass, I was working on:

- Implementing Python in C# (see PythonSharp)

- Implementing Ruby in C# (see RubySharp)

- AjSharp (see repo and post)

But this time I wanted to implement something with simple sintax and semantic. Indeed, I was playing with “simple” ideas for a compiler over Javascript, see SimpleScript (1) First Ideas.

Then, with Mass, I deliberately wanted to avoid:

- Multiple commands in the same line (I discarded ‘;’ like in Ruby)

- Syntax based in spaces and indentation (Python discarded)

- Function invocation using only the name; Mass impose the explicit use of parenthesis (Ruby discarded; Mass is like JavaScript)

- Base values and classes (integers, strings, lists, etc…) having a crowd of methods (like Ruby and Python). No, Mass prefers to expose and use the underlying language/class library.

Then, I wanted:

- Functional values, as first-class citizens, like in JavaScript. So, having to put explicit parenthesis to invoke a function allows me to use the name of the function as a functional value

- Dynamic objects: each object can be extended at any moment, with new instance variables, object functions, a la JavaScript

- Syntax based in lines: each command has its own line. No command separation

- Syntax based in keywords: the end of a command list is marked with ‘end’, no braces

- As far as possible, only one way to do something, instead of the many ways motto a la Perl

- Complete keywords, then ‘define’ instead of ‘def’

- Simple inheritance at classes. But Mass could be expressive without written classes, using native classes from .NET framework and other libraries. It could be used as an scripting language.

- Explicit setting of variables that are out of the local scope (a topic for next posts)

- Variable scope by file, like in the require of JavaScript/NodeJs/CommonJS

- Module by file, with a require that automatically searches in directories, a la NodeJs/CommonJs. Notably, Mass can consume node_modules folder, so Mass module can be published and installed using NPM!

- Package manager, using NPM. You can use package.json to declare the dependencies, and publish new modules at NPM (using ‘mass-‘ as the suggested namespace).

In upcoming posts, I will write more details about implementation, guiding design ideas, examples. But now, you can see the code and the test examples at public repo. And yes, all was written by baby steps, using TDD.

Keep tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

April 2, 2013

New Month’s Resolutions: April 2013

Filed under: Open Source Projects — ajlopez @ 11:05 am

A new month starts. Review of my March’s resolutions:

- Start ClojSharp [complete] see repo
- Start NodeAima [pending]
- Update AjGenesisRuby [pending]
- Start RubySharp [complete] see repo
- Update SimpleGo [complete] see repo
- Update SimpleChess [pending]
- Update Inmob [complete] see repo
- Update AjConsorSite [complete] see repo

Instead of working on AjGenesisRb, I was working on

- Update and publish AjLispRuby as a gem [complete] see repo
- Migrate and update Todocontenidos to GitHub [complete] see repo
- Publish current version of AjGenesis, and migrate it to GitHub [complete] see downloads, see repo, see Spanish post 

Additionally to ClojSharp and RubySharp, I started a new simple interpreted language

- Start Mass [complete] see repo

And

- Give a talk about TDD [complete] see Spanish post

April’s resolutions:

- Update Mass
- Start NodeAima
- Give a one-day course on Node.js
- Update AjGenesis
- Update AjGenesisRuby
- Update SimpleGo
- Update RubySharp
- Update ClojSharp

More work and fun is coming.

Keep tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

March 3, 2013

New Month’s Resolutions: March 2013

Filed under: Open Source Projects — ajlopez @ 5:54 pm

Time for review my February 2013 resolutions:

- Start SimpleDatabase [complete] see repo
- Update SimplePermissions [pending]
- Update SimpleStorm (implements ack, maybe integrate with MultiNodes) [pending]
- Update SimpleMapReduce [complete] see repo
- Update SimpleKeeper, leader, distributed example [complete] see repo
- Update AjFabriqNode, integrate with new SimpleMessages, maybe with MultiNodes [pending]
- Update MultiNodes [complete] see repo
- Update AjGenesisNode, to use global commands and tasks [pending]
- Give a full-day course about Node.js [complete] see Spanish post

Additionally, I was working on:

- Update Inmob [complete] see repo
- Update AjConsorSite [complete] see repo
- Update AjGroupsJs [complete] see repo
- Update SimpleBoard [complete] see repo
- Update SimpleChess [complete] see repo
- Update SimpleGo [complete] see repo
- Update NodeSamples [complete] see repo

My resolutions for March 2013:

- Start ClojSharp
- Start NodeAima
- Update AjGenesisRb
- Start RubySharp
- Update SimpleGo
- Update SimpleChess
- Update Inmob
- Update AjConsorSite

More work, more fun ;-)

Keep tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

February 2, 2013

New Month’s Resolutions: February 2013

Filed under: C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 3:07 pm

Time to review my January Resolutions:

- Start SimpleScript [complete] see repo
- Start SimpleBoard [complete] see repo
- Start SimpleChess [complete] see repo
- Start SimpleGo [complete] see repo
- Start and publish a version of SimpleMapReduce, with local and distributed sample [partial] see repo (only local version)
- Start and publish a version of SimpleFunc, object with functions serialization [complete] see repo
- Start Memolap, C# in-memory multidimensional OLAP-like library and sample [partial] see repo (sample WIP)
- Start SimpleMemolap, the same but in JavaScript/Node.js [complete] see repo
- Start SimpleRules, forward-chaining rule engine, that compiles to JavaScript [complete] see repo

Additionally, I was working on:

- Update AjConsorSite [complete] see repo
- Update Inmob [complete] see repo
- Start SimpleKeeper, Zookeeper-like server [complete] see repo
- Start and publish first version MultiNodes [complete] see repo
- Update SimpleStorm, publish new version using SimpleQueue 0.0.2 [complete] see repo
- Update SimpleQueue publish new version [complete] see repo
- Update SimpleRemote publish new version using SimpleMessages 0.0.3, and async methods [complete] see repo
- Publish first version AjFabriqNode [complete] see repo
- Start and publish first version NodeDelicious [complete] see repo
- Refactor SimpleBroadcast to use SimpleMessages 0.0.3 [complete] see repo
- Publish new version SimpleMessages 0.0.3 [complete] see repo
- Start and publish first version MProc, middleware layer for message processing Node.js, [complete] see repo
- Start and publish first version SimpleTags, engine to manage items with arbitrary data and tags [complete] see repo
- Start and publish first version ObjectStream, bidirectional and unidirectional object streams for Node.js [complete] see repo
- Start and publish first version SimplePipes, yet another flow library in Node.js [complete] see repo
- Start and publish first version SimpleSudoku, sudoku solver in JavaScript/Node.js [complete] see repo
- Start and publish first version SimplePermissions, permissions by Subject, Role, and Context. Model in-memory [complete] see repo
- Start and publish first version SimpleGlobals, inspired by Mumps Globals [complete] see repo
- Start and publish first version SimpleInvoke, chained invocation of functions with callbacks, JavaScript/NodeJs [complete] see repo

For this new month:

- Start SimpleDatabase
- Update SimplePermissions
- Update SimpleStorm (implements ack, maybe integrate with MultiNodes)
- Update SimpleMapReduce
- Update SimpleKeeper, leader, distributed example
- Update AjFabriqNode, integrate with new SimpleMessages, maybe with MultiNodes
- Update MultiNodes
- Update AjGenesisNode, to use global commands and tasks

I will give a seminar (full day) of Node.js at Rosario, Argentina

Keep tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

January 19, 2013

CobolScript (4) Web Pages with Templates

Filed under: COBOL, CobolScript, JavaScript, NodeJs, Open Source Projects — ajlopez @ 3:47 pm

Previous Post

In the previous post I presented CobolScript generating output using templates. It can be used to generate web page output, too. The sample is at:

https://github.com/ajlopez/CobolScript/tree/master/samples/templateweb

The launch program in JavaScript is simple:

var cobs = require('../..'),
    http = require('http'),
    fs = require('fs');

var program = cobs.compileTemplateFile('./factorial.cobp');

http.createServer(function(req, res) {
    program.run(cobs.getRuntime({ request: req, response: res }));
}).listen(8000);

console.log('Server started, listening at port 8000');

The key part is the call to compile the file template. It produce a compiled JavaScript function to be invoked. The call of program.run executes the template, with a giving runtime context. The runtime context is build giving the request and response objects of the current incoming request. That runtime derives all the output of the CobolScript program to the response output. So, the template doesn’t know about web request and response. You can see the runtime object as a service provider to the compiled CobolScript program. Its properties can be accessed if you define a LINKAGE SECTION as in classic COBOL. But this feature was not used in this simple sample.

The template file

<h1>Factorial</h1>
<p>Page generated by CobolScript, using templates</p>
<table>
<tr><th align='right'>n</th><th align='right'>n!
</th>
</tr>
<#
local n.
perform show-factorial using n varying n from 1 to 10.
#>
</table>
<#
.
stop run.

show-factorial section using n.
local result.
perform factorial using n giving result.
#>
<tr>
<td align='right'>${n}</td><td align='right'>${result}
</td>
</tr>
<#
.

factorial section using n.
local m.
if n = 1 then return n.
subtract 1 from n giving m.
perform factorial using m giving m.
multiply n by m.
return m.
#>

Launch the server with

node server

Navigate to localhost:8000, the result:

Next post: a dynamic web site accessing MySQL database, using CobolScript.

Keep tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

January 16, 2013

Some projects with in-memory model

Filed under: JavaScript, NodeJs, Open Source Projects — ajlopez @ 5:05 pm

I’m a big proponent of having models (domain models) residing in memory. Of course, in the appropriate context. There are many problems that could have a better solution if we forgot the “use the database, Luke” approach. One notable solution/project is Redis, that using a single thread implements a complete and powerful key-value store in memory, with options for persistence and clustering (redis cluster still under development).

I’m playing implementing personal projects using an in-memory model. There are:

SimpleMemolap: Multidimensional OLAP-like model, in JavaScript/Node.js with a web sample.

Memolap: Multidimensional OLAP-like model, in C#. Web sample in progress.

SimplePermissions: Subjects, Roles, Permissions, Contexts, all in memory.

AjKeyvs: Redis-like in-memory key-value store, in C#. I should add the client protocol. I should review the internal algorithms, but all was coded using TDD, ready for refactors.

SimpleStore: Simple Key Value Store in memory, a la Redis, JavaScript/Node.js. In process or exposed to remote clients using additional modules. Work in Progress.

SimpleQueue: in-memory queue implementation, JavaScript/Node.js. It could be used in process or acceded via exposing the methods to remote calls.

SimpleTags: having items with arbitrary data, tagged with strings or key-value objects.

Maybe I could add SimpleRules: rule engine where the facts reside in-memory (many rule engines use directly the memory to represent the world state).

And even I could mention: SimpleBoggle, where the list of words to resolve a Boggle game it’s loaded in memory ;-)

To be started:

SimpleDatabase: in-memory database, with SQL-like query language, JavaScript/Node.js

Most of these projects could be extended to have pluggable persistence. But now I’m interested in having something that works to play with. I’m having fun implementing internal algorithms, trees, search, inverted indexes, etc. And practicing TDD every day.

Keep tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

Older Posts »

Theme: Shocking Blue Green. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 28 other followers