Angel \”Java\” Lopez on Blog

December 10, 2014

SimpleLisp (2) Compiling Lisp Values and Variables to JavaScript

Previos Post

Let’s review the project’s implementation

https://github.com/ajlopez/SimpleLisp

the Lisp compiler to JavaScript, written in JavaScript, following the workflow of TDD (Test-Driven Development). All is “work in progress”, so today I will explain part of the implementation but it could be change in the future, when new use cases were added, and new ways of doing things were implemented.

I took the decision that each Lisp symbol is a JavaScript variable. So, the compilation of a symbol is:

exports['compile symbol'] = function (test) {
    test.equal(sl.compile('a'), 'a');
}

The sl is the SimpleLisp module, loaded in this test file (test/compile.js)

An integer and an string are compiled to natural values in JavaScript:

exports['compile integer'] = function (test) {
    test.equal(sl.compile('42'), '42');
}

exports['compile string'] = function (test) {
    test.equal(sl.compile('"foo"'), '"foo"');
}

It’s quoted values, too:

exports['compile quoted integer'] = function (test) {
    test.equal(sl.compile("'42"), '42');
}

exports['compile quoted string'] = function (test) {
    test.equal(sl.compile("'\"foo\""), '"foo"');
}

I decided to compile Lisp nil to JavaScript null. The boolean values are the same:

exports['compile nil'] = function (test) {
    test.equal(sl.compile('nil'), 'null');
}

exports['compile booleans'] = function (test) {
    test.strictEqual(sl.compile('false'), 'false');
    test.strictEqual(sl.compile('true'), 'true');
}

But, what happens if an expression serie is compiled? I build an anonymous function, invoked without arguments, and the last expression value is returned:

exports['compile two symbols'] = function (test) {
    test.equal(sl.compile('a b'), '(function () { a; return b; })()');
}

Next topics: compilation of a Lisp lisp, and more quoted values, macros, ect.

Stay tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

November 25, 2014

SimpleLisp (1) Compiling Lisp to JavaScript

Next Post

I implemented Lisp as interpreter, using C#, Java, and JavaScript. See:

https://github.com/ajlopez/AjLisp
https://github.com/ajlopez/AjLispJava
https://github.com/ajlopez/AjLispJs

The implementation of a Lisp is a good programming exercise. Lisp is a simple and powerful language, with functions as first class citizens. And with the “twist” of implementing functions that don’t evaluate their arguments, and macros.

This time, I want to implement a Lisp, but as a compiler. I started to write a Lisp compiler in JavaScript, that generates JavaScript. The project:

https://github.com/ajlopez/SimpleLisp

As usual, I worked using the Test-Driven Development workflow. With simple use cases, I implemented the needed features. This is my first Lisp compiler, so I’m trying new implementation approaches. I know Clojure, as a basis for a Lisp compiler. I should implement:

Symbols: identifiers witn name and associated value. Now, I’m compiling them to JavaScript variables. In SimpleLisp, a symbol can be defined at top, in a let block, or as a function argument. Then, I produce a top variable (or a least, a module variable), or a local variable in let, or an argument in function.

Functions: I’m translating a normal function in Lisp to a normal function in JavaScript. The main difference is that Lisp functions returns a value, there are no commands, all are expression, as in Ruby. A list in SimpleLisp is then compiled to a function call in JavaScript.

Special Forms: Their implementation is a novelty to me. In a compiler, I could generate directly the final code for each list with a head that is an special form. So, I’m compiling directly to JavaScript any list with head if, do, let, etc… . A list (if ….) is compiled to a JavaScript if, but returning a value.

Macros: In a compiler, I could adopt a new implementation: to expand the macro at COMPILING TIME. I’m not sure yet if I can take this approach in every case. Notably, in Clojure a macro is not a functional value: a macro cannot be passed as an argument. A macro make sense only at compiling time.

A final use case: write a web site using Node.js/Express in SimpleLisp.

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

November 11, 2014

SimpleScript (2) The Lexer

Previous Post
Next Post

These days, I was working improving my SimpleScript compiler to JavaScript. Today, I want to comment the implementation of the lexer, the repo is

http://github.com/ajlopez/SimpleScript

Now, the lexer code resides in a dedicated file lib/lexer.js, that expose a module, that can be consumed from Node.js and from the browser. It starts defining the token types:

var lexer = (function () {
    var TokenType = { 
        Name: 1, 
        Integer: 2, 
        Real: 3, 
        String: 4, 
        NewLine: 5, 
        Separator: 6, 
        Assignment: 7 };

Then, it defines some operatores, delimiters, and the Token, with two elements, type and value.

var separators = ".,()[]";
var assignments = ["=", "+=", "-=", "*=", "/="];
var operators = ["+", "-", "*", "/", "==", "!=", "<", ">", "<=", ">="];

function Token(value, type) {
    this.value = value;
    this.type = type;
}

The main job is in the Lexer “class”, with the method nextToken:

function Lexer(text) {
    var length = text ? text.length : 0;
    var position = 0;
    var next = [];

    this.nextToken = function () {
        if (next.length > 0)
            return next.pop();

        skipSpaces();

        var ch = nextChar();

        if (ch === null)
            return null;

        if (ch === '"' || ch === "'")
            return nextString(ch);

        if (ch === '\n')
            return new Token(ch, TokenType.NewLine);

        if (ch === '\r') {
            var ch2 = nextChar();

            if (ch2 === '\n')
                return new Token(ch + ch2, TokenType.NewLine);

            if (ch2)
                pushChar(ch2);

            return new Token(ch, TokenType.NewLine);
        }

        if (isAssignment(ch))
            return new Token(ch, TokenType.Assignment);

        if (isOperator(ch))
            return nextOperator(ch);

        if (isSeparator(ch))
            return new Token(ch, TokenType.Separator);

        if (isFirstCharOfName(ch))
            return nextName(ch);

        if (isDigit(ch))
            return nextInteger(ch);
    }

Finally, the module expose a lexer factory and the enumeration of token types:

return {
    lexer: function (text) { return new Lexer(text); },
    TokenType: TokenType
}

The code was developed using Test-Driven Development workflow. There is a file test/lexer.js, fragment:

function getToken(text, value, type, test) {
    var lexer = sslexer.lexer(text);
    var token = lexer.nextToken();
    test.ok(token);
    test.equal(token.value, value);
    test.equal(token.type, type);
    test.equal(lexer.nextToken(), null);
};

exports['Get names'] = function (test) {
    getToken('foo', 'foo', TokenType.Name, test);
    getToken('foo123', 'foo123', TokenType.Name, test);
    getToken('foo_123', 'foo_123', TokenType.Name, test);
    getToken('_foo', '_foo', TokenType.Name, test);
}

exports['Get integer'] = function (test) {
    getToken('123', '123', TokenType.Integer, test);
    getToken('1234567890', '1234567890', TokenType.Integer, test);
}

Remember: no TDD, no paradise ;-)

Next topics: the parser, commands and expressions implementations, compilation to JavaScript.

Stay tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

October 31, 2014

Smalltalk: Links, News And Resources (20)

Filed under: Links, Programming Languages, Smalltalk — ajlopez @ 6:42 pm

Previous Post

Objective-Smalltalk talk at FOSDEM 2014 – YouTube
https://www.youtube.com/watch?feature=player_embedded&v=NZmgVsk1K6k

Alan Kay On Messaging
http://c2.com/cgi/wiki?AlanKayOnMessaging

Pharo Smalltalk – YouTube
https://www.youtube.com/channel/UCp3mNigANqkesFzdm058bvw

pathToolsFramework – Squeak Community Projects
https://www.hpi.uni-potsdam.de/hirschfeld/trac/SqueakCommunityProjects/wiki/pathToolsFramework

Pharo4 Plans and Dreams – YouTube
https://www.youtube.com/watch?v=mUV9E03u52g&feature=youtu.be&a

39 – Leandro Caniglia – self ideas asStream next – YouTube
https://www.youtube.com/watch?v=tuOqJU9BtkQ

20 – Nicolas Petton – Amber Smalltalk – YouTube
https://www.youtube.com/watch?v=L6ixtb9w8R0

32 – Hernan Wilkinson – Web Development In Smalltalk For Newbies – YouTube
https://www.youtube.com/watch?v=73MZkGEG-uc

15 – Dale Henrichs – tODE: And Now For Something Completely Different – YouTube
https://www.youtube.com/watch?v=m4LcZ4_1Yic

Smalltalk reddit
http://www.reddit.com/r/smalltalk

Where Smalltalk Went Wrong 2
http://www.ianbicking.org/where-smalltalk-went-wrong-2.html

Where Smalltalk Went Wrong
http://www.ianbicking.org/where-smalltalk-went-wrong.html

Saying Goodbye To Python
http://www.ianbicking.org/blog/2014/02/saying-goodbye-to-python.html

Interview with Esteban Lorenzano
http://nicolas-petton.fr/blog/interview-esteban.html

RMoD
http://rmod.inria.fr/web/
The goal of RMoD is to support remodularization of object-oriented applications. This objective is tackled from two complementary perspectives: reengineering and modularity constructs for programming languages.

PhaROS – FOSDEM 2014 – Slides
http://car.mines-douai.fr/2014/02/pharos-fosdem-2014-slides/

My Links
https://delicious.com/ajlopez/smalltalk

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

August 19, 2014

Smalltalk: Links, News And Resources (19)

Filed under: Links, Programming Languages, Smalltalk — ajlopez @ 3:33 pm

Previous Post
Next Post

ESUG: 2014
http://www.esug.org/wiki/pier/Conferences/2014

Pharo Open Source Smalltalk — News
http://www.pharo-project.org/news?dialog=pharo-3-at-fosdem&

mpw/Objective-Smalltalk
https://github.com/mpw/Objective-Smalltalk

Philippe Back Interviews Nicolas Petton on Amber Smalltalk by philippeback on SoundCloud – Hear the world’s sounds
https://soundcloud.com/philippeback/nicolas_petton_on_amber_smalltalk

pharo-project/pharo-vm
https://github.com/pharo-project/pharo-vm

smallworks: Why I’m using git in my Pharo projects
http://smallworks.eu/web/blog/2014-01-16-Why-Im-using-git

Dyla’14 at PLDI, Edinburgh, UK
http://www.lifl.fr/dyla14/

Morphic
http://wiki.squeak.org/squeak/30

docs.selflanguage.org/4.4/pdf/SelfHandbook-Self4.4-R2.pdf
http://docs.selflanguage.org/4.4/pdf/SelfHandbook-Self4.4-R2.pdf

stephane.ducasse.free.fr/FreeBooks/CollectiveNBlueBook/morphic.final.pdf
http://stephane.ducasse.free.fr/FreeBooks/CollectiveNBlueBook/morphic.final.pdf

Self Mallard (4.5.0) released | Self
http://blog.selflanguage.org/2014/01/12/self-mallard-4-5-0-released/

Self | Fun through simplicity
http://blog.selflanguage.org/

stic13 – YouTube
http://www.youtube.com/playlist?list=PL8aumV3VUz66Rs8-xcD8V57ZTBNYPlzmj

(349) Programming Languages: Why create any more programming languages? – Quora
http://www.quora.com/Programming-Languages/Why-create-any-more-programming-languages

Anyone interested in smalltalk? : learnprogramming
http://www.reddit.com/r/learnprogramming/comments/1to0wm/anyone_interested_in_smalltalk/

ST 4U 510: Dynamic Code Generation in Pharo
http://www.jarober.com/blog/blogView?entry=3565419024

Was Alan Kay wrong? And why does that matter?
http://braythwayt.com/2013/12/22/wrong.html

Smalltalk on Raspberry Pi | a blog on Smalltalk on the Raspberry Pi
http://pismalltalk.wordpress.com/

My Links
http://delicious.com/ajlopez/smalltalk

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez
http://github.com/ajlopez

July 27, 2014

AjTalk Implementing Smalltalk In C# (4) The Project

Filed under: .NET, AjTalk, C Sharp, Programming Languages, Smalltalk — ajlopez @ 5:48 pm

Previous Post

Year and a half since I didn’t write about this project in C #, but I was working on it, and the parallel implementation I have in JavaScript (in fact, I introduced some concepts of both the Smalltalks 2013 conference, Rosario, Argentina).

The project is at:

https://github.com/ajlopez/AjTalk

Current structure:

The projects are:

AjTalk: class library, with a compiler to bytecodes implementation. There is a compiler to JavaScript too. There is an interpreter of those bytecodes.

AjTalk.Compiler: a console project that uses the compiler to JavaScript.

AjTalk.Console: a simple REPL (Read Eval Print Loop)

AjTalk.Gui: a work-in-progress example using windows

AjTalk.Tests: the test I wrote, followin TDD (Test-Driven Development) flow.

I guess now I don’t need the compiler to JavaScript, because I have a native JavaScript implementation at:

https://github.com/ajlopez/AjTalkJs

It can compile to bytecodes or to JavaScript. The project is written in JavaScript.

The window implementation is very primitive. I plan to replace it for direct AjTalk program using .NET types and object.

One point that I find very interesting (in both projects, C # and JavaScript) is to have a modular implementation: instead of loading an image yet defined, my plan is to define modules to load, as happens in the Node.js environment . The modules would be published in the NPM (Node Package Manager’s), and each could indicate which modules need in your program instead of lifting all Smalltalk.

Another topic in this C # project I have implemented that can run more than one AjTalk machine at the same time. And I can even make a machine “help” to the other. For example, if the machine does not have to compile methods to new methods, machine B can go to his aid. Thus, A machine (and his image, I’m already recording binary image) can be kept small.

Another feature that I have implemented: the ability to send a message to an object, without waiting for a response, a “fire and forget”. And that the target object can attend ONE OF those messages as models of actors.

I have to review the implementation of remote objects (now based on the old Remoting. NET). But I think it is an important feature to maintain and explore.

Well, you see, many issues remain for other posts, such as implementation details Smalltalk concepts such as Object, Class, ClassDescription, Behavior, MetaClass, etc..

Nos leemos!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

July 22, 2014

ClojSharp Implementing Clojure In C# (1) The Project

Since the past decade I’m interested on Clojure. It’s a return to Lisp, but in modern times.. Clojure is implemented on the Java virtual machine, but is also implemented on. NET (AFAIK, using Dynamic Language Runtime). The version of Java compiles to Java bytecodes. There is also a version of JavaScript, called ClojureScript. The community is very active, and has brought an air of renewal within Java. It has features like immutability of his work-oriented structures in multithreading, avoiding concurrency problems. It supports Software Transactional Memory, but has been less used by what I see. Web frameworks have emerged, and has a project and dependencies manager called Leiningen. Some links in my Clojure posts.

To deliberate practice TDD (Test-Driven Development) and C#, I was building an interpreter (not a compiler to bytecodes, yet). The project:

https://github.com/ajlopez/ClojSharp

Current status:

It has a class library project, and a test project. There is also a REPL (Read Eval Print Loop) simple console application.

In Lisp, an expressions is composed by a verb and arguments, written in a list enclosed by parenthesis. For example, a list that at evaluation returns the sum of 1, 2 and 3:

(+ 1 2 3)

Here, the plus + sign is a symbol that in Lisp/Clojure refers to a form. A form receives arguments, operates on them, and returns a result. A typical form receives its arguments as evaluated expressions: the expressions are evaluated BEFORE the evaluation of the form:

(+ 1 (- 3 4) 3)

In the above example, the first argument to plus is a constant 1, and the second argument is an expression that is evaluated to –1 before the evaluation of the outer list.

But there are forms that not need evaluated arguments. They receive the arguments without evaluation. They are called special forms. I have implemented:

And I have implemented some (non special) forms as primitives in C#. Most of them could be implemented in Lisp itself, but being so basic that I preferred to implement them directly in C#, using TDD as usual:

Next topics: the Machine implementation, Context, how to use Parser and Lexer, more about expression evaluation, macros, tail recursion, etc.

Stay tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

July 19, 2014

SharpGo Implementing Go in C# (1) The Project

Filed under: .NET, C Sharp, Go, Google, Open Source Projects, Programming Languages, SharpGo — ajlopez @ 5:06 pm

There is another project of mine to practice TDD (Test-Driven Development) and C#, at:

https://github.com/ajlopez/SharpGo

It is an interpreter (not a compiler) of Go programming language, written in C#. It has a class library project, and a test project. Go is a typed language, so I cannot follow the path of build an expression tree, and evaluate it, as in RubySharp, PythonSharp and Mass. Instead, the interpreter should have many phases, as in ScalaSharp, to detect type inconsistencies, and use the declaration of unknown types at the parse process.

The AST is implemented:

The AST (Abstract Syntax Tree) nodes are in two groups:

- Command Nodes, where they should generate commands to be executed.

- Expression Nodes, that generates expressions to be evaluated. They have an associated type.

The commands declare variables with types. By now, I have all in place to start:

- Declare the types of some declarations of variables and functions
- Control that types are the right ones, ie, an integer variable should not be assigned to an string expression
- Start to emit commands and expression

As in other projects, I want to add access to native .NET types, libraries and objects. I should research how many features of Go I could implement in this approarch. An example: the implementation of Go structures, and their original semantic in C#. I could implement channels (years ago, I implemented channels in C# and AjSharp).

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

July 17, 2014

ScalaSharp Simple Scala In C# (1) The Idea

Filed under: .NET, C Sharp, Open Source Projects, Programming Languages, Scala, ScalaSharp — ajlopez @ 6:32 pm

I wrote something in this blog, but it’s time to introduce it to society. Last year a few months ago, I started the project:

https://github.com/ajlopez/ScalaSharp

A simple interpreter for a simplified Scala, C #. I do not intend to implement all Scala as a compiler for CLR. NET. My intention is to explore an alternative implementation of interpreter, adopting only some “light” features of Scala.

Without being a Scala programmer (or amateur or professional) my impression is that Scala as a language and library, is something like "too much", a bit overwhelming. People adopts Scala because it is more flexible, less need "boilerplate" etc.. But it seems like Scala went too far. Having programmed in Java and C#, I think the latter, adding a few things (delegates years, lambdas, extension methods, "generics" in the VM, and recently, in asynchronous input/output, async, await), has managed to stay healthy, without major complications. See how easy it has been to implement LINQ with evaluation "lazy" and others.

Instead, Scala seems to me like it’s too many things, all together. From what I saw, type inference is a feature that is not so necessary in the real projects. And yet I can not accept that the list of operators in Scala occupies more than one web page. But hey, that’s the current state of things. People Scala is working on a new implementation, called Dotty. But if you are just doing after years of language definition, I have to declare: "too little, too late."

That does not preclude recognizing that Scala has great triumphs, non-trivial applications like Play Framework and Akka. Akka is a very interesting project. I think we don’t need so many new language features (just the ones I mentioned for C #). We need more libraries that solve some issues (such as the implementation of an asynchronous HTTP message distribution server or distributed processing among actors).

To experiment with a simplified Scala, and to deliberately practice TDD (Test-Driven Development) in C #, I started the project ScalaSharp.

It is a challenge for me, because it is not a simple interpreter. I want to implement an AST (Abstract Syntax Tree) that detects inconsistencies between types. In my previous experiences, as RubySharp, PythonSharp, and Mass language, it was only necessary to assemble a tree expressions / commands and evaluate it. Now I’ll have to check that the types are correct. For example, one variable that is typed, and that their assignments, the expression to be evaluated and assigned for the type of the variable. You can see some of the work in progress:

https://github.com/ajlopez/ScalaSharp/tree/master/Src/ScalaSharp.Core/Ast

where there is a node tree with types, versus commands and expressions to be executed and evalauted:

https://github.com/ajlopez/ScalaSharp/tree/master/Src/ScalaSharp.Core/Commands
https://github.com/ajlopez/ScalaSharp/tree/master/Src/ScalaSharp.Core/Expressions

Upcoming topics: implementation details, control types in the AST transformation commands / expressions. The good thing to do with TDD is that it allows me to go implemented the solution as I define use cases.

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

July 16, 2014

RubySharp, implementing Ruby in C# (3)

Filed under: .NET, C Sharp, Open Source Projects, Programming Languages, Ruby, RubySharp — ajlopez @ 4:43 pm

Previous Post

In RubySharp, we can define new functions (methods of the current object), and invoke them. There are some built-in functions in C#:

Every function should implement the interface:

public interface IFunction
{
    object Apply(DynamicObject self, Context context, IList<object> values);
}

On apply, each function receives the object (self), the context for variables (locals, arguments, closure….), and a list of already evaluated arguments.

An example, the implemention of puts:

public class PutsFunction : IFunction
{
    private TextWriter writer;

    public PutsFunction(TextWriter writer)
    {
        this.writer = writer;
    }

    public object Apply(DynamicObject self, Context context, IList<object> values)
    {
        foreach (var value in values)
            this.writer.WriteLine(value);

        return null;
    }
}

It receives the object to which it is a method, the context, and a list of arguments. Each of the arguments was evaluated. The implementation simply sends the arguments to a TextWriter, one argument per line. The TextWriter is provided when the function object is created (in the Machine object, that represents the current running environment). This injected facilites the test of the function, example:

[TestMethod]
public void PutsTwoIntegers()
{
    StringWriter writer = new StringWriter();
    PutsFunction function = new PutsFunction(writer);

    Assert.IsNull(function.Apply(null, null, new object[] { 123, 456 }));

    Assert.AreEqual("123\r\n456\r\n", writer.ToString());
}

The above code was born using the TDD workflow.

Let’s see another built-in function, require:

public class RequireFunction : IFunction
{
    private Machine machine;

    public RequireFunction(Machine machine)
    {
        this.machine = machine;
    }

    public object Apply(DynamicObject self, Context context, IList<object> values)
    {
        string filename = (string)values[0];
        return this.machine.RequireFile(filename);
    }
}

This time, the job of loading a file is delegated to the Machin object, injected in the constructor.

And finally, let’s review the code of a defined function (defined in RubySharp):

public class DefinedFunction : IFunction
{
    private IExpression body;
    private IList<string> parameters;
    private Context context;

    public DefinedFunction(IExpression body, IList<string> parameters, Context context)
    {
        this.body = body;
        this.context = context;
        this.parameters = parameters;
    }

    public object Apply(DynamicObject self, Context context, IList<object> values)
    {
        Context newcontext = new Context(self, this.context);

        int k = 0;
        int cv = values.Count;

        foreach (var parameter in this.parameters) 
        {
            newcontext.SetLocalValue(parameter, values[k]);
            k++;
        }

        return this.body.Evaluate(newcontext);
    }
}

In this case, a new contexts is built, containing the new arguments to the defined function, but with the original closure, injected in the constructor. The context provided in the Apply is not used.

Next topics: some command implementations, the Machine object, the REPL, etc…

Stay tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

Older Posts »

The Shocking Blue Green Theme. Create a free website or blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 69 other followers