AjLisp in Ruby (1) Structure, Classes and Tests

Next Post

I’m learning and practicing Ruby writing my AjLisp interpreter, as usual (early this year, I ported it to Javascript). TDD is my friend: write a test, run in red, code, run in green, refactor, and so on. My code, work in progress, is at:

https://github.com/ajlopez/AjLispRb

Initially, I wrote a single file, following the great and simple example:

http://kanemar.com/2006/03/04/screencast-of-test-driven-development-with-ruby-part-1-a-simple-example/

Notice that Ruby has a ‘test/unit’ package, already in place, ready to use, after installation. After some research, I spitted the file in production code, and test code. I want to build a gem (Ruby package), so I studied the tutorial:

http://guides.rubygems.org/

I have pending readings:

http://speakerdeck.com/u/pat/p/cut-polish-a-guide-to-crafting-gems
http://blog.thepete.net/2010/11/creating-and-publishing-your-first-ruby.html

My code is not a gem, yet. But it have some gem structure:

The lib contains a single file:

require 'ajlisp/list.rb'
require 'ajlisp/named_atom.rb'
require 'ajlisp/context.rb'
require 'ajlisp/string_source.rb'
require 'ajlisp/token.rb'
require 'ajlisp/lexer.rb'
require 'ajlisp/parser.rb'
require 'ajlisp/primitive.rb'
require 'ajlisp/primitive_first.rb'
require 'ajlisp/primitive_rest.rb'
require 'ajlisp/primitive_cons.rb'
require 'ajlisp/primitive_list.rb'
require 'ajlisp/primitive_closure.rb'
require 'ajlisp/fprimitive.rb'
require 'ajlisp/fprimitive_quote.rb'
require 'ajlisp/fprimitive_lambda.rb'
require 'ajlisp/fprimitive_flambda.rb'
require 'ajlisp/fprimitive_let.rb'
require 'ajlisp/fprimitive_closure.rb'
require 'ajlisp/fprimitive_define.rb'
require 'ajlisp/primitive_add.rb'
module AjLisp
@context = Context.new
@context.setValue "quote", FPrimitiveQuote.instance
@context.setValue "first", PrimitiveFirst.instance
@context.setValue "rest", PrimitiveRest.instance
@context.setValue "cons", PrimitiveCons.instance
@context.setValue "list", PrimitiveList.instance
@context.setValue "lambda", FPrimitiveLambda.instance
@context.setValue "flambda", FPrimitiveFLambda.instance
@context.setValue "let", FPrimitiveLet.instance
@context.setValue "define", FPrimitiveDefine.instance
@context.setValue "+", PrimitiveAdd.instance
def self.context
    return @context
end
def self.evaluate(context, item)
    if item.is_a? List or item.is_a? NamedAtom
        return item.evaluate(context)
    end

    return item
end
end

I wrote some primitives (normal forms, and special forms: the latter don’t evaluate their parameters before apply, like quote or define). Notice that the additional files are in a subfolder ajlisp inside lib, why? Because when this code is installed as a gem, all lib file code could be include with require(‘thenameofrubyfile’). If I put other files than ajlisp.rb, i.e. list.rb or atom.rb, those file names could be required, and maybe a name collision with other gems could occur. Then, the recommended practice (see other gem code in your Ruby installation folder) is to put additional files below lib folder.

In the test folder there is a top test.rb file that includes other test files:

require 'ajlisp'
require 'test/unit'
require "test_list.rb"
require "test_named_atom.rb"
require "test_context.rb"
require "test_string_source.rb"
require "test_token.rb"
require "test_lexer.rb"
require "test_parser.rb"
require "test_primitive_first.rb"
require "test_primitive_rest.rb"
require "test_primitive_cons.rb"
require "test_primitive_list.rb"
require "test_primitive_closure.rb"
require "test_primitive_add.rb"
require "test_fprimitive_quote.rb"
require "test_fprimitive_lambda.rb"
require "test_fprimitive_let.rb"
require "test_fprimitive_closure.rb"
require "test_fprimitive_flambda.rb"
require "test_fprimitive_define.rb"
require "test_evaluate"

so you can run the tests from command line:

ruby –Ilib;test test\test.rb

If you have Windows, the runtest.cmd file contains this line.

Some test code:

require 'ajlisp'
require 'test/unit'
class TestList < Test::Unit::TestCase
#... 
    def test_create_with_first
        list = AjLisp::List.new("foo")
        assert_equal("foo", list.first)
        assert_nil(list.rest)
    end

    def test_create_with_first_and_rest
        rest = AjLisp::List.new("bar")
        list = AjLisp::List.new("foo", rest)
        assert_equal("foo", list.first)
        assert_not_nil(list.rest)
        assert_equal("bar", list.rest.first)
        assert_nil(list.rest.rest)
    end

    def test_create_from_array
        list = AjLisp::List.make [1, "a", "foo"]
        assert_not_nil list
        assert_equal 1, list.first
        assert_equal "a", list.rest.first
        assert_equal "foo", list.rest.rest.first
        assert_nil list.rest.rest.rest
    end
#..
end

Every list is an object of this class, list.rb:

module AjLisp
class List
    attr_reader :first
    attr_reader :rest

    def initialize(first=nil, rest=nil)
        @first = first
        @rest = rest
    end

    def evaluate(context)
        form = AjLisp::evaluate(context, @first)
        form.evaluate(context, self)
    end

    def self.make(array)
        if array and array.length > 0
            first = array.shift

            if first.is_a? Array
                first = make(first)
            elsif first.is_a? Symbol
                first = NamedAtom.new first.to_s
            end

            return List.new first, make(array)
        end 

        return nil
    end
end
end

The accessors first and rest are read-only. Thanks to the untyped nature of Ruby (like in Javascript) the implementation of this interpreter is straightforward, or without much code ceremony, at least.

In my new tests, I included the code into the AjLisp module, so I don’t need to write AjLisp:: prefix before referencing a class:

require 'ajlisp'
require 'test/unit'
module AjLisp
class TestLexer < Test::Unit::TestCase
    def test_get_atom_token
        source = StringSource.new "atom"
        lexer = Lexer.new source
        token = lexer.nextToken

        assert_not_nil token
        assert_equal "atom", token.value
        assert_equal TokenType::ATOM, token.type
        assert_nil lexer.nextToken
    end
    def test_get_atom_token_with_spaces
        source = StringSource.new " atom "
        lexer = Lexer.new source
        token = lexer.nextToken

        assert_not_nil token
        assert_equal "atom", token.value
        assert_equal TokenType::ATOM, token.type
        assert_nil lexer.nextToken
    end
#...
end

Next topics: some implementation details, primitives vs fprimitives, context (nested environments with name/values), lambda and closures, lexer and parser.

Next steps: complete primitives (let, letrec, definef, do, if…), macro (mlambda, definem, macro expansion…)

Keep tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez