Angel \”Java\” Lopez on Blog

March 11, 2009

TDD and Code Kata: Writing a Lexer for AjRuby

Filed under: C Sharp, Programming Languages, Software Development — ajlopez @ 7:20 am

Usually, I write programs as code katas, to train me in some software development topics. During last months, I wrote many mini interpreters, using TDD-like approach, code coverage, and C#. Past week, I was working in the kickoff of yet another one, AjRuby. The starting point: the lexer.

A lexer is a program that analizes a source code text, and returns the tokens, the "words" that compose the program. In the case of AjRuby, I wrote a lexer that follows the Ruby language grammar. It’s not complete yet, but it’s working.

You can see my progress at my code katas Google code project:

The solution has a library project, and a test project:

It has a Token class, that represents each token:

public class Token { public string Value { get; set; } public TokenType TokenType { get; set; } }

The TokenType is an enum:

public enum TokenType { Name, Integer, Real, Boolean, String, Operator, Separator }

There is a LexerTest.cs class, containing tests like:

[TestMethod] public void ShouldProcessName() { Lexer lexer = new Lexer("name"); Token token; token = lexer.NextToken(); Assert.IsNotNull(token); Assert.AreEqual(TokenType.Name, token.TokenType); Assert.AreEqual("name", token.Value); token = lexer.NextToken(); Assert.IsNull(token); }

There is a test for every kind of token to be processed. Other example:

[TestMethod] public void ShouldProcessNameWithInitialUnderscore() { Lexer lexer = new Lexer("_name"); Token token; token = lexer.NextToken(); Assert.IsNotNull(token); Assert.AreEqual(TokenType.Name, token.TokenType); Assert.AreEqual("_name", token.Value); token = lexer.NextToken(); Assert.IsNull(token); }

I wrote the tests using TDD ideas: write the test, compile with error, fix the compiler issues, run the tests in red, fix them, get green status. I skipped most of the refactor steps. I guess the tested code could be improved, but by now, I’m trying to get a working lexer. The tests are in green state, good!

My goal is to have > 80% of code coverage, at any time during the project development:

It’s good to write something only for fun and learning. For me, it’s part of the continuous improvement every software developer should pursuit.

Angel "Java" Lopez

1 Comment »

  1. […] (sí, sí, hay AjRuby, pero todavía en inception… 😉 .. leer TDD and Code Kata: Writing a lexer for AjRuby) […]

    Pingback by ALT.NET Hispano VAN (reunión virtual): IronRuby - Angel "Java" Lopez — January 21, 2010 @ 11:33 am

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at

%d bloggers like this: