Angel \”Java\” Lopez on Blog

May 23, 2015

SparkSharp, Spark in C# (1) First Ideas

Filed under: .NET, C Sharp, Open Source Projects, Spark, SparkSharp — ajlopez @ 11:20 pm

In these days, I visited Apache Spark project:

https://spark.apache.org/

And started to think about implementing some of its ideas in C#.

The original project has Datasets, that can be consumed, item by item, and processed by methods like map and reduce. A dataset can consume a text file, local files or distributed ones. The jobs to run over datasets, applying transformations, can be launched in many distributed nodes (I should review the consolidation of results).

I started a new C# project:

https://github.com/ajlopez/SparkSharp

To me, it is important to start with small steps, using TDD (Test-Driven Development) workflow. So, in my first commits, I wrote datasets that implement IEnumerable<T>. They have methods like Map, Reduce, Split, Take, Skip. Those methods were implemented writing the tests that express the expected API and behavior.

A dataset can be a simple wrapper of any IEnumerable<T>, or it can read a text file, reading lines.

All these datasets are local, reside in the same machine. My idea is to implement a dataset wrapper, to expose the dataset content to other machines, and write a client wrapper that runs in each machine. The client wrapper looks like a regular dataset, but when the client program needs the next item of the dataset, that item come from the remote original machine.

The remote dataset gives the next item to any client. Each item is delivered only to ONE client. So, the items can be consumed and processed by n remote clients, without having an item processed twice.

To implement such pair server/client, I should implement serialization/deserialization of an arbitrary type T. I will use my previous work in AjErl and Aktores to have such feature. Using TDD, I could assert the expected behavior of the serializaction/deserialization process. If in the future, I have a better idea for such process, like using an external robust open source serialization library, all the TDD tests will help me to make the switch.

But, baby steps. Next steps: improve current local datasets, maybe add a new variant of dataset, and write keyed datasets, created using MapToKey method (to implement)

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

May 19, 2015

AjErl Implementing Erlang in C# (2) Expressions

Filed under: .NET, AjErl, C Sharp, Erlang, Open Source Projects — ajlopez @ 9:58 am

Previous Post

Some time ago I write this project, an Erlang implementation as interpreter using C #. Let review today the implementation of expressions. An expression is evaluated in a context. For example, the variable X, is an expression that is evaluated in a context that tells how that variable is bound to a value or it is free.

All expressions are written using the workflow TDD (Test-Driven Development). After some expressions, this interface emerge in a refactor, and now all expressions implement it:

public interface IExpression
{
    object Evaluate(Context context, bool withvars = false);

    bool HasVariable();
}

The key method is Evaluate. It can allow or not that some variables are still unbound after the evaluate. There is a boolean method HasVariable that indicates the existence of variables in the expression. The simplest expression is a constant expression:

public class ConstantExpression : IExpression
{
    private object value;

    public ConstantExpression(object value)
    {
        this.value = value;
    }

    public object Value { get { return this.value; } }

    public object Evaluate(Context context, bool withvars = false)
    {
        return this.value;
    }

    public bool HasVariable()
    {
        return false;
    }
}

The Evaluate method does not use the provided context, it is only returning the original constant value, defined in the constructor. The expressions are grouped in a tree of expressions that is recursively evaluated.

An expression that evaluates a variable:

public class VariableExpression : IExpression
{
    private Variable variable;

    public VariableExpression(Variable variable)
    {
        this.variable = variable;
    }

    public Variable Variable { get { return this.variable; } }

    public object Evaluate(Context context, bool withvars = false)
    {
        if (!context.HasValue(this.variable.Name))
            if (!withvars)
                throw new Exception(string.Format("variable '{0}' is unbound", this.variable.Name));
            else
                return this.variable;

        return context.GetValue(this.variable.Name);
    }

    public bool HasVariable()
    {
        return true;
    }
}

This time, the context is used. If the variable is not bound to a value, it can return the unbound value, or raise an exception, depending on a boolean withValues. Some times, Erlang expects and needs that an expression be complete evaluated, having no unbound variables.

This code was implemented following the workflow of TDD. Some tests as examples (but it is only the final result, the important thing is the workflow, the dynamic of the programming activity). For a constant expression:

[TestMethod]
public void CreateSimpleConstantExpression()
{
    ConstantExpression expr = new ConstantExpression(10);

    Assert.AreEqual(10, expr.Value);
}

[TestMethod]
public void EvaluateSimpleConstantExpression()
{
    ConstantExpression expr = new ConstantExpression(10);

    Assert.AreEqual(10, expr.Evaluate(null));
}

For a variable expression:

[TestMethod]
public void CreateSimpleVariableExpression()
{
    Variable variable = new Variable("X");
    VariableExpression expression = new VariableExpression(variable);

    Assert.AreEqual(variable, expression.Variable);
}

[TestMethod]
public void EvaluateVariableExpression()
{
    Variable variable = new Variable("X");
    Context context = new Context();
    context.SetValue("X", 1);
    VariableExpression expression = new VariableExpression(variable);

    Assert.AreEqual(1, expression.Evaluate(context));
}

[TestMethod]
public void EvaluateUndefinedVariableExpression()
{
    Variable variable = new Variable("X");
    Context context = new Context();
    VariableExpression expression = new VariableExpression(variable);

    Assert.AreEqual(variable, expression.Evaluate(context, true));
}

[TestMethod]
public void RaiseIfEvaluateUndefinedVariableExpression()
{
    Variable variable = new Variable("X");
    Context context = new Context();
    VariableExpression expression = new VariableExpression(variable);

    try
    {
        expression.Evaluate(context, false);
        Assert.Fail();
    }
    catch (Exception ex)
    {
        Assert.AreEqual("variable 'X' is unbound", ex.Message);
    }
}

Upcoming topics: explore other expression implementations, data structure implementations for variables, maps, and more…

Stay tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

May 5, 2015

New Month’s Resolutions: May 2015

Filed under: .NET, C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 9:41 am

It’s the time for review my April’s resolutions and write the new ones:

– Write posts about JavaScript and Artificial Intelligence  [pending]
– Improve ClojJS [complete] see repo
– Add npm modules support to ClojJS [pending]
– Improve Chip8 emulator [pending]
– Improve BScript [complete] see repo

Additionally, I worked on:

– Improve AcquarellaJS, first operators in match dynamic object [complete] see repo
– Improve AjErl, refactor type utilities [complete] see repo
– Improve OStore, first operators in match [complete] see repo
– Improve SimpleLists, intersect operation [complete] see repo
– Improve MeteorSamples, using Windows version 1.1 [complete] see repo
– Participate in Google Code Jam, pre-round [complete] see repo
– Liqueed Project coding [complete] see repo
– Record Liqueed Project team programming session using TDD [complete] see post/video
– Write posts about Liqueed Project [complete] read English posts read Spanish posts

The new month’s resolutions are:

– Improve ClojJS
– Add NPM support to ClojJS
– Write posts about JavaScript and Artificial Intelligence
– Give a talk about Meteor
– Prepare a talk about Clojure or ClojureScript
– Improve BScript
– Improve AjErl, distributed features
– Improve Liqueed Project, kudos features

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

January 2, 2015

New Month’s Resolutions: January 2015

A new year begins. Time to review past month’s resolutions:

– Add distributed features to AjErl, Erlang-like interpreter in C# [partial] see repo
– Improve ClojSharp [complete] see repo
– Improve SimpleLisp [complete] see repo
– Code generation with AjGenesis for Node [pending]
– Start Basic Script Language for C# [complete] see repo

Additionally, I was working on:

– Improve ScalaSharp [complete] see repo
– First operations in Chip8 emulator [complete] see repo
– Write Clojure samples (Ring, Compojure) [complete] see repo
– Work on Liqueed project [complete] see repo
– Improve AjScript [complete] see repo

This new month’s resolutions:

– Continue add distributed features to AjErl, Erlang-like interpreter in C#
– Improve ClojSharp
– Improve ScalaSharp
– Improve Chip8 emulator
– Code generation with AjGenesis for Node
– Improve Basic Script Language for C#
– Write JavaScript and Artificial Intelligence posts

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

December 14, 2014

Aktores: Implementing Actor Model In C# (2) A Simple Sample

Filed under: .NET, Akka, Aktores, C Sharp, Open Source Projects — ajlopez @ 8:18 am

Previous Post

Let’s review a simple sample that creates two actors, and sends messages to them. The code is at:

https://github.com/ajlopez/Aktores/tree/master/Samples/PingPong

As usual, this is “work in progress”, and could be change in the future. I’m following TDD (Test-Driven Development) workflow, so some API calls could be change when I add new use cases (as distributed samples).

In this solution, we have an actor:

using Aktores.Core;

public class PingActor : Actor
{
    private int messagecount;

    public int MessageCount { get { return this.messagecount; } }

    public override void Receive(object message)
    {
        messagecount++;
        this.Sender.Tell("ping", this.Self);
    }
}

An actor inherits from abtract class Actor. The method to implement is Receive. This method receives a message (sent by other actor, or by other object in the system). Internally, Aktores sends only one message to our instance, we don’t worry about concurrency. The above actor, when receives a message, increments a counters and sends another message to the original sender. This sender is referenced by this.Sender, DURING the process of THIS message. This property can change in the process of another message.

We have another similar actor:

using Aktores.Core;

public class PongActor : Actor
{
    private int messagecount;

    public int MessageCount { get { return this.messagecount; } }

    public override void Receive(object message)
    {
        messagecount++;
        this.Sender.Tell("pong", this.Self);
    }
}

The main program creates an actor systems, creates two actor instances, one of type Ping and the other of type Pong, and sends to them many message to process:

public static void Main(string[] args)
{
    ActorSystem system = new ActorSystem(2);
    var pingactor = new PingActor();
    var pongactor = new PongActor();
    
    var ping = system.ActorOf(pingactor);
    var pong = system.ActorOf(pongactor);

    for (int k = 0; k < 10; k++)
    {
        ping.Tell("pong", pong);
        pong.Tell("ping", ping);
    }

    // ....
}

There is additional code to show the message counters. The parameter 2 in the constructor of ActorSystem indicates the number of internal threads to use to process message. I should review if such info should be specified here. Two threads are enough becase we have only two actor instances, but we could have more actors.

Stay tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

December 4, 2014

New Month’s Resolutions: December 2014

It’s time to review past month’s resolutions:

– Give a talk about JavaScript and Artificial Intelligence [complete] see repo see talk read Spanish post
– Neural network in JavaScript example [complete] see repo
– Genetic Algorithms in JavaScript example [complete] see repo
– More Code Generation using AjGenesis for NodeJs [pending]
– Basic Script Language for C# [pending]
– Improve SimpleScript for JavaScript [complete] see repo

Also, I was working on:

– Improve SimpleLisp, Lisp to JavaScript compiler [complete] see repo
– Improve RuScript, Ruby interpreter in JavaScript [complete] see repo
– Improve RustScript, Rust-like interpreter in JavaScript [complete] see repo
– Create SimpleDT, simple decision tree in JavaScript [complete] see repo
– Improve ClojSharp, Clojure-like interpreter in C# [complete] see repo
– Backgammon evaluation page [complete] see repo

This new month’s resolutions:

– Add distributed features to AjElr, Erlang-like interpreter in C#
– Improve ClojSharp
– Improve SimpleLisp
– Code generation with AjGenesis for Node
– Start Basic Script Language for C#

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

November 4, 2014

New Month’s Resolutions: November 2014

Filed under: .NET, AjTalk, Aktores, C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 5:25 pm

Time to review past month’s resolutions:

– Give a talk Aktores intro [complete] see repo see presentation
– Distributed AjErl [pending]
– More Express samples in AjTalkJs [complete] see repo
– Explore neural networks in JavaScript [complete] see repo
– Explore genetic algorithms in JavaScript [partial] see repo
– Explore other artificial intelligence topics in JavaScript [partial]
– More Distributed Aktores [pending]
– Start Distributed Smalltalk in AjTalkJs [complete] see repo

My new month’s resolutions:

– Give a talk about JavaScript and Artificial Intelligence
– Neural network in JavaScript example
– Genetic Algorithms in JavaScript example
– More Code Generation using AjGenesis for NodeJs
– Basic Script Language for C#
– Improve SimpleScript for JavaScript

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

October 7, 2014

New Month’s Resolutions: October 2014

Filed under: .NET, C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 5:50 pm

Time to write my technical new month’s resolutions. But first, review of past month’s ones:

– Distributed Aktores [complete] see repo
– Distributed AjErl [pending]
– Web support and example for RuScript [partial] see repo
– Complete Mochy [partial] see repo
– Improve RSharp [complete] see repo
– Work with NodeJs on Deep Learning, Internet of Things, Artificial Intelligence [pending]
– Work on Code Generation using AjGenesis for Node [complete] see main repo see Express repo see Sinatra repo see Laravel repo
– Work on Liqueed Project [complete] see repo

Additional, I worked on

– Improve AjScript [complete] see repo
– Refactor BasicScript tests to use SimpleUnit [complete] see repo
– Create SimpleColls, simple collection functions in JavaScript, alone or a la Linq [complete] see repo
– Refactor SimpleStore tests to use SimpleUnit [complete] see repo
– Improve AjTalkJs, Smalltalk in JavaScript, support for super, Express 4 first sample [complete] see repo
– Refactor MProc tests to use SimpleUnit [complete] see repo
– Refactor AjLogoJs tests to use SimpleUnit [complete] see repo
– Refactor SimpleKeeper tests to use SimpleUnit [complete] see repo
– Refactor SimpleTags tests to use SimpleUnit [complete] see repo
– Improve SimpleUnit, publish new version [complete] see repo
– Refactor SimplePipes tests to use SimpleUnit, and publish new version [complete] see repo

My new month’s resolutions:

– Give a talk Aktores intro
– Distributed AjErl
– More Express samples in AjTalkJs
– Explore neural networks in JavaScript
– Explore genetic algorithms in JavaScript
– Explore other artificial intelligence topics in JavaScript
– More Distributed Aktores
– Start Distributed Smalltalk in AjTalkJs

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

September 13, 2014

New Month’s Resolutions: September 2014

Filed under: .NET, C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 5:34 pm

A bit late, time to review my August’s Resolutions:

– Write jQuery mock to be used in console tests/Node.js [complete] see repo
– Write simple functions in JavaScript for array manipulation [complete] see repo
– Continue with Liqueed project [complete] see repo
– Start with Distributed Aktores [partial] see repo
– Startwith Distributed AjErl [pending]

Additionally, I worked on:

– Give an intro talk about Express [complete] see repo see presentation see samples
– Improve AjScript [complete] see repo
– Improve RuScript, Ruby interpreter in JavaScript [complete] see repo
– Refactor SimplePipes [complete] see repo
– Generate Node.js/Express server code from RAML [complete] see repo
– Create Dynamic Site from JSON to Express [complete] see repo
– Improve RustScript, Rust interpreter in JavaScript [complete] see repo
– Create Mochy, test framework a la Mocha [complete] see repo
– Create RSharp, R language interpreter in C# [complete] see repo
– Refactor (and simplify) SimpleRules, rule engine in JavaScript [complete] see repo
– First samples for SimpleMule [complete] see repo
– Improve output for SimpleUnit [complete] see repo

My new month’s resolutions:

– Distributed Aktores
– Distributed AjErl
– Web support and example for RuScript
– Complete Mochy
– Improve RSharp
– Work with NodeJs on Deep Learning, Internet of Things, Artificial Intelligence
– Work on Code Generation using AjGenesis for Node
– Work on Liqueed Project

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

August 10, 2014

New Month’s Resolutions: August 2014

Filed under: .NET, C Sharp, JavaScript, Open Source Projects — ajlopez @ 5:59 pm

Time to write new resolutions, and to review July 2014 ones:

– Improve AjGenesisNode code generation for Express [complete] see repo
– Improve AjGenesisNode code generation for Sinatra [partial] see repo
– First AjGenesisNode code generation for Laravel [pending]
– Module loading, export review in Mass Language [pending]
– Continue with SharpBus [complete] see repo
– Continue with Dynamic Site [pending]
– Give a talk about Node.js/Express [complete]
– Improve my personal site code [complete] see repo
– Write TDD Post [pending]
– Write C# Posts [complete] see posts
– Distributed use using Aktores [pending]
– Review, refactor and continue RustScript implementation [complete] see repo

Additionally, I worked on:

– Started SimpleList [complete] see repo
– Started SimpleJQuery [complete] see repo
– Improve SimpleMule [complete] see repo
– Started SharpGD, Graph Database in C# [complete] see repo
– Improve SimpleBus [complete] see repo
– Improve SharpGo [complete] see repo
– Improve ExpressSamples [complete] see repo
– Minor improvements in ScaScript [complete] see repo
– Improve ScalaSharp [complete] see repo
– Write Friends sample for JavaScriptSamples [complete] see repo
– Write Angular examples [complete] see repo
– Add remove method to OStore [complete] see repo
– Collaborate with Liqueed project [complete] see repo

New resolutons for August 2014:

– Write jQuery mock to be used in console tests/Node.js
– Write simple functions in JavaScript for array manipulation
– Continue with Liqueed project
– Start with Distributed Aktores
– Startwith Distributed AjErl

Also, I worked for thread count is 3.

Keep tuned!

Angel “Java” Lopez
http://www.ajlopez.com

Older Posts »

The Shocking Blue Green Theme. Create a free website or blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 68 other followers