Angel \”Java\” Lopez on Blog

May 23, 2015

SparkSharp, Spark in C# (1) First Ideas

Filed under: .NET, C Sharp, Open Source Projects, Spark, SparkSharp — ajlopez @ 11:20 pm

Next Post

In these days, I visited Apache Spark project:

https://spark.apache.org/

And started to think about implementing some of its ideas in C#.

The original project has Datasets, that can be consumed, item by item, and processed by methods like map and reduce. A dataset can consume a text file, local files or distributed ones. The jobs to run over datasets, applying transformations, can be launched in many distributed nodes (I should review the consolidation of results).

I started a new C# project:

https://github.com/ajlopez/SparkSharp

To me, it is important to start with small steps, using TDD (Test-Driven Development) workflow. So, in my first commits, I wrote datasets that implement IEnumerable. They have methods like Map, Reduce, Split, Take, Skip. Those methods were implemented writing the tests that express the expected API and behavior.

A dataset can be a simple wrapper of any IEnumerable, or it can read a text file, reading lines.

All these datasets are local, reside in the same machine. My idea is to implement a dataset wrapper, to expose the dataset content to other machines, and write a client wrapper that runs in each machine. The client wrapper looks like a regular dataset, but when the client program needs the next item of the dataset, that item come from the remote original machine.

The remote dataset gives the next item to any client. Each item is delivered only to ONE client. So, the items can be consumed and processed by n remote clients, without having an item processed twice.

To implement such pair server/client, I should implement serialization/deserialization of an arbitrary type T. I will use my previous work in AjErl and Aktores to have such feature. Using TDD, I could assert the expected behavior of the serializaction/deserialization process. If in the future, I have a better idea for such process, like using an external robust open source serialization library, all the TDD tests will help me to make the switch.

But, baby steps. Next steps: improve current local datasets, maybe add a new variant of dataset, and write keyed datasets, created using MapToKey method (to implement)

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

May 19, 2015

AjErl Implementing Erlang in C# (2) Expressions

Filed under: .NET, AjErl, C Sharp, Erlang, Open Source Projects — ajlopez @ 9:58 am

Previous Post

Some time ago I write this project, an Erlang implementation as interpreter using C #. Let review today the implementation of expressions. An expression is evaluated in a context. For example, the variable X, is an expression that is evaluated in a context that tells how that variable is bound to a value or it is free.

All expressions are written using the workflow TDD (Test-Driven Development). After some expressions, this interface emerge in a refactor, and now all expressions implement it:

public interface IExpression
{
    object Evaluate(Context context, bool withvars = false);

    bool HasVariable();
}

The key method is Evaluate. It can allow or not that some variables are still unbound after the evaluate. There is a boolean method HasVariable that indicates the existence of variables in the expression. The simplest expression is a constant expression:

public class ConstantExpression : IExpression
{
    private object value;

    public ConstantExpression(object value)
    {
        this.value = value;
    }

    public object Value { get { return this.value; } }

    public object Evaluate(Context context, bool withvars = false)
    {
        return this.value;
    }

    public bool HasVariable()
    {
        return false;
    }
}

The Evaluate method does not use the provided context, it is only returning the original constant value, defined in the constructor. The expressions are grouped in a tree of expressions that is recursively evaluated.

An expression that evaluates a variable:

public class VariableExpression : IExpression
{
    private Variable variable;

    public VariableExpression(Variable variable)
    {
        this.variable = variable;
    }

    public Variable Variable { get { return this.variable; } }

    public object Evaluate(Context context, bool withvars = false)
    {
        if (!context.HasValue(this.variable.Name))
            if (!withvars)
                throw new Exception(string.Format("variable '{0}' is unbound", this.variable.Name));
            else
                return this.variable;

        return context.GetValue(this.variable.Name);
    }

    public bool HasVariable()
    {
        return true;
    }
}

This time, the context is used. If the variable is not bound to a value, it can return the unbound value, or raise an exception, depending on a boolean withValues. Some times, Erlang expects and needs that an expression be complete evaluated, having no unbound variables.

This code was implemented following the workflow of TDD. Some tests as examples (but it is only the final result, the important thing is the workflow, the dynamic of the programming activity). For a constant expression:

[TestMethod]
public void CreateSimpleConstantExpression()
{
    ConstantExpression expr = new ConstantExpression(10);

    Assert.AreEqual(10, expr.Value);
}

[TestMethod]
public void EvaluateSimpleConstantExpression()
{
    ConstantExpression expr = new ConstantExpression(10);

    Assert.AreEqual(10, expr.Evaluate(null));
}

For a variable expression:

[TestMethod]
public void CreateSimpleVariableExpression()
{
    Variable variable = new Variable("X");
    VariableExpression expression = new VariableExpression(variable);

    Assert.AreEqual(variable, expression.Variable);
}

[TestMethod]
public void EvaluateVariableExpression()
{
    Variable variable = new Variable("X");
    Context context = new Context();
    context.SetValue("X", 1);
    VariableExpression expression = new VariableExpression(variable);

    Assert.AreEqual(1, expression.Evaluate(context));
}

[TestMethod]
public void EvaluateUndefinedVariableExpression()
{
    Variable variable = new Variable("X");
    Context context = new Context();
    VariableExpression expression = new VariableExpression(variable);

    Assert.AreEqual(variable, expression.Evaluate(context, true));
}

[TestMethod]
public void RaiseIfEvaluateUndefinedVariableExpression()
{
    Variable variable = new Variable("X");
    Context context = new Context();
    VariableExpression expression = new VariableExpression(variable);

    try
    {
        expression.Evaluate(context, false);
        Assert.Fail();
    }
    catch (Exception ex)
    {
        Assert.AreEqual("variable 'X' is unbound", ex.Message);
    }
}

Upcoming topics: explore other expression implementations, data structure implementations for variables, maps, and more…

Stay tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

May 5, 2015

New Month’s Resolutions: May 2015

Filed under: .NET, C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 9:41 am

It’s the time for review my April’s resolutions and write the new ones:

– Write posts about JavaScript and Artificial Intelligence  [pending]
– Improve ClojJS [complete] see repo
– Add npm modules support to ClojJS [pending]
– Improve Chip8 emulator [pending]
– Improve BScript [complete] see repo

Additionally, I worked on:

– Improve AcquarellaJS, first operators in match dynamic object [complete] see repo
– Improve AjErl, refactor type utilities [complete] see repo
– Improve OStore, first operators in match [complete] see repo
– Improve SimpleLists, intersect operation [complete] see repo
– Improve MeteorSamples, using Windows version 1.1 [complete] see repo
– Participate in Google Code Jam, pre-round [complete] see repo
– Liqueed Project coding [complete] see repo
– Record Liqueed Project team programming session using TDD [complete] see post/video
– Write posts about Liqueed Project [complete] read English posts read Spanish posts

The new month’s resolutions are:

– Improve ClojJS
– Add NPM support to ClojJS
– Write posts about JavaScript and Artificial Intelligence
– Give a talk about Meteor
– Prepare a talk about Clojure or ClojureScript
– Improve BScript
– Improve AjErl, distributed features
– Improve Liqueed Project, kudos features

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

April 8, 2015

New Month’s Resolutions: April 2015

Filed under: C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 8:33 pm

It’s the time for review my March’s resolutions and write the new ones:

– Improve ClojSharp [complete] see repo
– Improve ScalaSharp [complete] see repo
– Write posts about JavaScript and Artificial Intelligence [pending]
– First published version of ClojJS [partial] see repo
– Add npm modules support to ClojJS [pending]
– First published version of BScript [partial] see repo
– Improve Chip8 emulator [pending]
– Give a full-day course on NodeJS [complete]

Additionally, I worked on:

– Update SimpleAsync, removing setImmediate [complete] see repo
– Update SimpleBus to use SimpleUnit [complete] see repo
– Improve AcquarellaJS detecting comments [complete] see repo
– Improve SimpleArgs using flags [complete] see repo
– Improve DValues [complete] see repo
– Update Complexo tests to use SimpleUnit [complete] see repo
– Project Liqueed [complete] see repo

My resolutions for the new month:

– Write posts about JavaScript and Artificial Intelligence
– Improve ClojJS
– Add npm modules support to ClojJS
– Improve Chip8 emulator
– Improve BScript

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

March 3, 2015

New Month’s Resolutions: March 2015

Filed under: C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 4:30 pm

Time to review the past month’s resolutions, and to write the new ones

– Complete first version ClojJS [partial] see repo
– Complete first version BScript [partial] see repo
– Improve ClojSharp [complete] see repo
– Improve ScalaSharp [complete] see repo
– Complete first version Chip8 emulator [partial] see repo
– Write more posts about JavaScript and Artificial Intelligence [pending]

Although ClojJS has many pieces in place (like resolving requires using a clojjs.json file), there are many forms to be defined in clojure.cljs. I should release a first version with the current status, and then add support for detecting node_modules with clojjs.json definitions, build for browser version, etc.

Additionally, I worked on:

– Start SimpleDSL node.js/javascript module [complete] see repo
– Improve AjScript [complete] see repo
– Improve AcquarellaJS [complete] see repo
– Start DValues [complete] see repo

Then, this new month’s resolutions are:

– Improve ClojSharp
– Improve ScalaSharp
– Write posts about JavaScript and Artificial Intelligence
– First published version of ClojJS
– Add npm modules support to ClojJS
– First published version of BScript
– Improve Chip8 emulator
– Give a full-day course on NodeJS

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

February 3, 2015

New Month’s Resolutions: February 2015

Filed under: C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 5:14 pm

Again, time to review the past month’s resolutions, and to write down the new ones.

– Continue add distributed features to AjErl, Erlang-like interpreter in C# [pending]
– Improve ClojSharp [complete] repo
– Improve ScalaSharp [complete] repo
– Improve Chip8 emulator [complete] repo
– Code generation with AjGenesis for Node [pending]
– Improve Basic Script Language for C# [complete] repo
– Write JavaScript and Artificial Intelligence [complete] post 1 post 2 post 3

Additionally, I was working on:

– Start ClojJS, ClojureScript to JavaScript compiler and runtime [complete] repo
– Start DValues, data values store for internet of things [complete] repo
– Work on Liqueed project [complete] repo
– Improve AjScript, JavaScript-like interpreter in C# [complete] repo

This new month’s resolutions:

– Complete first version ClojJS
– Complete first version BScript
– Improve ClojSharp
– Improve ScalaSharp
– Complete first version Chip8 emulator
– Write more posts about JavaScript and Artificial Intelligence

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

January 2, 2015

New Month’s Resolutions: January 2015

A new year begins. Time to review past month’s resolutions:

– Add distributed features to AjErl, Erlang-like interpreter in C# [partial] see repo
– Improve ClojSharp [complete] see repo
– Improve SimpleLisp [complete] see repo
– Code generation with AjGenesis for Node [pending]
– Start Basic Script Language for C# [complete] see repo

Additionally, I was working on:

– Improve ScalaSharp [complete] see repo
– First operations in Chip8 emulator [complete] see repo
– Write Clojure samples (Ring, Compojure) [complete] see repo
– Work on Liqueed project [complete] see repo
– Improve AjScript [complete] see repo

This new month’s resolutions:

– Continue add distributed features to AjErl, Erlang-like interpreter in C#
– Improve ClojSharp
– Improve ScalaSharp
– Improve Chip8 emulator
– Code generation with AjGenesis for Node
– Improve Basic Script Language for C#
– Write JavaScript and Artificial Intelligence posts

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

December 14, 2014

Aktores: Implementing Actor Model In C# (2) A Simple Sample

Filed under: .NET, Akka, Aktores, C Sharp, Open Source Projects — ajlopez @ 8:18 am

Previous Post

Let’s review a simple sample that creates two actors, and sends messages to them. The code is at:

https://github.com/ajlopez/Aktores/tree/master/Samples/PingPong

As usual, this is “work in progress”, and could be change in the future. I’m following TDD (Test-Driven Development) workflow, so some API calls could be change when I add new use cases (as distributed samples).

In this solution, we have an actor:

using Aktores.Core;

public class PingActor : Actor
{
    private int messagecount;

    public int MessageCount { get { return this.messagecount; } }

    public override void Receive(object message)
    {
        messagecount++;
        this.Sender.Tell("ping", this.Self);
    }
}

An actor inherits from abtract class Actor. The method to implement is Receive. This method receives a message (sent by other actor, or by other object in the system). Internally, Aktores sends only one message to our instance, we don’t worry about concurrency. The above actor, when receives a message, increments a counters and sends another message to the original sender. This sender is referenced by this.Sender, DURING the process of THIS message. This property can change in the process of another message.

We have another similar actor:

using Aktores.Core;

public class PongActor : Actor
{
    private int messagecount;

    public int MessageCount { get { return this.messagecount; } }

    public override void Receive(object message)
    {
        messagecount++;
        this.Sender.Tell("pong", this.Self);
    }
}

The main program creates an actor systems, creates two actor instances, one of type Ping and the other of type Pong, and sends to them many message to process:

public static void Main(string[] args)
{
    ActorSystem system = new ActorSystem(2);
    var pingactor = new PingActor();
    var pongactor = new PongActor();
    
    var ping = system.ActorOf(pingactor);
    var pong = system.ActorOf(pongactor);

    for (int k = 0; k < 10; k++)
    {
        ping.Tell("pong", pong);
        pong.Tell("ping", ping);
    }

    // ....
}

There is additional code to show the message counters. The parameter 2 in the constructor of ActorSystem indicates the number of internal threads to use to process message. I should review if such info should be specified here. Two threads are enough becase we have only two actor instances, but we could have more actors.

Stay tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

November 4, 2014

New Month’s Resolutions: November 2014

Filed under: .NET, AjTalk, Aktores, C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 5:25 pm

Time to review past month’s resolutions:

– Give a talk Aktores intro [complete] see repo see presentation
– Distributed AjErl [pending]
– More Express samples in AjTalkJs [complete] see repo
– Explore neural networks in JavaScript [complete] see repo
– Explore genetic algorithms in JavaScript [partial] see repo
– Explore other artificial intelligence topics in JavaScript [partial]
– More Distributed Aktores [pending]
– Start Distributed Smalltalk in AjTalkJs [complete] see repo

My new month’s resolutions:

– Give a talk about JavaScript and Artificial Intelligence
– Neural network in JavaScript example
– Genetic Algorithms in JavaScript example
– More Code Generation using AjGenesis for NodeJs
– Basic Script Language for C#
– Improve SimpleScript for JavaScript

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

October 7, 2014

New Month’s Resolutions: October 2014

Filed under: .NET, C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 5:50 pm

Time to write my technical new month’s resolutions. But first, review of past month’s ones:

– Distributed Aktores [complete] see repo
– Distributed AjErl [pending]
– Web support and example for RuScript [partial] see repo
– Complete Mochy [partial] see repo
– Improve RSharp [complete] see repo
– Work with NodeJs on Deep Learning, Internet of Things, Artificial Intelligence [pending]
– Work on Code Generation using AjGenesis for Node [complete] see main repo see Express repo see Sinatra repo see Laravel repo
– Work on Liqueed Project [complete] see repo

Additional, I worked on

– Improve AjScript [complete] see repo
– Refactor BasicScript tests to use SimpleUnit [complete] see repo
– Create SimpleColls, simple collection functions in JavaScript, alone or a la Linq [complete] see repo
– Refactor SimpleStore tests to use SimpleUnit [complete] see repo
– Improve AjTalkJs, Smalltalk in JavaScript, support for super, Express 4 first sample [complete] see repo
– Refactor MProc tests to use SimpleUnit [complete] see repo
– Refactor AjLogoJs tests to use SimpleUnit [complete] see repo
– Refactor SimpleKeeper tests to use SimpleUnit [complete] see repo
– Refactor SimpleTags tests to use SimpleUnit [complete] see repo
– Improve SimpleUnit, publish new version [complete] see repo
– Refactor SimplePipes tests to use SimpleUnit, and publish new version [complete] see repo

My new month’s resolutions:

– Give a talk Aktores intro
– Distributed AjErl
– More Express samples in AjTalkJs
– Explore neural networks in JavaScript
– Explore genetic algorithms in JavaScript
– Explore other artificial intelligence topics in JavaScript
– More Distributed Aktores
– Start Distributed Smalltalk in AjTalkJs

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

« Newer PostsOlder Posts »

The Shocking Blue Green Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 72 other followers