Angel \”Java\” Lopez on Blog

May 5, 2015

New Month’s Resolutions: May 2015

Filed under: .NET, C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 9:41 am

It’s the time for review my April’s resolutions and write the new ones:

– Write posts about JavaScript and Artificial Intelligence  [pending]
– Improve ClojJS [complete] see repo
– Add npm modules support to ClojJS [pending]
– Improve Chip8 emulator [pending]
– Improve BScript [complete] see repo

Additionally, I worked on:

– Improve AcquarellaJS, first operators in match dynamic object [complete] see repo
– Improve AjErl, refactor type utilities [complete] see repo
– Improve OStore, first operators in match [complete] see repo
– Improve SimpleLists, intersect operation [complete] see repo
– Improve MeteorSamples, using Windows version 1.1 [complete] see repo
– Participate in Google Code Jam, pre-round [complete] see repo
– Liqueed Project coding [complete] see repo
– Record Liqueed Project team programming session using TDD [complete] see post/video
– Write posts about Liqueed Project [complete] read English posts read Spanish posts

The new month’s resolutions are:

– Improve ClojJS
– Add NPM support to ClojJS
– Write posts about JavaScript and Artificial Intelligence
– Give a talk about Meteor
– Prepare a talk about Clojure or ClojureScript
– Improve BScript
– Improve AjErl, distributed features
– Improve Liqueed Project, kudos features

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

April 25, 2015

Liqueed Project (6)

Filed under: Express, JavaScript, Liqueed, NodeJs, Test-Driven Development — ajlopez @ 4:26 pm

Previous Post

Let review another tests from test/personapi.js in project:

https://github.com/liquid-co-ops/liqueed

There are API entrypoints that are called using PUT or POST. They receive data in the body of the sent message. Using Express middleware, that message (formatted as JSON) is analized, parsed and converted into a JavaScript object. That object resides in the body property of the received request parameter. As the API uses that field, we should provide it in the test. An example:

exports['login person'] = function (test) {
    test.async();
    
    var request = {
        body: {
            username: persons[1].username,
            password: persons[1].username
        }
    };

    var response = {
        send: function (model) {
            test.ok(model);
            test.equal(model.id, persons[1].id);
            test.equal(model.name, persons[1].name);
            test.equal(model.username, persons[1].username);
            
            test.done();
        }
    };
    
    controller.loginPerson(request, response);
};

As in the previous example, we use the send function in the response object, to check the result.

Another example, sending not only the body but also sending parameters:

exports['change password first person'] = function (test) {
    test.async();
    
    var request = {
		params : {
			id : persons[0].id.toString()
		},
        body: {
            password: 'new' + persons[0].username
        }
    };

    async()
    .then(function (data, next) {
        var response = {
            send: function (model) {
                test.ok(model);
                test.strictEqual(model, true);
                
                next(null, null);
            }
        };
        
        controller.updatePassword(request, response);
    })
    .then(function (data, next) {
        var request = {
            body: {
                username: persons[0].username,
                password: 'new' + persons[0].username
            }
        };

        var response = {
            send: function (model) {
                test.ok(model);
                test.equal(model.id, persons[0].id);
                test.equal(model.name, persons[0].name);
                test.equal(model.username, persons[0].username);
                
                test.done();
            }
        };
        
        controller.loginPerson(request, response);
    })
    .run();
};
 

Next posts: MVC controller actions tests, tests launching the HTTP server, tests using a DSL (Domain-Specific Language)

Stay tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

April 20, 2015

Liqueed Project (5)

Filed under: Express, Liqueed, NodeJs, Test-Driven Development — ajlopez @ 5:38 pm

Previous Post  
Next Post

Let review two more tests from test/personapi.js. The second test is actually:

exports['get persons'] = function (test) {
    test.async();
    
    var request = {};
    var response = {
        send: function (model) {
            test.ok(model);
            test.ok(Array.isArray(model));
            test.ok(model.length);
            test.ok(model[0].id);
            test.ok(model[0].name);
            test.done();
        }
    };
    
    controller.list(request, response);
};

After building the domain model in memory (the first test taks), now we can test a controller action method, named getPersons. It requires to receive two arguments, request and response object. In the above code, two stubs were built and used. The response object has a send function that  it should be invoked to pass the tests. It receives the model to send to the client, as a JavaScript object (in the real environment, the object is serialized to JSON via Express response send method).

The third test is:

exports['get first person'] = function (test) {
    test.async();
    
    var request = {
        params: {
            id: persons[0].id.toString()
        }
    };

    var response = {
        send: function (model) {
            test.ok(model);
            test.equal(model.id, persons[0].id);
            test.equal(model.name, persons[0].name);
            test.done();
        }
    };
    
    controller.get(request, response);
};

The action needs something from request object, an aditional argument having the person primary key to be retrieved. There is a send method in request object, to test the result.

The tests are not using HTTP nor GET method. They are directly testing the controller code. Indeed, that code was written AFTER the tests, and its first execution was from test launch, without the need of launch a web server.

Next posts: other tests for API methods, sending objects, and tests for MVC controllers

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

April 15, 2015

Liqueed Project (4)

Filed under: Express, JavaScript, Liqueed, NodeJs, Test-Driven Development — ajlopez @ 8:42 pm

Previous Post
Next Post

The project:

https://github.com/liquid-co-ops/liqueed

was built using TDD (Test-Driven Development) worflow. In the previous post I showed a logic service example, implemented using tests. Today, I want to show that the MVC controllers was also coded using TDD. The controllers are JavaScript modules that expose functions as actions to be routed by Express. The actions receive and return JSON objects. They are the basis for the exposed API, to be consumed by the clients (actually, only one client, a simple single page application). For example, controllers\personapi.js starts declaring:

'use strict';

var service = require('../services/person');

It consumes the person service module.

There are associated test at test\personapi.js, its initial imports:

'use strict';

var controller = require('../controllers/personapi');

var loaddata = require('../utils/loaddata');
var db = require('../utils/db');
var async = require('simpleasync');

In the previous post, I mentioned that the test granularity I prefer for JavaScript is the module, not the function. So, all the tests in the module are execute, in declaration order. The first test is in charge of inicialization the domain model:

var persons;

exports['clear and load data'] = function (test) {
    var personService = require('../services/person');

    test.async();
    
    async()
    .then(function (data, next) { db.clear(next); })
    .then(function (data, next) { loaddata(next); })
    .then(function (data, next) { personService.getPersons(next); })
    .then(function (data, next) {
        persons = data;
        test.ok(persons);
        test.ok(persons.length);
        test.done();
    })
    .run();
};

The new thing to understand is the use of the module simpleasync, pointed by the async variable. I wrote the module to chain  functions. Each function receives two arguments: data, the success result of the previous executed function in chain, or the initial value triggered in the run chain function. And next, a callback to be invoked by the function, to execute the rest of the chain. The callback receives two arguments: err and data. So it can be used as the callback of other functions. If err is not null, the next functions in chain is not executed and the function defined in the chain fail method is run (this option is not used in the above code). In the above example  personService.getPersons(next) invokes the retrieve of the person list, using next as callback. The next chained function receives the person list in the data argument, and then, it saves it in a module variable, ready to be used by the rest of the tests.

It is not using a database. It using an in-memory domain model. That is the default “persistence”, and it is used in many of the defined tests. The initial domain model is loaded from testdata.json using the loaddata function:

{
    "projects": [
        {
            "name": "FaceHub",
            "periods": [
                { 
                    "name": "January 2014", 
                    "date": "2014-01-31", 
                    "amount": 100,
                    "assignments": [
                        { "from": "Alice", 
                            "to": "Bob", 
                            "amount": 50, 
                            "note": "Arrive earlier" },
                        { "from": "Alice", 
                            "to": "Charlie", 
                            "amount": 50 , 
                            "note": "Arrive earlier" },
                        { "from": "Bob", 
                            "to": "Alice", 
                            "amount": 60 , 
                            "note": "Arrive earlier" },
                        { "from": "Bob", 
                            "to": "Charlie", 
                            "amount": 40 , 
                            "note": "Arrive earlier" },
                        { "from": "Charlie", 
                            "to": "Alice", 
                            "amount": 35 , 
                            "note": "Arrive earlier" },
                        { "from": "Charlie", 
                            "to": "Bob", 
                            "amount": 65 , 
                            "note": "Arrive earlier" }
                    ]
                },
                { "name": "February 2014", 
                    "date": "2014-02-28", 
                    "amount": 100 }
            ],
            "team": [ "Alice", "Bob", "Charlie" ],
//....

The module personapi.js exports some functions to be used as actions:

module.exports = {
    list: list,
    get: get,
    getProjects: getProjects,
    loginPerson: loginPerson,
    getPendingShareProjects:getPendingShareProjects,
    updatePassword: updatePassword
}

Topics for the next posts: more API tests, routing of actions, persistence, etc.

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

April 8, 2015

New Month’s Resolutions: April 2015

Filed under: C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 8:33 pm

It’s the time for review my March’s resolutions and write the new ones:

– Improve ClojSharp [complete] see repo
– Improve ScalaSharp [complete] see repo
– Write posts about JavaScript and Artificial Intelligence [pending]
– First published version of ClojJS [partial] see repo
– Add npm modules support to ClojJS [pending]
– First published version of BScript [partial] see repo
– Improve Chip8 emulator [pending]
– Give a full-day course on NodeJS [complete]

Additionally, I worked on:

– Update SimpleAsync, removing setImmediate [complete] see repo
– Update SimpleBus to use SimpleUnit [complete] see repo
– Improve AcquarellaJS detecting comments [complete] see repo
– Improve SimpleArgs using flags [complete] see repo
– Improve DValues [complete] see repo
– Update Complexo tests to use SimpleUnit [complete] see repo
– Project Liqueed [complete] see repo

My resolutions for the new month:

– Write posts about JavaScript and Artificial Intelligence
– Improve ClojJS
– Add npm modules support to ClojJS
– Improve Chip8 emulator
– Improve BScript

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

March 18, 2015

Liqueed Project (3)

Filed under: Express, JavaScript, Liqueed, NodeJs, Test-Driven Development — ajlopez @ 5:24 pm

Previous Post
Next Post

I’m a big supporter of TDD (Test-Driven Development), and its workflow for software development. The software is created based on the tests that describe the expected behaviour, using the simplest possible implementation. For Liqueed Project, TDD was adopted during the creation of most of the code.

An example: there is a file, services\person.js, that is a JavaScript module. It manages the logic of the persons. A person can participate in zero, one or more projects, have sharepoints, vote, etc. And the system keeps the list of persons, retrieve a person by id, by username, … Part of the logic is in the service module. It was written using test/person.js for the tests of the logic. One test, one impementation, refactor, and so on. At the beginning of the test mode, there are some requires:

'use strict';

var service = require('../services/person');
var pservice = require('../services/project');
var async = require('simpleasync');

The services persons and project are used. simpleasync is a simple module I wrote to chaining callbacks. It was an interesting experience, and I’m doing “dog-fooding” in many projects.

Then, two module variables are declared, to be reused in the tests. They are the ids of some persons:

var annaid;
var lauraid;

To run the tests, I’m using simpleunit, another module I wrote, again, to practice JavaScript, TDD and simplicity. It is inspired by nodeunit, but it is simpler. simpleunit executes the module exported functions, providing a test object, something alike to assert in NodeJs.

The first tests exercises the creation of a person:

exports['add person'] = function (test) {
    test.async();

    service.addPerson({ name: 'Anna' }, function (err, result) {
        test.ok(!err);
        test.ok(result);
        annaid = result;
        test.done();
    });
};

It has a callback, so the function could end BEFORE the creation of the person. So, the test.async() tells to simpleunit to wait until the test.done() is reached. The above test only check the return of an id for the new person, and the value is saved in a module variable. This is something to discuss, but I found this way an easy one. The second tests depends on the first one. In C#, in other technologies, I wrote more isolated tests. But for me, in JavaScript, the level of isolation is the module, not the test function. In a module, a test function could depend on the execution of a previous one.

The second test:

exports['get person by id'] = function (test) {
    test.async();

    service.getPersonById(annaid, function (err, result) {
        test.ok(!err);
        test.ok(result);
        test.equal(result.name, 'Anna');
        test.equal(result.id, annaid);
        test.equal(result.username, 'anna');
        test.done();
    });
};

Using the id generated by the first test, the person is retrieved, and the data is checked.

This test series doesn’t use a predefined domain, and it does not assume a previous well-knoen status. And these tests are implemented using an in-memory domain. In the initial iterations, this in-memory domain was the only implementation. Only after the implementation of the first big use case, we added a domain persisted in MongoDB.

Stay tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

March 3, 2015

New Month’s Resolutions: March 2015

Filed under: C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 4:30 pm

Time to review the past month’s resolutions, and to write the new ones

– Complete first version ClojJS [partial] see repo
– Complete first version BScript [partial] see repo
– Improve ClojSharp [complete] see repo
– Improve ScalaSharp [complete] see repo
– Complete first version Chip8 emulator [partial] see repo
– Write more posts about JavaScript and Artificial Intelligence [pending]

Although ClojJS has many pieces in place (like resolving requires using a clojjs.json file), there are many forms to be defined in clojure.cljs. I should release a first version with the current status, and then add support for detecting node_modules with clojjs.json definitions, build for browser version, etc.

Additionally, I worked on:

– Start SimpleDSL node.js/javascript module [complete] see repo
– Improve AjScript [complete] see repo
– Improve AcquarellaJS [complete] see repo
– Start DValues [complete] see repo

Then, this new month’s resolutions are:

– Improve ClojSharp
– Improve ScalaSharp
– Write posts about JavaScript and Artificial Intelligence
– First published version of ClojJS
– Add npm modules support to ClojJS
– First published version of BScript
– Improve Chip8 emulator
– Give a full-day course on NodeJS

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

February 20, 2015

Liqueed Project (2)

Filed under: Express, JavaScript, Liqueed, NodeJs — ajlopez @ 8:07 pm

Previous Post
Next Post

Liqueed is a Node.js project, repository at

https://github.com/liquid-co-ops/liqueed

And it has a package.json file, the current content:

{
  "name": "liqueed",
  "version": "0.0.2alpha",
  "private": true,
  "scripts": {
    "start": "node ./bin/www",
    "test": "simpleunit ./test ./testserver"
  },
  "dependencies": {
    "express": "~4.2.0",
    "static-favicon": "~1.0.0",
    "morgan": "~1.0.0",
    "cookie-parser": "~1.0.1",
    "body-parser": "~1.0.0",
    "debug": "~0.7.4",
    "ejs": "~0.8.5",
    "express-ejs-layouts": "^1.1.0",
    "ostore": "0.0.4",
    "simplelists": "0.0.4",
    "simpleasync": "0.0.5",
    "mongodb": "~1.4.19",
    "bcrypt-nodejs": "0.0.3",
    "express-session": "~1.9.3"
  },
  "devDependencies": {
    "simpleunit": "0.0.5",
    "simplejquery": "0.0.1",
    "zombie": "2.5.1"
  }
}

It is a private application, so you cannot publish it to the public npm repository.

It is a typical Express application, the initial code was generated with the express-generator. The file bin\www contains the source code to launch the application. It uses express modules, like morgan, cookie-parser, body-parser, express-session, but there are others that were included to be used from the logic of application server:

– ostore: Simple JavaScript objects repository, in-memory, used in many test and in the initial version, implementing persistence in-memory, without the need for a database.

– mongodb: the current app uses MongoDB as NoSQL persistence

– simplelists: a simple utility library with list/array functions

– simpleasync: a library to chain asynchronous callsbacks.

– bcript-nodejs: encryption module written in JavaScript

In the tests (written using Test-Driven Design workflow), it uses:

– simpleunit: test library, based on internal use of assert, exporting the test functions.

There are new tests to exercise the Single Page Application of public\index.html, using:

– zombie: it visits a page as a browser

Topics for the next posts: application structure, written tests, exposed API, MVC admin pages, persistence, SPA client, etc.

Stay tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

February 9, 2015

Programando for Internet of Things

Usually, I practice programming every day, applying TDD (Test-Driven Development) workflow. There are non-public projects, but the majority of that practice is public, in my GitHub account. In my opinion, Node.js is a very interesting technology: ubiquos, powerful and simple. I can implement many ideas using Node.js/JavaScript, in a easier way than using other technologies (Java, .NET, Scala, Ruby, Python, Clojure, Smalltalk). The only price to pay: understand and use the JavaScript callbacks.

There are many ideas and implementation, in current applications, public projects, startups. An interesting topic is Internet of Things (OK, a “buzzword”). Many startups are fighting in that battle field. Maybe, many of them will be adquired by Google, Apple, Microsoft. These are interesting time. But this is a time for implement ideas, in open source projects. Then, an startup can leverage the open source world. I think that the important part of an startup is the execution, not the idea or the implementation..

Meanwhile, I want to implement some ideas in my public projects. This post is written to describe the landscape of ideas to implement:

– Collect informaiton from devices connected to the Internet. To have a data repository, the data is send by SDKs, one SDK per device type. Then, use the repository and the collected data in other projects.

– Apply Artificial Intelligence (OK, it is a wide term, but the better we have), to all the collected data, discovering patterns, using machine learning, deep learning. I wrote some JavaScript projects to explore such paths.

– Take decisions, execute actions, using expert systems or other systems. My first implementations, again, are in JavaScript/NodeJs. Having somethink like IFTTT but open source, consuming Internet of Things data.

– Run all these implementations as distributed applicacionts, not only horizontal scalability or cloud computing. Node.js, again, is an interesting vehicle to implement first experiments and applications. Only if needed, then switch to compiled languages and technologies.

I apologize my auto-reference to my projects, but it is the way to explain why I’m writing them.

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

February 6, 2015

Sending messages to Slack using JavaScript/Node.js

Filed under: JavaScript, NodeJs, Slack — ajlopez @ 12:42 pm

A year ago, Slack entered in my radar:

https://slack.com/

I read “Slack is a platform for team communication; everything in one place, instantly searcheable, available wherever you go”.

In general, I am somewhat skeptical to online tools like this, I prefer a simpler combination. For the above statement, I simply use gmail and mailing lists per project or team. The interesting story behind Slack founder:

http://www.wired.com/2014/08/the-most-fascinating-profile-youll-ever-read-about-a-guy-and-his-boring-startup/

My links about Slack at:

https://delicious.com/ajlopez/slack

Example:

https://medium.com/@slackhq/11-useful-tips-for-getting-the-most-of-slack-5dfb3d1af77

Past wednesday, at afternoon, I started to explore how to send programmatically a message to Slack, using their exposed API, using Node.js (I think Node is the simplest technology to use in this kind of experiments). More general info about the API at:

https://api.slack.com/

My current experiment code:

https://github.com/ajlopez/NodeSamples/tree/master/Slack

I searched in Google for some Node.js module to access Slack, my first result:

https://github.com/xoxco/node-slack

Initially I installed it using npm, in a working directory. My first try:

var domain = process.env.SLACK_DOMAIN;
var token = process.env.SLACK_TOKEN;

var Slack = require('node-slack');
var slack = new Slack(domain, token);

var channel = process.argv[2];
var message = process.argv[3];

console.log('Sending message', message, 'to channel', channel);

slack.send({
    text: message,
    channel: '#' + channel,
    username: username
});

You must set environment variables: SLACK_DOMAIN and SLACK_TOKEN. When you enter to a Slack group, the group has an associated domain. For example, I’m a member of https://squeakros.slack.com the group for Smalltalk Squek Rosario. Other group is https://nodejsar.slack.com, Node.js Argentina. The provided token correspond to a user AND an applicaction, it can be obtained from:

https://api.slack.com/web

The above program run, without errors, being invoked via

node run.js general “Hello, world”

(on Windows). The first argument after run.js is the target channel, without the initial #, and the second argument is the message text. All OK, but the message didn’t arrive to the channel. After a bit of research, I found the Pull Request:

https://github.com/xoxco/node-slack/pull/8

that it was not merged with the original source, yet. Some months ago, the Slack API changed. The suggested pull requests uses another access point, a web hook URL provided from:

https://your-domain.slack.com/services/new/incoming-webhook

I applied the suggestion I found in one comment, and I put directly the link to the new code, in my package.json:

{
  "name": "slack-demo",
  "private": true,
  "version": "0.0.1",
  "dependencies": {
    "node-slack": "git://github.com/Dahlgren/node-slack.git#hook-url"
  }
}

An now, my program use the web hook URL, provided via a variable environment:

var hookUrl = process.env.SLACK_WEBHOOK;
var username = process.env.SLACK_USER;

var Slack = require('node-slack');
//var slack = new Slack(domain, token);
var slack = new Slack(hookUrl);

var channel = process.argv[2];
var message = process.argv[3];

console.log('Sending message', message, 'to channel', channel);

slack.send({
    text: message,
    channel: '#' + channel,
    username: username
});

Now, it works! Topics to research: any way to use domain/token again? other modules?.

Stay tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

Older Posts »

The Shocking Blue Green Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 68 other followers