Angel \”Java\” Lopez on Blog

December 1, 2016

Building A Blockchain (15)

Filed under: Bitcoin, C Sharp, Ethereum, FinTech, JavaScript, NodeJs, Open Source Projects — ajlopez @ 10:28 am

Previous Post
Next Post

One interesting issue to solve, in my blockchains implementations (C#, JavaScript/NodeJS), is the node intercommunication. The nodes should interchange data, like new blocks, new transactions, status messages…. And, for a running node, it should discover other nodes in the network to be its peers. The process is named: peer discovery.

I want to start to write down some ideas:

– Each node has a node id, and a network id, so other nodes can check that info to accept the node as its peer or not

– A node could have configured a hardcoded list of initial nodes, to use as peers

– But it could have another list: a list of special nodes that knows other nodes in the network. This nodes are not peers of the first node. They are helpers, node registries, that knows other peers in the network, that can be used as peer by a new node in the network. Usually, this lists is not a list of IPs, but of machines by name, in a DNS controlled by the blockchain network infrastructure.

When a new node starts to run, it communicate its existence to this list of peer registry nodes, and actively query them for initial peers.

Each node has a number of maximum peers to use and connect. When one of these connections drops, or it is not suitable for be a peer node, the node tries other known peers, or ask new peers to the registry servers.

One way to ensure a good distribution of connection, is identify the known nodes in zones (maybe, node id modulus a low number). When a node of zone 2 needs peers, the registry servers sends to it peers of zones 1 and 3. In this way, the node start to be connected with more and more peers. But avoiding to know ALL the peers in the network: only some peers of its adjacent zones. This is for security reasons: if the FULL list of peers is not general available, it’s better to avoid global network attacks.

As usual, I should design all these with TDD, guided by simplicity and initial use cases.

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

 

November 5, 2016

New Month’s Resolutions: November 2016

Filed under: Blockchain, C Sharp, Ethereum, JavaScript, Open Source Projects, Solidity — ajlopez @ 9:09 pm

End of year is coming. Time to write my new month’s resolutions. And my past month’s ones.

– Improve CrysSharp [complete] see repo
– Improve SharpGo [complete] see repo
– Improve BlockchainSharp [complete] see repo
– Improve SimpleBlockchain [complete] see repo
– Continue Solidity Compiler [complete] see repo
– Continue ChineseP [pending]
– Write about implementing a block chain [pending]
– Write about simplicity and implementing a two-way peg [complete] see post

Additionally

– I gave a talk about Connecting Blockchains [complete] see slides
– Example source code, WIP, PegSharp [complete] see repo

My new month’s resolutions

– Improve CrysSharp
– Improve GoSharp 
– Improve BlockchainSharp
– Improve SimpleBlockchain
– Continue Solidity Compiler
– Continue ChineseP
– Continue PegSharp

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

October 6, 2016

New Month’s Resolutions: October 2016

Filed under: Blockchain, C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 10:57 am

Intensive month at my daily job, preparing a milestone at @RSKSmart. My past month’s resolutions:

– Improve CrysSharp [complete] see repo
– Improve BlockchainSharp [pending]
– Improve SimpleBlockchain [pending]
– Continue Solidity Compiler [complete] see repo
– Continue ChineseP [pending]

I also worked on:

– Improve SharpGo [complete] see repo

My new month’s resolutions:

– Improve CrysSharp
– Improve GoSharp 
– Improve BlockchainSharp
– Improve SimpleBlockchain
– Continue Solidity Compiler
– Continue ChineseP
– Write about implementing a block chain
– Write about simplicity and implementing a two-way peg

Stay tuned!

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

September 5, 2016

New Month’s Resolutions: September 2016

Filed under: C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 11:32 am

Another intensive month at work, coding for @RSKsmart, time to write down the new month’s resolutions. But first, let review the past month ones:

– Improve CrysSharp [complete] see repo
– Improve CrysJS [pending]
– Improve BlockchainSharp [pending]
– Improve SimpleBlockchain [pending]
– Start Solidity Compiler [complete] see repo

I also worked on:

– Improve SimpleDSL [complete] see repo
– Start ChineseP, chinese practice in NodeJS [complete] see repo

My new month’s resolutions:

– Improve CrysSharp
– Improve BlockchainSharp
– Improve SimpleBlockchain
– Continue Solidity Compiler
– Continue ChineseP

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

August 8, 2016

New Month’s Resolutions: August 2016

July was an intensive month in my software development job. But I had time to work on personal projects. Review of my past month resolutions:

– Improve WangTiles [pending]
– Improve WangTilesJS [pending]
– Improve CrysSharp [complete] see repo
– Improve CrysJS [complete] see repo
– Improve SimpleForth [pending]
– Improve BlockchainSharp [complete] see repo
– Improve SimpleBlockchain [complete] see repo

I also worked on:

– Start ChinesP, chinese practice web site [complete] see repo
– Update SimpleLists [complete] see repo

My new month’s resolutions

– Improve CrysSharp
– Improve CrysJS
– Improve BlockchainSharp
– Improve SimpleBlockchain
– Start Solidity Compiler

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

July 4, 2016

New Month’s Resolutions: July 2016

Filed under: Blockchain, C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 11:24 am

It’s time to write down my new month’s resolutions, and to review the past month’s ones.

– Improve WangTiles [pending]
– Improve WangTilesJS [complete] see repo
– Improve CrysSharp  [complete] see repo
– Improve GoSharp [pending]
– Improve SimpleForth [complete] see repo
– Improve BlockchainSharp [complete] see repo
– Improve SimpleBlockchain [complete] see repo

Additionally, I worked on:

– Improve CrysJS [complete] see repo

My new resolutions:

– Improve WangTiles
– Improve WangTilesJS
– Improve CrysSharp
– Improve CrysJS
– Improve SimpleForth
– Improve BlockchainSharp
– Improve SimpleBlockchain

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

June 8, 2016

New Month’s Resolutions: June 2016

Filed under: Blockchain, C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 9:01 am

Let’s review my previous new month’s resolutions:

– Improve WangTiles [pending]
– Improve WangTilesJS [complete] see repo
– Improve CrysSharp [complete] see repo
– Improve BlockchainSharp [complete] see repo
– Improve SimpleBlockchain [complete] see repo
– Prepare and give a Bitcoin related talk [complete]

I also worked on:

– Improve SimpleForth [complete] see repo

My new month’s resolutions:

– Improve WangTiles
– Improve WangTilesJS
– Improve CrysSharp
– Improve GoSharp 
– Improve SimpleForth
– Improve BlockchainSharp
– Improve SimpleBlockchain

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

May 30, 2016

Building A Blockchain (11)

Previous Post
Next Post

I refactored the blockchain implementation in my personal JavaScript/NodeJS project:

https://github.com/ajlopez/SimpleBlockchain

In the previous post I showed some tests I wrote using TDD (Test-Driven Development) workflow. Now, I want to show the current implementation, that is an evolution from the past week one.

The implementation is at:

https://github.com/ajlopez/SimpleBlockchain/blob/master/lib/blockchains.js

There is a JavaScript “class”, implemented as a function:

function Blockchain(block) {
    var self = this;
    var blocks = [block];
    var blockstore = stores.blockstore();
    blockstore.save(block);
    
    this.bestBlock = function () { return blocks[blocks.length - 1]; }

In my new implementation, I’m using a block store. This object was able to save a block, and retrieve a block by hash, or retrieve the blocks with same number. But now, I added the retrieve by parent hash: that is, given a block hash, blockstore can retrieve all its know children. Given such functionality, the blockchain implementation evolved to use that feature. In this way, I don’t need to have a tree of blocks: given a block, I can retrieve all the tree of its descendants, from the blockstore.

Notice that the state of the blockchain has a naive implementation: only a JavaScript array, where the index coincides with the block number, starting with genesis block having 0 as number. My plan is to refactor this implementation, to support thousands or millions of blocks, using a block store based on disk. But now, using this naive implementation, I could explore the behavior of the node application.

The exposed method to add a block is:

this.add = function (block) {
    if (blockstore.hasBlockHash(block.hash))
        return;
        
    blockstore.save(block);
    
    if (getUnknownAncestor(block) != null)
        return;
    
    tryAdd(block);
}

If the block is know, then it implies it was already processed, then return.

If not, the block is saved in the in-memory store, and the first unknown ancestor hash is calculated. Maybe, the store has not ALL the ancestor chain up to genesis block. In this case, I cannot process the block.

If the block has a chain of ancestor that connects it to the genesis block, then I try to add the block to the blockchain:

function tryAdd(block) {
    if (isBestBlockChild(block))
        blocks.push(block);
    else if (isBetterBestBlock(block))
        tryFork(block);
        
    tryChildren(block);
}

It it is a direct child of the best block, it is added to the block (remember, a naive implementation, a simple array). If not, but it is a better block (it has a higher number), a fork to the block is performed. In any case, the children of the block are processed: maybe, the new block is the missing link to new candidate chains based on previously known blocks that were disconnected from genesis until the arrival of the new block.

These are the predicates I wrote:

function isBestBlockChild(block) {
    var bblock = self.bestBlock();
    
    return bblock.hash === block.parentHash && bblock.number === block.number - 1;
}

function isBetterBestBlock(block) {
    var bblock = self.bestBlock();
    
    return bblock.number < block.number;
}

This is the process of the children:

function tryChildren(block) {
    var children = blockstore.getChildren(block.hash);

    for (var n in children)
        tryAdd(children[n]);
}

Note: again, it is a naive implementation, that implies a recursion using tryAdd, that calls tryChildren. I could refactor to avoid recursion, a pending task.

Given a block, this function calculates the upward chain to its first ancestor included in the current blockchain, and then change the blockchain to have that chain:

function tryFork(block) {
    var newbranch = [block];
    var parentHash = block.parentHash;
    
    while (parentHash && blockstore.hasBlockHash(parentHash)) {
        var parent = blockstore.getByHash(parentHash);
        
        if (parent.hash === blocks[parent.number].hash)
            return switchToFork(newbranch);
            
        newbranch.push(parent);
        
        parentHash = parent.parentHash;
    }
}

The find of the first unknow ancestor of a block:

function getUnknownAncestor(block) {
    var parentHash = block.parentHash;

    while (parentHash && blockstore.hasBlockHash(parentHash)) {
        var parent = blockstore.getByHash(parentHash);
        
        if (parent.hash === blocks[parent.number].hash)
            return null;
        
        parentHash = parent.parentHash;
    }
    
    return parentHash;
}

The change of the blockchain to a new fork:

function switchToFork(newbranch) {
    for (var n = newbranch.length; n-- > 0😉 {
        var block = newbranch[n];
        blocks[block.number] = block;
    }
}

All the test passed. As I mentioned, my plan is to improve the implementation. But, as usual in my TDD workflow, I prefer to give baby steps, make it works, and only then, make it right and make it fast. I had good experiences using this way of writing code, obtaining simple and solid implementations, having all the test to help me to change any internal implementation, without pain.

Encouraged by this result, I also started to refactor my blockchain implementation in my personal C# project, too:

https://github.com/ajlopez/BlockchainSharp

I started to use a block store with GetByParentHash into my BlockChain code:

https://github.com/ajlopez/BlockchainSharp/blob/master/Src/BlockchainSharp/Core/BlockChain.cs

I wrote about my previous implementation in this post.

Stay tuned!

Angel “Java” Lopez

@ajlopez

May 9, 2016

Building A Blockchain (8)

Previous Post
Next Post

In previous post, I wrote about the virtual machine I’m implementing in my personal blockchain project, in C#, using TDD (Test-Driven Development) workflow:

https://github.com/ajlopez/SimpleBlockchain

I added a simple bytecode compiler, in order to simplify some tests. The class is not needed for production code, but it is included in the core project:

public class BytecodeCompiler
{
    private IList<byte> bytes = new List<byte>();

    public void Stop()
    {
        this.Compile(Bytecodes.Stop);
    }

    public void Add()
    {
        this.Compile(Bytecodes.Add);
    }

    public void Subtract()
    {
        this.Compile(Bytecodes.Subtract);
    }
    
    // more methods
    
    public byte[] ToBytes()
    {
        return this.bytes.ToArray();
    }
}

It collects a series of bytecodes, with optional arguments. There is a method to return a byte array with the compiled program. A sample test:

[TestMethod]
public void LessThan()
{
    BytecodeCompiler compiler = new BytecodeCompiler();

    compiler.Push(2);
    compiler.Push(2);
    compiler.LessThan();
    compiler.Push(0);
    compiler.Push(1);
    compiler.LessThan();
    compiler.Push(1);
    compiler.Push(0);
    compiler.LessThan();

    Machine machine = new Machine();

    machine.Execute(compiler.ToBytes());

    var stack = machine.Stack;

    Assert.IsNotNull(stack);
    Assert.AreEqual(3, stack.Size);
    Assert.AreEqual(DataWord.Zero, stack.ElementAt(2));
    Assert.AreEqual(DataWord.Zero, stack.ElementAt(1));
    Assert.AreEqual(DataWord.One, stack.ElementAt(0));
}

I also wrote a simple line compiler, named SimpleCompiler. Now, I can use it in my machine tests:

[TestMethod]
public void IsZeroUsingSimpleCompiler()
{
    string program = "push 2\n" +
        "iszero\n" +
        "push 0\n" +
        "iszero";

    SimpleCompiler compiler = new SimpleCompiler(program);

    Machine machine = new Machine();

    machine.Execute(compiler.Compile());

    var stack = machine.Stack;

    Assert.IsNotNull(stack);
    Assert.AreEqual(2, stack.Size);
    Assert.AreEqual(DataWord.Zero, stack.ElementAt(1));
    Assert.AreEqual(DataWord.One, stack.ElementAt(0));
}

These are auxiliary classes, but they simplify the writing of new tests. It is part of my simplification strategy: after some initial and long tests, try to simplify the writing of the newer ones.

In the next post, a surprise: a new blockchain project, but using NodeJS, JavaScript.

Stay tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

May 8, 2016

New Month’s Resolutions: May 2016

Filed under: C Sharp, JavaScript, NodeJs, Open Source Projects, Wang Tiles — ajlopez @ 5:27 pm

Time to write my new month’s resolutions, and review the previous month’s ones:

– Improve WangTiles [complete] see repo
– Improve BlockchainSharp [complete] see repo
– Start Blockchain in JavaScript [complete] see repo
– Work on EthSharp [pending]
– Improve SimpleGA [pending]
– Improve AjGenesisNode-Express [pending]
– Work on CrysJS [pending]
– Work on CrysSharp [pending]

Additionally, I worked on:

– Improve SimpleForth [complete] see repo
– Start WangTilesJS [complete] see repo

My new month’s resolutions are:

– Improve WangTiles
– Improve WangTilesJS
– Improve CrysSharp
– Improve BlockchainSharp
– Improve SimpleBlockchain
– Prepare and give a Bitcoin related talk

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

« Newer PostsOlder Posts »

Create a free website or blog at WordPress.com.