Angel \”Java\” Lopez on Blog

September 5, 2016

New Month’s Resolutions: September 2016

Filed under: C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 11:32 am

Another intensive month at work, coding for @RSKsmart, time to write down the new month’s resolutions. But first, let review the past month ones:

– Improve CrysSharp [complete] see repo
– Improve CrysJS [pending]
– Improve BlockchainSharp [pending]
– Improve SimpleBlockchain [pending]
– Start Solidity Compiler [complete] see repo

I also worked on:

– Improve SimpleDSL [complete] see repo
– Start ChineseP, chinese practice in NodeJS [complete] see repo

My new month’s resolutions:

– Improve CrysSharp
– Improve BlockchainSharp
– Improve SimpleBlockchain
– Continue Solidity Compiler
– Continue ChineseP

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

August 8, 2016

New Month’s Resolutions: August 2016

July was an intensive month in my software development job. But I had time to work on personal projects. Review of my past month resolutions:

– Improve WangTiles [pending]
– Improve WangTilesJS [pending]
– Improve CrysSharp [complete] see repo
– Improve CrysJS [complete] see repo
– Improve SimpleForth [pending]
– Improve BlockchainSharp [complete] see repo
– Improve SimpleBlockchain [complete] see repo

I also worked on:

– Start ChinesP, chinese practice web site [complete] see repo
– Update SimpleLists [complete] see repo

My new month’s resolutions

– Improve CrysSharp
– Improve CrysJS
– Improve BlockchainSharp
– Improve SimpleBlockchain
– Start Solidity Compiler

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

July 4, 2016

New Month’s Resolutions: July 2016

Filed under: Blockchain, C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 11:24 am

It’s time to write down my new month’s resolutions, and to review the past month’s ones.

– Improve WangTiles [pending]
– Improve WangTilesJS [complete] see repo
– Improve CrysSharp  [complete] see repo
– Improve GoSharp [pending]
– Improve SimpleForth [complete] see repo
– Improve BlockchainSharp [complete] see repo
– Improve SimpleBlockchain [complete] see repo

Additionally, I worked on:

– Improve CrysJS [complete] see repo

My new resolutions:

– Improve WangTiles
– Improve WangTilesJS
– Improve CrysSharp
– Improve CrysJS
– Improve SimpleForth
– Improve BlockchainSharp
– Improve SimpleBlockchain

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

June 8, 2016

New Month’s Resolutions: June 2016

Filed under: Blockchain, C Sharp, JavaScript, NodeJs, Open Source Projects — ajlopez @ 9:01 am

Let’s review my previous new month’s resolutions:

– Improve WangTiles [pending]
– Improve WangTilesJS [complete] see repo
– Improve CrysSharp [complete] see repo
– Improve BlockchainSharp [complete] see repo
– Improve SimpleBlockchain [complete] see repo
– Prepare and give a Bitcoin related talk [complete]

I also worked on:

– Improve SimpleForth [complete] see repo

My new month’s resolutions:

– Improve WangTiles
– Improve WangTilesJS
– Improve CrysSharp
– Improve GoSharp 
– Improve SimpleForth
– Improve BlockchainSharp
– Improve SimpleBlockchain

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

May 30, 2016

Building A Blockchain (11)

Previous Post
Next Post

I refactored the blockchain implementation in my personal JavaScript/NodeJS project:

https://github.com/ajlopez/SimpleBlockchain

In the previous post I showed some tests I wrote using TDD (Test-Driven Development) workflow. Now, I want to show the current implementation, that is an evolution from the past week one.

The implementation is at:

https://github.com/ajlopez/SimpleBlockchain/blob/master/lib/blockchains.js

There is a JavaScript “class”, implemented as a function:

function Blockchain(block) {
    var self = this;
    var blocks = [block];
    var blockstore = stores.blockstore();
    blockstore.save(block);
    
    this.bestBlock = function () { return blocks[blocks.length - 1]; }

In my new implementation, I’m using a block store. This object was able to save a block, and retrieve a block by hash, or retrieve the blocks with same number. But now, I added the retrieve by parent hash: that is, given a block hash, blockstore can retrieve all its know children. Given such functionality, the blockchain implementation evolved to use that feature. In this way, I don’t need to have a tree of blocks: given a block, I can retrieve all the tree of its descendants, from the blockstore.

Notice that the state of the blockchain has a naive implementation: only a JavaScript array, where the index coincides with the block number, starting with genesis block having 0 as number. My plan is to refactor this implementation, to support thousands or millions of blocks, using a block store based on disk. But now, using this naive implementation, I could explore the behavior of the node application.

The exposed method to add a block is:

this.add = function (block) {
    if (blockstore.hasBlockHash(block.hash))
        return;
        
    blockstore.save(block);
    
    if (getUnknownAncestor(block) != null)
        return;
    
    tryAdd(block);
}

If the block is know, then it implies it was already processed, then return.

If not, the block is saved in the in-memory store, and the first unknown ancestor hash is calculated. Maybe, the store has not ALL the ancestor chain up to genesis block. In this case, I cannot process the block.

If the block has a chain of ancestor that connects it to the genesis block, then I try to add the block to the blockchain:

function tryAdd(block) {
    if (isBestBlockChild(block))
        blocks.push(block);
    else if (isBetterBestBlock(block))
        tryFork(block);
        
    tryChildren(block);
}

It it is a direct child of the best block, it is added to the block (remember, a naive implementation, a simple array). If not, but it is a better block (it has a higher number), a fork to the block is performed. In any case, the children of the block are processed: maybe, the new block is the missing link to new candidate chains based on previously known blocks that were disconnected from genesis until the arrival of the new block.

These are the predicates I wrote:

function isBestBlockChild(block) {
    var bblock = self.bestBlock();
    
    return bblock.hash === block.parentHash && bblock.number === block.number - 1;
}

function isBetterBestBlock(block) {
    var bblock = self.bestBlock();
    
    return bblock.number < block.number;
}

This is the process of the children:

function tryChildren(block) {
    var children = blockstore.getChildren(block.hash);

    for (var n in children)
        tryAdd(children[n]);
}

Note: again, it is a naive implementation, that implies a recursion using tryAdd, that calls tryChildren. I could refactor to avoid recursion, a pending task.

Given a block, this function calculates the upward chain to its first ancestor included in the current blockchain, and then change the blockchain to have that chain:

function tryFork(block) {
    var newbranch = [block];
    var parentHash = block.parentHash;
    
    while (parentHash && blockstore.hasBlockHash(parentHash)) {
        var parent = blockstore.getByHash(parentHash);
        
        if (parent.hash === blocks[parent.number].hash)
            return switchToFork(newbranch);
            
        newbranch.push(parent);
        
        parentHash = parent.parentHash;
    }
}

The find of the first unknow ancestor of a block:

function getUnknownAncestor(block) {
    var parentHash = block.parentHash;

    while (parentHash && blockstore.hasBlockHash(parentHash)) {
        var parent = blockstore.getByHash(parentHash);
        
        if (parent.hash === blocks[parent.number].hash)
            return null;
        
        parentHash = parent.parentHash;
    }
    
    return parentHash;
}

The change of the blockchain to a new fork:

function switchToFork(newbranch) {
    for (var n = newbranch.length; n-- > 0😉 {
        var block = newbranch[n];
        blocks[block.number] = block;
    }
}

All the test passed. As I mentioned, my plan is to improve the implementation. But, as usual in my TDD workflow, I prefer to give baby steps, make it works, and only then, make it right and make it fast. I had good experiences using this way of writing code, obtaining simple and solid implementations, having all the test to help me to change any internal implementation, without pain.

Encouraged by this result, I also started to refactor my blockchain implementation in my personal C# project, too:

https://github.com/ajlopez/BlockchainSharp

I started to use a block store with GetByParentHash into my BlockChain code:

https://github.com/ajlopez/BlockchainSharp/blob/master/Src/BlockchainSharp/Core/BlockChain.cs

I wrote about my previous implementation in this post.

Stay tuned!

Angel “Java” Lopez

@ajlopez

May 9, 2016

Building A Blockchain (8)

Previous Post
Next Post

In previous post, I wrote about the virtual machine I’m implementing in my personal blockchain project, in C#, using TDD (Test-Driven Development) workflow:

https://github.com/ajlopez/SimpleBlockchain

I added a simple bytecode compiler, in order to simplify some tests. The class is not needed for production code, but it is included in the core project:

public class BytecodeCompiler
{
    private IList<byte> bytes = new List<byte>();

    public void Stop()
    {
        this.Compile(Bytecodes.Stop);
    }

    public void Add()
    {
        this.Compile(Bytecodes.Add);
    }

    public void Subtract()
    {
        this.Compile(Bytecodes.Subtract);
    }
    
    // more methods
    
    public byte[] ToBytes()
    {
        return this.bytes.ToArray();
    }
}

It collects a series of bytecodes, with optional arguments. There is a method to return a byte array with the compiled program. A sample test:

[TestMethod]
public void LessThan()
{
    BytecodeCompiler compiler = new BytecodeCompiler();

    compiler.Push(2);
    compiler.Push(2);
    compiler.LessThan();
    compiler.Push(0);
    compiler.Push(1);
    compiler.LessThan();
    compiler.Push(1);
    compiler.Push(0);
    compiler.LessThan();

    Machine machine = new Machine();

    machine.Execute(compiler.ToBytes());

    var stack = machine.Stack;

    Assert.IsNotNull(stack);
    Assert.AreEqual(3, stack.Size);
    Assert.AreEqual(DataWord.Zero, stack.ElementAt(2));
    Assert.AreEqual(DataWord.Zero, stack.ElementAt(1));
    Assert.AreEqual(DataWord.One, stack.ElementAt(0));
}

I also wrote a simple line compiler, named SimpleCompiler. Now, I can use it in my machine tests:

[TestMethod]
public void IsZeroUsingSimpleCompiler()
{
    string program = "push 2\n" +
        "iszero\n" +
        "push 0\n" +
        "iszero";

    SimpleCompiler compiler = new SimpleCompiler(program);

    Machine machine = new Machine();

    machine.Execute(compiler.Compile());

    var stack = machine.Stack;

    Assert.IsNotNull(stack);
    Assert.AreEqual(2, stack.Size);
    Assert.AreEqual(DataWord.Zero, stack.ElementAt(1));
    Assert.AreEqual(DataWord.One, stack.ElementAt(0));
}

These are auxiliary classes, but they simplify the writing of new tests. It is part of my simplification strategy: after some initial and long tests, try to simplify the writing of the newer ones.

In the next post, a surprise: a new blockchain project, but using NodeJS, JavaScript.

Stay tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

May 8, 2016

New Month’s Resolutions: May 2016

Filed under: C Sharp, JavaScript, NodeJs, Open Source Projects, Wang Tiles — ajlopez @ 5:27 pm

Time to write my new month’s resolutions, and review the previous month’s ones:

– Improve WangTiles [complete] see repo
– Improve BlockchainSharp [complete] see repo
– Start Blockchain in JavaScript [complete] see repo
– Work on EthSharp [pending]
– Improve SimpleGA [pending]
– Improve AjGenesisNode-Express [pending]
– Work on CrysJS [pending]
– Work on CrysSharp [pending]

Additionally, I worked on:

– Improve SimpleForth [complete] see repo
– Start WangTilesJS [complete] see repo

My new month’s resolutions are:

– Improve WangTiles
– Improve WangTilesJS
– Improve CrysSharp
– Improve BlockchainSharp
– Improve SimpleBlockchain
– Prepare and give a Bitcoin related talk

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

May 2, 2016

Building A Blockchain (7)

Previous Post
Next Post

In my personal blockchain project:

https://github.com/ajlopez/BlockchainSharp

I want to execute simple code chain, using a virtual machine. The programs are usually called smart contracts. I adopted the Ethereum virtual machine design (read Ethereum Yellow Paper). Some classes:

A DataWord represents a number using 32 bytes. I implemented simple arithmetic operations using System.Numerics.BigInteger internally. I can create DataWord from integers or from byte arrays.

The Stack is an stack of DataWords, manipulated by the Machine. There is an enumeration, named Bytecodes, with the values of byte codes to be executed in the machine implementation. A program consists of a series of bytecodes, contained in a byte array. The Machine.Execute method is the place where the bytecodes are executed, manipulating the stack. Excerpt:

public void Execute(byte[] bytecodes)
{
    int pc = 0;
    int pl = bytecodes.Length;

    while (pc < pl)
    {
        byte bytecode = bytecodes[pc++];

        switch (bytecode)
        {
            case (byte)Bytecodes.Stop:
                return;
            case (byte)Bytecodes.Add:
                this.stack.Push(this.stack.Pop().Add(this.stack.Pop()));
                break;
            case (byte)Bytecodes.Multiply:
                this.stack.Push(this.stack.Pop().Multiply(this.stack.Pop()));
                break;
            case (byte)Bytecodes.Subtract:
                this.stack.Push(this.stack.Pop().Subtract(this.stack.Pop()));
                break;
                
            // more operations
        }
    }
}

I will add Storage and Memory to the execution of a program. The Storage will be associated and persisted with the contract account. Each contract is an account, with address, balance, but with code and storage state. The Memory will be created and used during the execution of the program, but it won’t be persisted: it is a temporary memory, used only at execution time.

I have a simple bytecode compiler, and a line compiler, to facilitate the creation of programs.

Next posts: description of compilers, block execution with transactions, storage and memory, persistent stores.

Keep tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

April 25, 2016

Building A Blockchain (6)

Previous Post
Next Post

These days, I added transaction processing to my personal project:

https://github.com/ajlopez/BlockchainSharp

In the previous post, I described the immutable Trie structure. Now, I have an AccountState that is saved by hash in a such Trie:

public class AccountState
{
    private BigInteger balance;

    public AccountState(BigInteger balance)
    {
        if (BigInteger.Compare(BigInteger.Zero, balance) > 0)
            throw new InvalidOperationException("Invalid balance");

        this.balance = balance;
    }

    public BigInteger Balance { get { return this.balance; } }

    public AccountState AddToBalance(BigInteger amount)
    {
        return new AccountState(BigInteger.Add(this.balance, amount));
    }

    public AccountState SubtractFromBalance(BigInteger amount)
    {
        return new AccountState(BigInteger.Subtract(this.balance, amount));
    }
}

I decided to implement the Ethereum way: having an account state, instead of inputs and outputs. The only property is Balance, but I will add more data. Notice that negative balances are rejeceted. Then, I added a TransactionProcessor:

public class TransactionProcessor
{
    private Trie<AccountState> states;

    public TransactionProcessor(Trie<AccountState> states)
    {
        this.states = states;
    }

    public Trie<AccountState> States { get { return this.states; } }

    public bool ExecuteTransaction(Transaction transaction)
    {
        var states = this.states;

        try
        {
            foreach (var av in transaction.Inputs)
            {
                var addr = av.Address.ToString();
                var state = states.Get(addr);
                var newstate = state.SubtractFromBalance(av.Value);
                states = states.Put(addr, newstate);
            }

            foreach (var av in transaction.Outputs)
            {
                var addr = av.Address.ToString();
                var state = states.Get(addr);
                var newstate = state.AddToBalance(av.Value);
                states = states.Put(addr, newstate);
            }

            this.states = states;

            return true;
        }
        catch (Exception ex)
        {
            return false;
        }
    }
}

If the transaction is processed, a new account state trie is generated, and ExecuteTransaction returns true. If the transaction is rejected, the initial accout state trie still has the original values. A typical test:

[TestMethod]
public void ExecuteTransaction()
{
    var transaction = CreateTransaction(100);

    var addr1 = transaction.Inputs.First().Address;
    var addr2 = transaction.Outputs.First().Address;

    var states = new Trie<AccountState>(new AccountState(BigInteger.Zero));

    states = states.Put(addr1.ToString(), new AccountState(new BigInteger(200)));

    var processor = new TransactionProcessor(states);

    Assert.IsTrue(processor.ExecuteTransaction(transaction));

    var newstates = processor.States;

    Assert.IsNotNull(newstates);
    Assert.AreNotSame(states, newstates);

    Assert.AreEqual(new BigInteger(200), states.Get(addr1.ToString()).Balance);
    Assert.AreEqual(BigInteger.Zero, states.Get(addr2.ToString()).Balance);

    Assert.AreEqual(new BigInteger(100), newstates.Get(addr1.ToString()).Balance);
    Assert.AreEqual(new BigInteger(100), newstates.Get(addr2.ToString()).Balance);
}

The auxiliary CreateTransaction method creates a transaction with an amount, and two random addresses.

I’m thinking to have only one sender account and receiving account per transaction, as in Ethereum. The change is easy, I have all the TDD tests to help me to do redesigns without a lot of work.

Next topics: executing blocks with transactions, saving state in persistent store, virtual machine and its bytecodes, a simple compiler, etc…

Stay tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

April 17, 2016

Building A Blockchain (5)

Filed under: Bitcoin, Blockchain, C Sharp, Ethereum, Test-Driven Development — ajlopez @ 10:02 am

Previous Post
Next Post

I was working a lot on my personal project:

https://github.com/ajlopez/BlockchainSharp

implementing a blockchain in C#, using TDD (Test-Driven Development) workflow. One element I need to implement, is the store of account states (initially their balances). The balance of an account should be retrieved by account id (a hash). But in many use cases, I need to know the balances of accounts at a given time. It is not enough to have the LATESTS balances. So, I implemented a data structure, a trie, but with immutable properties.

A trie is a tree where the non-leaf nodes stores part of the key:

In the above example, the value V1 is associated with key AAA, and the value V5 is associated with key ACA. When I change the value associated with key ABC from V4 to V7, part of a new trie is created, so the original one persists without modification:

I can access the original key/values using the “old root”, and then, I can switch to use the “new root”, at any moment. If any node/root is not reachable from a variable, garbage collector will release the memory associated with that node.

I wrote a typed trie implementation, using TDD. An example test:

[TestMethod]
public void PutAndGetKeyValue()
{
    Trie<string> trie = new Trie<string>();

    var trie2 = trie.Put("012", "foo");

    Assert.IsNotNull(trie2);
    Assert.AreNotSame(trie2, trie);
    Assert.IsNull(trie.Get("012"));
    Assert.AreEqual("foo", trie2.Get("012"));
}

An example that shows the persistent of tries after updates:

[TestMethod]
public void ReplaceValue()
{
    Trie<string> trie = new Trie<string>();

    var trie2 = trie.Put("012", "foo");
    var trie3 = trie2.Put("012", "bar");

    Assert.IsNotNull(trie2);
    Assert.AreNotSame(trie2, trie);
    Assert.IsNull(trie.Get("012"));
    Assert.AreEqual("foo", trie2.Get("012"));

    Assert.IsNotNull(trie3);
    Assert.AreNotSame(trie3, trie2);
    Assert.AreEqual("bar", trie3.Get("012"));
}

My idea is to use Trie<AccountState> as a store for account balances. At the end of a block (with transactions) processing, there is an account state trie. Ant the end of the next block processing, another trie will be generated. At any moment, I could retrieve the account balances at time “block 1”, and at time “block 2”, using the generated tries.

Stay tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

« Newer PostsOlder Posts »

Blog at WordPress.com.