Monthly Archives: April 2016

Redis: Links, News And Resources (8)

Previous Post
Next Post

Sketching & Scaling: Everyday HyperLogLog | Official Kiip Blog
http://blog.kiip.me/engineering/sketching-scaling-everyday-hyperloglog/

Scaling SQL with Redis – David Cramer’s Blog
http://cramer.io/2014/05/12/scaling-sql-with-redis/

Walkthrough: Setting up a Redis cluster on Windows Azure
http://haishibai.blogspot.com.ar/2014/01/walkthrough-setting-up-redis-cluster-on.html

Robert Ian Hawdon, B.Sc. (Hons) | [SysOps] Installing a High Availability Redis service on CentOS 6.X in Windows Azure
http://robertianhawdon.me.uk/2014/02/11/sysops-installing-a-high-availability-redis-service-on-centos-6-x-in-windows-azure/comment-page-1/#comments

How Disqus Went Realtime with 165K Messages Per Second and Less than .2 Seconds Latency – High Scalability –
http://highscalability.com/blog/2014/4/28/how-disqus-went-realtime-with-165k-messages-per-second-and-l.html

Sharing Users Session Between Django and Flask with Redis | Toptal
http://www.toptal.com/django/django-flask-and-redis-sharing-user-sessions-between-frameworks

luca3m/redis3m
https://github.com/luca3m/redis3m

Redis Administration – Redis
http://redis.io/topics/admin

Resharding Redis In a Live Environment – Juicebox Games
http://blog.juiceboxmobile.com/2014/01/15/resharding-redis-in-a-live-environment/

Redis Lists as Viral Inboxes – Juicebox Games
http://blog.juiceboxmobile.com/2013/01/01/redis-lists-as-viral-inboxes/

Redis new data structure: the HyperLogLog – Antirez weblog
http://antirez.com/news/75

josiahcarlson/lua-call
https://github.com/josiahcarlson/lua-call/

indexing – Alphabetical index with millions of rows in redis – Stack Overflow
http://stackoverflow.com/questions/21311863/alphabetical-index-with-millions-of-rows-in-redis

The Yaksis
http://www.yaksis.com/posts/vowpal_wabbit-the-redis-of-the-data-science-community.html

Redis Scripting with MRuby — Luca Guidi
http://lucaguidi.com/2014/02/26/redis-scripting-with-mruby-pt1.html

Session Handling for 1 million requests per hour | CloudMagic Blog
http://blog.cloudmagic.com/2014/02/18/session-handling-for-1-million-requests-per-hour/

Redis Master-Slave on Docker
http://www.hvflabs.com/posts/redis-master-slave-on-docker

Building a Multi-Master Distributed Redis in Erlang
http://www.infoq.com/presentations/redis-erlang

My Links
http://delicious.com/ajlopez/redis

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

Redis: Links, News And Resources (7)

Previous Post
Next Post

Securing Redis with Sedona – Will Urbanski on Vimeo
http://vimeo.com/79743671

Announcement: Redis Monitoring in SPM | Sematext Blog
http://blog.sematext.com/2014/01/30/announcement-redis-monitoring-in-spm/

Running Redis as a Windows Service – CodeProject
http://www.codeproject.com/Articles/715967/Running-Redis-as-a-Windows-Service

Forrst | How I make models in Node.js (using Redis as database) – Some code from jackbach
http://forrst.com/posts/How_I_make_models_in_Node_js_using_Redis_as_dat-90W

Redis Cluster and limiting divergences. – Antirez weblog
http://antirez.com/news/70

Using Redis sorted sets to build a scalable real-time web waiting list. | The Carbon Emitter
http://blog.carbonfive.com/2014/01/17/using-redis-sorted-sets-to-build-a-scalable-real-time-web-waiting-list/

forcedotcom/rmux
https://github.com/forcedotcom/rmux

High Scalability – High Scalability – How HipChat Stores and Indexes Billions of Messages Using ElasticSearch and Redis
http://highscalability.com/blog/2014/1/6/how-hipchat-stores-and-indexes-billions-of-messages-using-el.html

Knossos: Redis and linearizability
http://aphyr.com/posts/309-knossos-redis-and-linearizability

brightmarch/picto
https://github.com/brightmarch/picto

softwaredoug/subredis
https://github.com/softwaredoug/subredis

OptimalBits/bull
https://github.com/OptimalBits/bull

Some fun with Redis Cluster testing – Antirez weblog
http://antirez.com/news/69

Redis as AP system, reloaded – Antirez weblog
http://antirez.com/news/68

Redis cluster tutorial – Redis
http://redis.io/topics/cluster-tutorial

Redis Sentinel Documentation – Redis
http://redis.io/topics/sentinel

My Links
http://delicious.com/ajlopez/redis

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

Building A Blockchain (6)

Previous Post
Next Post

These days, I added transaction processing to my personal project:

https://github.com/ajlopez/BlockchainSharp

In the previous post, I described the immutable Trie structure. Now, I have an AccountState that is saved by hash in a such Trie:

public class AccountState
{
    private BigInteger balance;

    public AccountState(BigInteger balance)
    {
        if (BigInteger.Compare(BigInteger.Zero, balance) > 0)
            throw new InvalidOperationException("Invalid balance");

        this.balance = balance;
    }

    public BigInteger Balance { get { return this.balance; } }

    public AccountState AddToBalance(BigInteger amount)
    {
        return new AccountState(BigInteger.Add(this.balance, amount));
    }

    public AccountState SubtractFromBalance(BigInteger amount)
    {
        return new AccountState(BigInteger.Subtract(this.balance, amount));
    }
}

I decided to implement the Ethereum way: having an account state, instead of inputs and outputs. The only property is Balance, but I will add more data. Notice that negative balances are rejeceted. Then, I added a TransactionProcessor:

public class TransactionProcessor
{
    private Trie<AccountState> states;

    public TransactionProcessor(Trie<AccountState> states)
    {
        this.states = states;
    }

    public Trie<AccountState> States { get { return this.states; } }

    public bool ExecuteTransaction(Transaction transaction)
    {
        var states = this.states;

        try
        {
            foreach (var av in transaction.Inputs)
            {
                var addr = av.Address.ToString();
                var state = states.Get(addr);
                var newstate = state.SubtractFromBalance(av.Value);
                states = states.Put(addr, newstate);
            }

            foreach (var av in transaction.Outputs)
            {
                var addr = av.Address.ToString();
                var state = states.Get(addr);
                var newstate = state.AddToBalance(av.Value);
                states = states.Put(addr, newstate);
            }

            this.states = states;

            return true;
        }
        catch (Exception ex)
        {
            return false;
        }
    }
}

If the transaction is processed, a new account state trie is generated, and ExecuteTransaction returns true. If the transaction is rejected, the initial accout state trie still has the original values. A typical test:

[TestMethod]
public void ExecuteTransaction()
{
    var transaction = CreateTransaction(100);

    var addr1 = transaction.Inputs.First().Address;
    var addr2 = transaction.Outputs.First().Address;

    var states = new Trie<AccountState>(new AccountState(BigInteger.Zero));

    states = states.Put(addr1.ToString(), new AccountState(new BigInteger(200)));

    var processor = new TransactionProcessor(states);

    Assert.IsTrue(processor.ExecuteTransaction(transaction));

    var newstates = processor.States;

    Assert.IsNotNull(newstates);
    Assert.AreNotSame(states, newstates);

    Assert.AreEqual(new BigInteger(200), states.Get(addr1.ToString()).Balance);
    Assert.AreEqual(BigInteger.Zero, states.Get(addr2.ToString()).Balance);

    Assert.AreEqual(new BigInteger(100), newstates.Get(addr1.ToString()).Balance);
    Assert.AreEqual(new BigInteger(100), newstates.Get(addr2.ToString()).Balance);
}

The auxiliary CreateTransaction method creates a transaction with an amount, and two random addresses.

I’m thinking to have only one sender account and receiving account per transaction, as in Ethereum. The change is easy, I have all the TDD tests to help me to do redesigns without a lot of work.

Next topics: executing blocks with transactions, saving state in persistent store, virtual machine and its bytecodes, a simple compiler, etc…

Stay tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

ReactJS: Links And Resources (4)

Previous Post
Next Post

Jed Watson – Going Mobile with React at react-europe 2015 – YouTube
https://www.youtube.com/watch?v=ctwmd5L1U_Q

Pure UI
http://rauchg.com/2015/pure-ui/

Tag Archive for “javascript” | Scotch
https://scotch.io/tag/javascript

Lee Byron – Exploring GraphQL at react-europe 2015 – YouTube
https://www.youtube.com/watch?v=WQLzZf34FJ8

Dan Abramov – Live React: Hot Reloading with Time Travel at react-europe 2015 – YouTube
https://www.youtube.com/watch?v=xsSnOQynTHs

React Native Playground: Share and test your React Native code in the browser
https://rnplay.org/

WinJS 4.0 Is Compatible with AngularJS, React, Bootstrap and Knockout
http://www.infoq.com/news/2015/06/winjs-4-angular-react

Mutations in Relay // Speaker Deck
https://speakerdeck.com/laneyk/mutations-in-relay

Harrison Harnisch: React native and Meteor — Devshop SF April 2015 – YouTube
https://www.youtube.com/watch?v=7BF5LHn2B5s&feature=em-subs_digest

Max Harris: Two weeks with React Meteor — Devshop SF March 2015 – YouTube
https://www.youtube.com/watch?v=-QtrkXKvQFc&feature=youtu.be&list=PLTUf4ytkmI8Q3mSRIrkPfjkERTxHopzKG

CS 294-101 01 Pete Hunt: React vs The World – YouTube
https://www.youtube.com/watch?v=MC376f3QWYw

Reagent • TodoMVC
http://todomvc.com/examples/reagent/

My Links
https://delicious.com/ajlopez/reactjs

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

Building A Blockchain (5)

Previous Post
Next Post

I was working a lot on my personal project:

https://github.com/ajlopez/BlockchainSharp

implementing a blockchain in C#, using TDD (Test-Driven Development) workflow. One element I need to implement, is the store of account states (initially their balances). The balance of an account should be retrieved by account id (a hash). But in many use cases, I need to know the balances of accounts at a given time. It is not enough to have the LATESTS balances. So, I implemented a data structure, a trie, but with immutable properties.

A trie is a tree where the non-leaf nodes stores part of the key:

In the above example, the value V1 is associated with key AAA, and the value V5 is associated with key ACA. When I change the value associated with key ABC from V4 to V7, part of a new trie is created, so the original one persists without modification:

I can access the original key/values using the “old root”, and then, I can switch to use the “new root”, at any moment. If any node/root is not reachable from a variable, garbage collector will release the memory associated with that node.

I wrote a typed trie implementation, using TDD. An example test:

[TestMethod]
public void PutAndGetKeyValue()
{
    Trie<string> trie = new Trie<string>();

    var trie2 = trie.Put("012", "foo");

    Assert.IsNotNull(trie2);
    Assert.AreNotSame(trie2, trie);
    Assert.IsNull(trie.Get("012"));
    Assert.AreEqual("foo", trie2.Get("012"));
}

An example that shows the persistent of tries after updates:

[TestMethod]
public void ReplaceValue()
{
    Trie<string> trie = new Trie<string>();

    var trie2 = trie.Put("012", "foo");
    var trie3 = trie2.Put("012", "bar");

    Assert.IsNotNull(trie2);
    Assert.AreNotSame(trie2, trie);
    Assert.IsNull(trie.Get("012"));
    Assert.AreEqual("foo", trie2.Get("012"));

    Assert.IsNotNull(trie3);
    Assert.AreNotSame(trie3, trie2);
    Assert.AreEqual("bar", trie3.Get("012"));
}

My idea is to use Trie<AccountState> as a store for account balances. At the end of a block (with transactions) processing, there is an account state trie. Ant the end of the next block processing, another trie will be generated. At any moment, I could retrieve the account balances at time “block 1”, and at time “block 2”, using the generated tries.

Stay tuned!

Angel “Java” Lopez

http://www.ajlopez.com

http://twitter.com/ajlopez

Building A Blockchain (4)

Previous Post
Next Post

In this post, I want to describe the key parts of a blockchain example I’m writing at:

https://github.com/ajlopez/BlockchainSharp

using C# and TDD (Test-Driven Development). The essential things to be implemented are:

– Blocks
– Transactions

the concepts of:

– Account
– Account State
– Smart Contracts (executed in a virtual machine)

and utilities for:

– Encode/Decode entities to be transmited/received via the network of nodes
– Hash calculations
– Immutable tries
– Sign transactions
– Verify transaction sign
– Verify block integrity
– Verify transactions are valid
– Proof of Work calculation (as usual in Bitcoin and Ethereum implementations)
– Store some entities: complete blocks, partial blocks (block headers in Ethereum), account state, … in general, tries with storage

Some days ago, I started to write a Transaction class, having pairs of accounts and values. I adopted Ethereum approach: instead of having inputs/outputs as in Bitcoin, I have accounts with balances/states. Each account is identified by an address (actually a random hexadecimal string):

Each transaction can have one or more inputs and outputs. Each input/output has an account address and a value. The values are instances of BigInteger, so I can manage large integer numbers. It’s the way the criptocurrencies implementations allow the use of value fractions: the unity is a power of 10, and the amounts are expressed in integer quantity of minimal fractions.

Next steps: add transactions to blocks, apply transaction to account states, validate that input values are available, store states in an immutable trie,…

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

New Month’s Resolutions: Aprill 2016

A new month started, and it’s time to write the new resolutions. And to review the past month ones:

– Improve AjGenesisNode-Express [pending]
– Work on CrysJS [complete] see repo
– Work on CrysSharp [pending]
– Improve my SimpleGA samples [pending
– Work on SharpGo [pending]
– Work on EthSharp [complete] see repo

Also I worked on:

– Improve SimpleForth in JavaScript [complete] see repo
– Start WangTiles in C# [complete] see repo
– Publish new version of SimpleArgs [complete] see repo
– Start BlockchainSharp in C# [complete] see repo
– Improve SimpleDT samples [complete] see repo
– Move AjGo to GitHub [complete] see repo
– Improve RuScript [complete] see repo
– Add Code Coverate to ethereumjs rlp [complete] see repo
– New Machine Learning in JavaScript presentation [complete] see repo see talk

My new resolutions:

– Improve WangTiles
– Improve BlockchainSharp
– Start Blockchain in JavaScript
– Work on EthSharp
– Improve SimpleGA
– Improve AjGenesisNode-Express
– Work on CrysJS
– Work on CrysSharp

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez