Distributed Computing: Links, News and Resources (1)

Next Post

I just realized I never published a list of links of one of my preferred topic. This is the first post:

Storm – a real time Hadoop like system in Clojure
http://pseudofish.com/blog/2011/09/26/storm-a-real-time-hadoop-like-system-in-clojure/

Hadoop Programming Challenge
http://bigdatauniversity.com/web/hadoop-programming-challenge.php

The Design of Distributed Applications
http://groups.google.com/group/the-design-of-distributed-applications

Thoughts around REST, DDD, and CQRS: Models, Queries, and Commands
http://groups.google.com/group/the-design-of-distributed-applications/t/f2295ec60ef87c77

Akka 2.x roadmap…
https://docs.google.com/document/pub?id=1CMz_MEQA8oPcGw9oaFdq_KYYFB_5qZjsDYYwuXfZhBU&pli=1

Chess@home
http://chessathome.org/

Welcome to Apache Pig
http://pig.apache.org/
Apache Pig is a platform for analyzing large data sets that consists of a high-level language for expressing data analysis programs, coupled with infrastructure for evaluating these programs. The salient property of Pig programs is that their structure is amenable to substantial parallelization, which in turns enables them to handle very large data sets.

Welcome to Hama project
http://incubator.apache.org/hama/
Apache Hama is a distributed computing framework based on BSP (Bulk Synchronous Parallel) computing techniques for massive scientific computations, e.g., matrix, graph and network algorithms. It was inspired by Google’s Pregel, but different in the sense that it’s purely BSP and common model, not just for graph.

InfoQ: Things Break, Riak Bends
http://www.infoq.com/presentations/Things-Break-Riak-Bends

HPCC Systems | Open-source. Fast. Scalable. Simple
http://hpccsystems.com/
HPCC (High Performance Computing Cluster) is a massive parallel-processing computing platform that solves Big Data problems. The platform is now Open Source!

SmartFrog
http://wiki.smartfrog.org/wiki/display/sf/SmartFrog+Home
SmartFrog is a powerful and flexible Java-based software framework for configuring, deploying and managing distributed software systems.

Mesos: Dynamic Resource Sharing for Clusters
http://www.mesosproject.org/
Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications, or frameworks. It can run Hadoop, MPI, Hypertable, Spark (a new framework for low-latency interactive and iterative jobs), and other applications. Mesos is open source in the Apache Incubator.

Dryad – Microsoft Research
http://research.microsoft.com/en-us/projects/Dryad/

InfoQ: Secure Distributed Programming on ECMAScript 5 + HTML5
http://www.infoq.com/presentations/Secure-Distributed-Programming

Ceph as a scalable alternative to the Hadoop Distributed File System
http://www.usenix.org/publications/login/2010-08/openpdfs/maltzahn.pdf

Data-driven Apps With Microsoft Velocity Distributed Caching
http://msdn.microsoft.com/en-us/magazine/dd861287.aspx

Spark
http://www.spark-project.org/
Spark is an open source cluster computing system that aims to make data analytics fast — both fast to run and fast to write.

Distributed computing fallacies and REST
http://lostechies.com/jimmybogard/2011/05/27/distributed-computing-fallacies-and-rest/

Presentation Schedule // CS 525: Advanced Distributed Systems // Spring 2011
http://www.cs.uiuc.edu/class/sp11/cs525/sched.htm

InfoQ: Francesco Cesarini and Simon Thompson on Erlang
http://www.infoq.com/interviews/cesarini-thompson-erlang

TypeSafe
http://typesafe.com/
Scala and Akka are deployed in production at some of the largest web properties and financial institutions in the world, and run on the battle-tested Java runtime environment. Deploy with confidence.

Introducing Riak Core
http://blog.basho.com/2010/07/30/introducing-riak-core/

Actors: A Model of Concurrent Computation in Distributed Systems
http://dspace.mit.edu/handle/1721.1/6952

The Hadoop Distributed File System
http://storageconference.org/2010/Papers/MSST/Shvachko.pdf

InfoQ: Concurrency Control in Data Replication
http://www.infoq.com/articles/Concurrency-Control-Data-Replication

Build a distributed realtime tweet search system in no time. Part 1/2
http://sna-projects.com/blog/2011/02/build-a-distributed-realtime-tweet-search-system-in-no-time-part-12/

Windows Azure futures: Turning the cloud into a supercomputer
http://www.zdnet.com/blog/microsoft/windows-azure-futures-turning-the-cloud-into-a-supercomputer/8592

Episode 1: Distributed Systems Host Introductions
http://distributedpodcast.com/2010/episode-1-host-introductions

Distributed Podcast
http://distributedpodcast.com

Frangipani: A Scalable Distributed File System
http://www.systemswemake.com/blog/28/frangipani/

Systems We Make
http://www.systemswemake.com

Fault tolerance techniques for distributed systems
http://www.ibm.com/developerworks/rational/library/114.html

Swarm: A true distributed programming language
http://blog.locut.us/2008/10/07/swarm-a-true-distributed-programming-language/

MSDN Magazine: Distributed Apps
http://msdn.microsoft.com/en-us/magazine/gg232773.aspx

Scalable System Design Patterns
http://horicky.blogspot.com/2010/10/scalable-system-design-patterns.html
Load Balancer, Scatter and Gather, Result Cache, Shared Space, Pipe and Filter, Map Reduce, Bulk Synchronous Parallel, Execution Orchestrator

My Links
http://www.delicious.com/ajlopez/distributedcomputing

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez