Category Archives: Distributed Computing

Distributed Computing: Links, News And Resources (7)

Previous Post

mateodelnorte/servicebus · GitHub
https://github.com/mateodelnorte/servicebus

Return of the Borg: How Twitter Rebuilt Google’s Secret Weapon | Wired Enterprise | Wired.com
http://www.wired.com/wiredenterprise/2013/03/google-borg-twitter-mesos/

2011 GAFS Omega John Wilkes – YouTube
http://www.youtube.com/watch?v=0ZFMlO98Jkc

The history of Hadoop: From 4 nodes to the future of data — Tech News and Analysis
http://gigaom.com/2013/03/04/the-history-of-hadoop-from-4-nodes-to-the-future-of-data/

Understanding the Parallelism of a Storm Topology – Michael G. Noll
http://www.michael-noll.com/blog/2012/10/16/understanding-the-parallelism-of-a-storm-topology/

Gluster Community Website
http://www.gluster.org/about/
GlusterFS is an open source, distributed file system capable of scaling to several petabytes (actually, 72 brontobytes!) and handling thousands of clients.

Introducing Akka – Simpler Scalability, Fault-Tolerance, Concurrency & Remoting through Actors ?
http://jonasboner.com/2010/01/04/introducing-akka/

Making reliable distributed systems in the presence of sodware errors
http://www.sics.se/~joe/thesis/armstrong_thesis_2003.pdf

Pomelo home
http://pomelo.netease.com/
distributed game server framework for node.js

Design Alternatives for Parallel and Distributed Blackboard Systems
http://dancorkill.home.comcast.net/~dancorkill/pubs/parallel-distributed-chapter.pdf

Zipkin: a Distributed Tracing Framework
http://www.infoq.com/presentations/Zipkin

Puppet Labs: IT Automation Software for System Administrators
https://puppetlabs.com/

Remoting (Scala) — Akka Documentation
http://doc.akka.io/docs/akka/current/scala/remoting.html

Let it crash • Balancing Workload Across Nodes with Akka 2
http://letitcrash.com/post/29044669086/balancing-workload-across-nodes-with-akka-2

Anansi
https://github.com/nko/anansi
Distributed MapReduce implementation for JavaScript that runs on the worlds largest computational network. Consumer PCs.

Federation
http://underflow.ca/federation/
https://github.com/jacobgroundwater/federation
Federation is inspired by Akka and Erlang, and borrows some semantics from the actor model. While there are actors, many features familiar to Akka and Erlang are not included. Federation prioritizes being useful to the Node.js community over being faithful to the pure actor model.

Twitter’s programmers speed Hadoop development | Big Data – InfoWorld
http://www.infoworld.com/d/big-data/twitters-programmers-speed-hadoop-development-211931

bigcompany/big · GitHub
https://github.com/bigcompany/big

Capistrano
https://github.com/capistrano/capistrano
Remote multi-server automation tool

GPU Technology Conference Agenda Chocked Full of Supercomputing Sessions | insideHPC.com
http://insidehpc.com/2013/01/25/gpu-technology-conference-agenda-chocked-full-of-supercomputing-sessions/

My Links
http://delicious.com/ajlopez/distributedcomputing

Keep tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

Spark: Links and Resources (1)

There are two project named Spark in Java: a web framework, and a distributed map reduce runner.

Scalable Machine Learning | edX
https://www.edx.org/course/scalable-machine-learning-uc-berkeleyx-cs190-1x

Spark Programming Guide – Spark 1.3.1 Documentation
https://spark.apache.org/docs/latest/programming-guide.html

Overview – Spark 1.3.1 Documentation
https://spark.apache.org/docs/latest/

firmata/spark
https://github.com/firmata/spark

firmata/protocol
https://github.com/firmata/protocol

Top Best Tools for Java Programmers | Devzum – Its all about Design & Development
http://devzum.com/2015/01/15/10-best-java-tools-that-every-java-programmers-should-know/

Sparkling, A Clojure API for Apache Spark
https://gorillalabs.github.io/sparkling/

Getting Started with Sparkling
https://gorillalabs.github.io/sparkling/articles/getting_started.html

Developing Single Page Web Applications using Java 8, Spark, MongoDB, and AngularJS – OpenShift Blog
https://blog.openshift.com/developing-single-page-web-applications-using-java-8-spark-mongodb-and-angularjs/

Graylog2/spark
https://github.com/Graylog2/spark

Spark Framework – A tiny Java web framework
http://sparkjava.com/

Apache Spark with Scala
http://www.slideshare.net/frodriguezolivera/apache-spark-41601032

yieldbot/flambo
https://github.com/yieldbot/flambo

Windows on Devices
https://www.windowsondevices.com/

MapReduce and Spark | Cloudera VISION
http://vision.cloudera.com/mapreduce-spark/

spark-summit.org/wp-content/uploads/2013/10/Baldeschwieler-SparkSummit2013v2.pdf
http://spark-summit.org/wp-content/uploads/2013/10/Baldeschwieler-SparkSummit2013v2.pdf

Mesosphere · Learn how to use Apache Mesos
http://mesosphere.io/learn/

Got a Minute? Spin up a Spark cluster on your laptop with Docker. | AMPLab – UC Berkeley
https://amplab.cs.berkeley.edu/2013/10/23/got-a-minute-spin-up-a-spark-cluster-on-your-laptop-with-docker/

Spark Streaming Programming Guide – Spark 0.7.3 Documentation
http://spark.incubator.apache.org/docs/0.7.3/streaming-programming-guide.html

Spark: Open Source Superstar Rewrites Future of Big Data | Wired Enterprise | Wired.com
http://www.wired.com/wiredenterprise/2013/06/yahoo-amazon-amplab-spark/

Spark | Lightning-Fast Cluster Computing
http://spark-project.org/

Learning Spark – O’Reilly Media
http://shop.oreilly.com/product/0636920028512.do

http://www.cs.berkeley.edu/~matei/papers/2011/tr_spark.pdf
http://www.cs.berkeley.edu/~matei/papers/2011/tr_spark.pdf

My Links
http://delicious.com/ajlopez/spark

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

Programando for Internet of Things

Usually, I practice programming every day, applying TDD (Test-Driven Development) workflow. There are non-public projects, but the majority of that practice is public, in my GitHub account. In my opinion, Node.js is a very interesting technology: ubiquos, powerful and simple. I can implement many ideas using Node.js/JavaScript, in a easier way than using other technologies (Java, .NET, Scala, Ruby, Python, Clojure, Smalltalk). The only price to pay: understand and use the JavaScript callbacks.

There are many ideas and implementation, in current applications, public projects, startups. An interesting topic is Internet of Things (OK, a “buzzword”). Many startups are fighting in that battle field. Maybe, many of them will be adquired by Google, Apple, Microsoft. These are interesting time. But this is a time for implement ideas, in open source projects. Then, an startup can leverage the open source world. I think that the important part of an startup is the execution, not the idea or the implementation..

Meanwhile, I want to implement some ideas in my public projects. This post is written to describe the landscape of ideas to implement:

– Collect informaiton from devices connected to the Internet. To have a data repository, the data is send by SDKs, one SDK per device type. Then, use the repository and the collected data in other projects.

– Apply Artificial Intelligence (OK, it is a wide term, but the better we have), to all the collected data, discovering patterns, using machine learning, deep learning. I wrote some JavaScript projects to explore such paths.

– Take decisions, execute actions, using expert systems or other systems. My first implementations, again, are in JavaScript/NodeJs. Having somethink like IFTTT but open source, consuming Internet of Things data.

– Run all these implementations as distributed applicacionts, not only horizontal scalability or cloud computing. Node.js, again, is an interesting vehicle to implement first experiments and applications. Only if needed, then switch to compiled languages and technologies.

I apologize my auto-reference to my projects, but it is the way to explain why I’m writing them.

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

Akka: Links, News And Resources (10)

Previous Post

VaughnVernon/Dotsero
https://github.com/VaughnVernon/Dotsero

Distributed Publish Subscribe in Cluster — Akka Documentation
http://doc.akka.io/docs/akka/2.3.2/contrib/distributed-pub-sub.html

Kamon | Core | Documentation
http://kamon.io/core/tracing/core-concepts/

Parleys.com – Resilient Applications with Akka Persistence by Björn Antonsson, Konrad Malawski, Patrik Nordwall
http://www.parleys.com/play/53a7d2c3e4b0543940d9e53b/chapter0/about

AKKA 5 Year Anniversary
http://typesafe.com/akka-five-year-anniversary

How One Developer Set Out To Make The Internet Of Things Manageable – ReadWrite
http://readwrite.com/2014/07/10/akka-jonas-boner-concurrency-distributed-computing-internet-of-things

typesafehub/activator-akka-distributed-workers
https://github.com/typesafehub/activator-akka-distributed-workers/

etaty/rediscala
https://github.com/etaty/rediscala

agibsonccc/java-deeplearning
https://github.com/agibsonccc/java-deeplearning

mpollmeier/akka-patterns
https://github.com/mpollmeier/akka-patterns

Xitrum
http://xitrum-framework.github.io/xitrum/

My Links
http://delicious.com/ajlopez/akka

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

Akka: Links, News And Resources (9)

Previous Post
Next Post

Building Reactive Applications with Akka
http://hakking-at.scalabcn.org/

Typed Actors — Akka Documentation
http://doc.akka.io/docs/akka/snapshot/scala/typed-actors.html

Reactive DDD with Akka | Write less, do more!
http://pkaczor.blogspot.co.uk/2014/04/reactive-ddd-with-akka.html

Publish/Subscribe using Scala and Akka EventStream
http://www.benhowell.net/examples/2014/04/18/scala-and-the-akka-eventstream/

Vaadin in Akka – Blog – vaadin.com
https://vaadin.com/blog/-/blogs/vaadin-in-akka

Endre Sándor Varga – Without Present or Past: How to Think Distributed – The Hard Way, With the rise of distributed databases, cluster management tools, co…
http://www.ustream.tv/recorded/46670663

Reactive Programming with Akka and Scala | comSysto Blog
http://blog.comsysto.com/2014/05/09/reactive-programming-with-akka-and-scala/

Let it crash
http://letitcrash.com/

woq/de.woq.osgi.java
https://github.com/woq/de.woq.osgi.java

michaelpisula/akka-journal-inmemory
https://github.com/michaelpisula/akka-journal-inmemory

Jonas Bonér – Going Reactive: Event-Driven, Scalable, Resilient & Responsive Systems, The demands and expectations for applications have changed dramat…
http://www.ustream.tv/recorded/46625425

Get started quicker with our docker image
http://kamon.io/teamblog/2014/04/27/get-started-quicker-with-our-docker-image/

Java is the new C: Comparision of different concurrency models: Actors, CSP, Disruptor and Threads
http://java-is-the-new-c.blogspot.com.ar/2014/01/comparision-of-different-concurrency.html

chanan/SignalJ
https://github.com/chanan/SignalJ

April 17, 2014 | Typesafe announces Akka Streams, a part of the Reactive Streams Initiative | Typesafe
https://typesafe.com/blog/typesafe-announces-akka-streams

akka/akka-stream/src/main/scala/akka/stream at release-2.3 · akka/akka
https://github.com/akka/akka/tree/release-2.3/akka-stream/src/main/scala/akka/stream

A very simple Ping Pong example using Akka’s local actors.
https://gist.github.com/ConnorDoyle/1521658

Orleans Actors Analysis – Tech stuff that interests me
http://shinolajla.tumblr.com/post/81199345476/orleans-actors-analysis

Abstractivate: ForkJoinPool: the Other ExecutorService
http://blog.jessitron.com/2014/02/forkjoinpool-other-executorservice.html

Running an Akka Cluster with Docker Containers | Adventures in HttpContext
http://blog.michaelhamrah.com/2014/03/running-an-akka-cluster-with-docker-containers/

Getting Started with Actor Based Programming Using Scala and Akka – Reactive.TIPS – The Official Blog of Reactive.IO
http://www.reactive.io/tips/2014/03/28/getting-started-with-actor-based-programming-using-scala-and-akka/

My Links
http://delicious.com/ajlopez/akka

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

Erlang: Links, News And Resources (12)

Previous Post

AKKA 5 Year Anniversary
http://typesafe.com/akka-five-year-anniversary

erlang-laboratory – Erlang Laboratory — Erlang development in Windows as simple as 1, 2, 3 – Google Project Hosting
https://code.google.com/p/erlang-laboratory/

Michael Feathers – Microservices and the Failure of Encapsulation
https://michaelfeathers.silvrback.com/microservices-and-the-failure-of-encapsulaton

(390) Python (programming language): With the increase of the processors speeds, will high level languages (like python) take over low level languages (like C/C ) since difference in execution speed will not differ that much? – Quora
http://www.quora.com/Python-programming-language-1/With-the-increase-of-the-processors-speeds-will-high-level-languages-like-python-take-over-low-level-languages-like-C-C++-since-difference-in-execution-speed-will-not-differ-that-much

Damien Katz: What Sucks About Erlang
http://damienkatz.net/2008/03/what_sucks_abou.html

pyotrgalois/erlskeletor
https://github.com/pyotrgalois/erlskeletor

Why Erlang Is Awesome
http://veldstra.org/whyerlang/

Erlang Integers | Programmer’s Paradox
http://www.programmersparadox.com/2008/02/05/erlang-integers/

Handling very large integers in Erlang | Teton Technical
http://tetontech.wordpress.com/2008/05/24/handling-very-large-integers-in-erlang/

cloudozer/ling
https://github.com/cloudozer/ling

inaka/erlang_training · GitHub
https://github.com/inaka/erlang_training

inaka/spawnedshelter · GitHub
https://github.com/inaka/spawnedshelter

pyotrgalois/spawnedshelter
https://github.com/pyotrgalois/spawnedshelter

My Links
http://delicious.com/ajlopez/erlang

Stay tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

Akka: Links, News And Resources (8)

Previous Post
Next Post

Streaming Twitter API in Akka and Spray | Cake Solutions Team Blog
http://www.cakesolutions.net/teamblogs/2013/12/08/streaming-twitter-api-in-akka-and-spray/

Akka persistence webinar
http://www.slideshare.net/patriknw/akka-persistence-webinar

Intro to Akka persistence with Patrik Nordwall – YouTube
https://www.youtube.com/watch?v=r5lecCBazvE

Community Page Reinvisioned | Akka
http://akka.io/news/2014/03/12/community-page-update.html

Prabhath’s Bolg: Integrating an Akka.io actor system with Play framework (A distributed message classifier with Akka.io and Play framework)
http://psuminda.blogspot.com.es/2014/02/integrating-akkaio-actor-system-with.html

Martin’s Programming Blog: Distributed Actors in Clojure
http://martinsprogrammingblog.blogspot.com.ar/2012/05/distributed-actors-in-clojure.html

Fun doesn’t mean compromising scalability – all that jazz
http://jazzy.id.au/default/2013/12/17/fun_doesnt_mean_compromising_scalability.html

Using Scala Implicits to Implement a Messaging Protocol
http://derekwyatt.org/2014/01/01/using-scala-implicits-to-implement-a-messaging-protocol.html

Camel — Akka Documentation
http://doc.akka.io/docs/akka/snapshot/scala/camel.html

Post: Akka Extras Email
http://blog.eigengo.com/blog_posts/akka-extras-email

oxlade39/STorrent
https://github.com/oxlade39/STorrent
BitTorrent client written in Scala using Akka

scala – How to create minimal Akka actor? – Stack Overflow
http://stackoverflow.com/questions/17632234/how-to-create-minimal-akka-actor/17638135#17638135

scala/src/library/scala/concurrent/impl/ExecutionContextImpl.scala at master · scala/scala
https://github.com/scala/scala/blob/master/src/library/scala/concurrent/impl/ExecutionContextImpl.scala#L33

My Links
http://delicious.com/ajlopez/akka

Keep tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez