Parallel Programming: Links, News And Resources (3)

Previous Post

Concurrent Programming Using The Disruptor
http://www.infoq.com/presentations/Concurrent-Programming-Using-The-Disruptor
Trisha Gee introduces Disruptor, a concurrency framework based on a data structure – a ring buffer – that enables fast message passing in a parallel environment.

Dataflow Programming: Handling Huge Data Loads Without Adding Complexity
http://www.drdobbs.com/go-parallel/article/231400148

Task Parallel Library and Servers, Part 1: Introduction
http://bradwilson.typepad.com/blog/2012/04/tpl-and-servers-pt1.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+BradWilson+%28Brad+Wilson%29
Asynchronous (multi-threaded) programming is not easy.
Asynchronous programming with .NET 4 is a little easier.
Asynchronous programming with .NET 4.5 is a lot easier.
The async and await keywords available in C# and VB on .NET 4.5 mean you get to consume Task Parallel Library tasks with code that looks linear and synchronous, but is in actuality asynchronous, with none of the callback handlers or thread synchronization stuff getting in the way.

Lang.NEXT Trip Report
http://bartoszmilewski.com/2012/04/06/lang-next-trip-report/
The two topics that stood out were: functional programming and DSLs. It seems like every language tries to assimilate some functional elements in order to deal with parallelism…

Pervasive Parallelism in Scala
http://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2012/Pervasive-Parallelism-in-Scala

UC: A Set-Based Language for Data-Parallel Programming
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.38.6672

Dr.Dobb’s Go Parallel
http://www.drdobbs.com/go-parallel/

STXXL: Standard Template Library for Extra Large Data Sets.
http://stxxl.sourceforge.net/

Parallel Implementations
http://www.altdevblogaday.com/2011/11/22/parallel-implementations/
by John Carmack

Models for distributed parallelism
http://lambda-the-ultimate.org/node/4405

Referentially transparent nondeterminism
http://pchiusano.blogspot.com.ar/2011/06/referentially-transparent.html

Solving the Shakespeare Million Monkeys Problem in Real-time with Parallelism and SignalR
http://www.hanselman.com/blog/SolvingTheShakespeareMillionMonkeysProblemInRealtimeWithParallelismAndSignalR.aspx

OpenCL
http://en.wikipedia.org/wiki/OpenCL
Open Computing Language (OpenCL) is a framework for writing programs that execute across heterogeneous platforms consisting of central processing unit (CPUs), graphics processing unit (GPUs), and other processors.

The Flow Programming Language: An implicitly-parallelizing, programmer-safe language.
http://www.flowlang.net/

Thinkin’ About Async
http://msdn.microsoft.com/en-us/magazine/hh456392.aspx

Pause and Play with Await
http://msdn.microsoft.com/en-us/magazine/hh456403.aspx

Go-flavored JavaScript
http://www.johntantalo.com/blog/go-flavored-javascript/

Parallel bars
http://www.economist.com/node/18750706?story_id=18750706
Computing: Parallel programming, once an obscure niche, is the focus of increasing interest as “multicore” chips proliferate in ordinary PCs

Welcome to Apache Pig!
http://pig.apache.org/
Apache Pig is a platform for analyzing large data sets that consists of a high-level language for expressing data analysis programs, coupled with infrastructure for evaluating these programs. The salient property of Pig programs is that their structure is amenable to substantial parallelization, which in turns enables them to handle very large data sets.

Parallel Data Warehouse News and Hadoop Interoperability Plans
http://blogs.technet.com/b/dataplatforminsider/archive/2011/08/08/parallel-data-warehouse-news-and-hadoop-interoperability-plans.aspx

Parallel Processing Using the Map Reduce Programming Model
http://blog.diskodev.com/parallel-processing-using-the-map-reduce-prog

The HPCC Systems Platform
http://hpccsystems.com/
HPCC (High Performance Computing Cluster) is a massive parallel-processing computing platform that solves Big Data problems. The platform is now Open Source!

My Links
http://delicious.com/ajlopez/parallel

Keep tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

Parallel Programming: Links, News And Resources (2)

Previous Post
Next Post

More links about this interesting topic. I think parallel programming is the way to explore new algorithms, specially in artificial intelligence and distributed computing.

Dryad – Microsoft Research
http://research.microsoft.com/en-us/projects/Dryad/

Ville Tuulos on Big Data and Map/Reduce in Erlang and Python with Disco
http://www.infoq.com/interviews/tuulos-erlang-mapreduce
Ville Tuulos talks about Disco, the Map/Reduce framework for Python and Erlang, real-world data mining with Python, the advantages of Erlang for distributed and fault tolerant software, and more.

ICSE 2011: Danny Dig – Retrofitting Parallelism into a Sequential World
http://channel9.msdn.com/posts/ICSE-2011-Danny-Dig
Dr. Danny Dig is a Principal Investigator at the Universal Parallel Computing Research Center (UPCRC Illinois). UPCRC’s stated mission is to make parallel programming synonymous with programming.

Patterns for Parallel Programming: Understanding and Applying Parallel Patterns with the .NET Framework 4
http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=19222
A document providing an in-depth tour of implementing a variety of parallel patterns using the .NET Framework 4.

Haskell: First results from GHC’s new garbage collector
http://hackage.haskell.org/trac/ghc/blog/new-gc-preview
For a number of months now, we have been designing and developing a new garbage collector for GHC, with the primary goal of improving scaling on multicores.

Actor-based Programming
http://www.infoq.com/presentations/Actor-based-Programming
Jamie Ridgway explains what actors are, why we need them, what they are helpful for, the languages built around this programming paradigm, along with some demos showing actor-based apps.

Functional Programming eXchange 2011: Simon Peyton Jones on Managing parallelism: embrace diversity, but control side effects
http://skillsmatter.com/podcast/scala/talk-by-haskell-expert-simon-peyton-jones
If you want to program a parallel computer, it obviously makes sense to start with a computational paradigm in which parallelism is the default (ie functional programming), rather than one in which computation is based on sequential flow of control (the imperative paradigm). And yet, and yet … functional programmers have been singing this tune since the 1980s, but do not yet rule the world.

Articles on Parallel Programming with the .NET Framework 4
http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=12594
A set of articles that provide information on parallel programming with the .NET Framework 4.

Dynamic parallel async requests in MVC
http://petemontgomery.wordpress.com/2011/05/12/dynamic-parallel-async-requests-in-mvc/
Thanks to AsyncControllers, it’s quite easy to make several asynchronous requests in parallel to different web services, and have MVC gather the results.

GPars
http://gpars.codehaus.org/
Groovy Parallel Systems
The GPars framework offers Java developers intuitive and safe ways to handle Java or Groovy tasks concurrently. Leveraging the enormous flexibility of the Groovy programing language and building on proven Java technologies, we aim to make concurrent programming for multi-core hardware intuitive, robust and enjoyable.

InfoQ: Parallel Programming Patterns: Data Parallelism
http://www.infoq.com/presentations/Parallel-Programming-Patterns
Ralph Johnson presents several data parallelism patterns, including related Java, C# and C++ libraries from Intel and Microsoft, comparing it with other forms of parallelism such as actor programming.

Task Parallel Library: 5 of n – CodeProject
http://www.codeproject.com/KB/cs/Tasks5.aspx
A look into using the Task Parallel Library.

Mired in code: Easy parallel processing in Clojure
http://blog.mired.org/2011/03/easy-parallel-processing-in-clojure.html
For those not familiar with it, the eight queens problem is to place eight queens on a chess board such that none of them are attacking each other. Since queens attack in eight directions, this means that none of them are in the same row or column, and that none of them share a diagonal. It’s interesting, but not incredibly difficult, to do this.

W. Daniel Hillis
http://en.wikipedia.org/wiki/W._Daniel_Hillis
William Daniel “Danny” Hillis (born September 25, 1956, in Baltimore, Maryland) is an American inventor, entrepreneur, and author. He co-founded Thinking Machines Corporation, a company that developed the Connection Machine, a parallel supercomputer designed by Hillis at MIT. He is also co-founder of the Long Now Foundation, Applied Minds, Metaweb, and author of The Pattern on the Stone: The Simple Ideas That Make Computers Work.

Scala Team Wins ERC Grant | The Scala Programming Language
http://www.scala-lang.org/node/8579
The Scala research group at EPFL is excited to announce that they have won a 5 year European Research Grant of over 2.3 million Euros to tackle the “Popular Parallel Programming” challenge. This means that the Scala team will nearly double in size to pursue a truly promising way for industry to harness the parallel processing power of the ever increasing number of cores available on each chip.

Organizing Functional Code for Parallel Execution; or, foldl and foldr Considered Slightly Harmful
http://vimeo.com/6624203
Guy L. Steele, Jr.; Sun Microsystems
International Conference on Functional Programming (ICFP)
Edinburgh 2009
ACM SIGPLAN

InfoQ: How to Think about Parallel Programming: Not!
http://www.infoq.com/presentations/Thinking-Parallel-Programming
Guy L. Steele Jr. believes that it should not be the programmer’s job to think about parallelism, but languages should provide ways to transparently run tasks in parallel. This requires a new approach in building languages supporting algorithms built on independence and build-and-conquer principles rather than on linear decomposition of problems.

Parallel Programming with .NET
http://blogs.msdn.com/b/pfxteam/
All about Async/Await, System.Threading.Tasks, System.Collections.Concurrent, System.Linq, and more…

await anything;
http://blogs.msdn.com/b/pfxteam/archive/2011/01/13/10115642.aspx
One of the very cool things about the new await keyword in C# and Visual Basic is that it’s pattern based.  It works great with Task and Task<TResult>, and awaiting those two types will represent the vast majority of uses, but they’re by no means the only types that can be awaited.  The languages support awaiting any instance that exposes the right method (either instance method or extension method): GetAwaiter.

Professional Parallel Programming with C#: Master Parallel Extensions with .NET 4 (Wrox Programmer to Programmer)
http://www.amazon.com/Professional-Parallel-Programming-Master-Extensions/dp/0470495995

No Callbacks Required: StratifiedJS Returns Sequential Programming to Javascript
http://www.infoq.com/articles/stratifiedjs
Javascript is fundamentally single threaded, no parallel threads in sight. Asynchronous programming is necessary for any task that might block; Javascript programming usually involves creating and passing around a lot of callbacks, essentially forcing the developer to manually translate sequential code into continuation passing style.

Parallelism: In the Cloud, Cluster and Client
http://blogs.msdn.com/b/amit_chatterjee/archive/2010/12/19/parallelism-in-the-cloud-cluster-and-client.aspx
Software and digital experience is now becoming ubiquitous in our lives. Software is everywhere – it is stocking supermarket shelves, delivering electricity and water to our homes, storing our personal information in computers around the world, running nuclear plants and even controlling doomsday weapons! Moreover, with the popularity of the Internet and mobile devices, and with the advances in graphical and natural user interfaces, the creative human minds and the innovative companies are making the digital age more personal, social and indispensible.
This is leading to three fundamental trends in the world today:
Data explosion – the phenomenal growth of raw digital data
Demand for computational power for transforming raw data into meaningful information
Exponential growth in the number of users getting on to the digital super highway

My Links
http://delicious.com/ajlopez/parallel

More links are coming.

Keep tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

Parallel Programming: Links, News and Resources (1)

Next Post

Parallel programming, leveraging multicore processor, or even distributed machines, is an actual topic. Some of the links I read/collected about this topic.

Patterns for Parallel Programming: Understanding and Applying Parallel Patterns with the .NET Framework 4
http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=19222

ztellman’s calx at master – GitHub
https://github.com/ztellman/calx
Calx is an idiomatic wrapper for OpenCL, which is an abstraction layer for parallel computation.

Workshop on Parallelism, and a “breakthrough” in combinatorial geometry
http://geomblog.blogspot.com/2010/11/workshop-on-parallelism-and.html

Back to (Parallel) Basics: Don’t Block Your Threads, Make Async I/O Work For You
http://www.hanselman.com/blog/BackToParallelBasicsDontBlockYourThreadsMakeAsyncIOWorkForYou.aspx

The Renaissance Virtual Machine
http://soft.vub.ac.be/~smarr/rvm-open-source-release/

smarr’s RoarVM at master – GitHub
https://github.com/smarr/RoarVM
RoarVM, formerly known as the Renaissance Virtual Machine (RVM) is developed as part of a IBM Research project to investigate programming paradigms and languages for manycore systems of the future.

YouTube – Clojure Demo
http://www.youtube.com/watch?v=CFCYVfApPUc&feature=share

Thinking about massively parallel Smalltalk
http://wiki.squeak.org/squeak/537

InfoQ: Functional Approaches To Parallelism and Concurrency
http://www.infoq.com/presentations/Functional-Approaches-To-Parallelism

Threading in C# – Free E-book
http://www.albahari.com/threading/
C# supports parallel execution of code through multithreading. A thread is an independent execution path, able to run simultaneously with other threads.

Helios: Heterogeneous Multiprocessing with Satellite Kernels
http://research.microsoft.com/apps/pubs/default.aspx?id=81154

Threading in C# – Part 5 – Parallel Programming
http://www.albahari.com/threading/part5.aspx
In this section, we cover the multithreading APIs new to Framework 4.0 for leveraging multicore processors:These APIs are collectively known (loosely) as PFX (Parallel Framework).

CRDTs: Consistency without concurrency control
http://pagesperso-systeme.lip6.fr/Marc.Shapiro/papers/RR-6956.pdf

The Weekly Source Code 56 – Visual Studio 2010 and .NET Framework 4 Training Kit – Code Contracts, Parallel Framework and COM Interop
http://www.hanselman.com/blog/TheWeeklySourceCode56VisualStudio2010AndNETFramework4TrainingKitCodeContractsParallelFrameworkAndCOMInterop.aspx

J-Machine Project Page
http://cva.stanford.edu/projects/j-machine/
The J-Machine is a fine grained concurrent computer designed by the MIT Concurrent VLSI Architecture group (now located at Stanford University) in conjunction with Intel Corporation.

Parallel Programming with Hints
http://bartoszmilewski.com/2010/05/11/parallel-programming-with-hints/
Wouldn’t it be nice to be able to write sequential programs and let the compiler or the runtime automatically find opportunities for parallel execution?

Beyond Locks and Messages: The Future of Concurrent Programming
http://bartoszmilewski.com/2010/08/02/beyond-locks-and-messages-the-future-of-concurrent-programming/

Parallelism in .NET – Part 18, Task Continuations with Multiple Tasks
http://reedcopsey.com/2010/07/19/parallelism-in-net-part-18-task-continuations-with-multiple-tasks/

InfoQ: Multicore Programming in Haskell
http://www.infoq.com/presentations/Multicore-Programming-in-Haskell

Microsoft Biology Foundation 1.0 Released
http://blogs.msdn.com/b/pfxteam/archive/2010/07/12/10037496.aspx

Designing Parallel Algorithms: Part 1
http://drdobbs.com/go-parallel/article/showArticle.jhtml?articleID=223100878

Dr.Dobb’s Go Parallel
http://drdobbs.com/go-parallel/

DataAllure: Hadoop for DNA sequence analysis
http://blog.dataallure.com/2010/04/hadoop-for-dna-sequence-analysis.html

A Pattern Language for Parallel Programming
http://www.infoq.com/presentations/Pattern-Language-Parallel-Programming
Ralph Johnson presents a pattern language that he and his colleagues are working on in an attempt to solve the hard issues of parallel programming through a set of design patterns: Structural Patterns, Computational Patterns, Parallel Algorithm Strategy Patterns, Implementation Strategy Patterns, and Concurrent Execution Patterns.

Back to (Parallel) Basics: Do you really want to do that? or Why doesn’t the new Parallel.For support BigInteger?
http://www.hanselman.com/blog/BackToParallelBasicsDoYouReallyWantToDoThatOrWhyDoesntTheNewParallelForSupportBigInteger.aspx

InfoQ: Billy Newport Discusses Parallel Programming in Java
http://www.infoq.com/interviews/billy-newport-parallel

Multi-core Parallelization in Clojure – a Case Study
http://www.slideshare.net/adorepump/multicore-parallelization-in-clojure-a-case-study

Patterns of Parallel Programming: A Tutorial on Fundamental Patterns and Practices for Parallelism
http://www.microsoftpdc.com/2009/Patterns-of-Parallel-Programming

The Design of a Task Parallel Library
http://research.microsoft.com/pubs/77368/TheDesignOfATaskParallelLibrary(oopsla2009).pdf

My Links
http://delicious.com/ajlopez/parallel

Keep tuned!

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez