Category Archives: Parallel Programming

Parallel Programming: Links, News And Resources (3)

Previous Post

Concurrent Programming Using The Disruptor
Trisha Gee introduces Disruptor, a concurrency framework based on a data structure – a ring buffer – that enables fast message passing in a parallel environment.

Dataflow Programming: Handling Huge Data Loads Without Adding Complexity

Task Parallel Library and Servers, Part 1: Introduction
Asynchronous (multi-threaded) programming is not easy.
Asynchronous programming with .NET 4 is a little easier.
Asynchronous programming with .NET 4.5 is a lot easier.
The async and await keywords available in C# and VB on .NET 4.5 mean you get to consume Task Parallel Library tasks with code that looks linear and synchronous, but is in actuality asynchronous, with none of the callback handlers or thread synchronization stuff getting in the way.

Lang.NEXT Trip Report
The two topics that stood out were: functional programming and DSLs. It seems like every language tries to assimilate some functional elements in order to deal with parallelism…

Pervasive Parallelism in Scala

UC: A Set-Based Language for Data-Parallel Programming

Dr.Dobb’s Go Parallel

STXXL: Standard Template Library for Extra Large Data Sets.

Parallel Implementations
by John Carmack

Models for distributed parallelism

Referentially transparent nondeterminism

Solving the Shakespeare Million Monkeys Problem in Real-time with Parallelism and SignalR

Open Computing Language (OpenCL) is a framework for writing programs that execute across heterogeneous platforms consisting of central processing unit (CPUs), graphics processing unit (GPUs), and other processors.

The Flow Programming Language: An implicitly-parallelizing, programmer-safe language.

Thinkin’ About Async

Pause and Play with Await

Go-flavored JavaScript

Parallel bars
Computing: Parallel programming, once an obscure niche, is the focus of increasing interest as “multicore” chips proliferate in ordinary PCs

Welcome to Apache Pig!
Apache Pig is a platform for analyzing large data sets that consists of a high-level language for expressing data analysis programs, coupled with infrastructure for evaluating these programs. The salient property of Pig programs is that their structure is amenable to substantial parallelization, which in turns enables them to handle very large data sets.

Parallel Data Warehouse News and Hadoop Interoperability Plans

Parallel Processing Using the Map Reduce Programming Model

The HPCC Systems Platform
HPCC (High Performance Computing Cluster) is a massive parallel-processing computing platform that solves Big Data problems. The platform is now Open Source!

My Links

Keep tuned!

Angel “Java” Lopez

Parallel Programming: Links, News And Resources (2)

Previous Post
Next Post

More links about this interesting topic. I think parallel programming is the way to explore new algorithms, specially in artificial intelligence and distributed computing.

Dryad – Microsoft Research

Ville Tuulos on Big Data and Map/Reduce in Erlang and Python with Disco
Ville Tuulos talks about Disco, the Map/Reduce framework for Python and Erlang, real-world data mining with Python, the advantages of Erlang for distributed and fault tolerant software, and more.

ICSE 2011: Danny Dig – Retrofitting Parallelism into a Sequential World
Dr. Danny Dig is a Principal Investigator at the Universal Parallel Computing Research Center (UPCRC Illinois). UPCRC’s stated mission is to make parallel programming synonymous with programming.

Patterns for Parallel Programming: Understanding and Applying Parallel Patterns with the .NET Framework 4
A document providing an in-depth tour of implementing a variety of parallel patterns using the .NET Framework 4.

Haskell: First results from GHC’s new garbage collector
For a number of months now, we have been designing and developing a new garbage collector for GHC, with the primary goal of improving scaling on multicores.

Actor-based Programming
Jamie Ridgway explains what actors are, why we need them, what they are helpful for, the languages built around this programming paradigm, along with some demos showing actor-based apps.

Functional Programming eXchange 2011: Simon Peyton Jones on Managing parallelism: embrace diversity, but control side effects
If you want to program a parallel computer, it obviously makes sense to start with a computational paradigm in which parallelism is the default (ie functional programming), rather than one in which computation is based on sequential flow of control (the imperative paradigm). And yet, and yet … functional programmers have been singing this tune since the 1980s, but do not yet rule the world.

Articles on Parallel Programming with the .NET Framework 4
A set of articles that provide information on parallel programming with the .NET Framework 4.

Dynamic parallel async requests in MVC
Thanks to AsyncControllers, it’s quite easy to make several asynchronous requests in parallel to different web services, and have MVC gather the results.

Groovy Parallel Systems
The GPars framework offers Java developers intuitive and safe ways to handle Java or Groovy tasks concurrently. Leveraging the enormous flexibility of the Groovy programing language and building on proven Java technologies, we aim to make concurrent programming for multi-core hardware intuitive, robust and enjoyable.

InfoQ: Parallel Programming Patterns: Data Parallelism
Ralph Johnson presents several data parallelism patterns, including related Java, C# and C++ libraries from Intel and Microsoft, comparing it with other forms of parallelism such as actor programming.

Task Parallel Library: 5 of n – CodeProject
A look into using the Task Parallel Library.

Mired in code: Easy parallel processing in Clojure
For those not familiar with it, the eight queens problem is to place eight queens on a chess board such that none of them are attacking each other. Since queens attack in eight directions, this means that none of them are in the same row or column, and that none of them share a diagonal. It’s interesting, but not incredibly difficult, to do this.

W. Daniel Hillis
William Daniel “Danny” Hillis (born September 25, 1956, in Baltimore, Maryland) is an American inventor, entrepreneur, and author. He co-founded Thinking Machines Corporation, a company that developed the Connection Machine, a parallel supercomputer designed by Hillis at MIT. He is also co-founder of the Long Now Foundation, Applied Minds, Metaweb, and author of The Pattern on the Stone: The Simple Ideas That Make Computers Work.

Scala Team Wins ERC Grant | The Scala Programming Language
The Scala research group at EPFL is excited to announce that they have won a 5 year European Research Grant of over 2.3 million Euros to tackle the “Popular Parallel Programming” challenge. This means that the Scala team will nearly double in size to pursue a truly promising way for industry to harness the parallel processing power of the ever increasing number of cores available on each chip.

Organizing Functional Code for Parallel Execution; or, foldl and foldr Considered Slightly Harmful
Guy L. Steele, Jr.; Sun Microsystems
International Conference on Functional Programming (ICFP)
Edinburgh 2009

InfoQ: How to Think about Parallel Programming: Not!
Guy L. Steele Jr. believes that it should not be the programmer’s job to think about parallelism, but languages should provide ways to transparently run tasks in parallel. This requires a new approach in building languages supporting algorithms built on independence and build-and-conquer principles rather than on linear decomposition of problems.

Parallel Programming with .NET
All about Async/Await, System.Threading.Tasks, System.Collections.Concurrent, System.Linq, and more…

await anything;
One of the very cool things about the new await keyword in C# and Visual Basic is that it’s pattern based.  It works great with Task and Task<TResult>, and awaiting those two types will represent the vast majority of uses, but they’re by no means the only types that can be awaited.  The languages support awaiting any instance that exposes the right method (either instance method or extension method): GetAwaiter.

Professional Parallel Programming with C#: Master Parallel Extensions with .NET 4 (Wrox Programmer to Programmer)

No Callbacks Required: StratifiedJS Returns Sequential Programming to Javascript
Javascript is fundamentally single threaded, no parallel threads in sight. Asynchronous programming is necessary for any task that might block; Javascript programming usually involves creating and passing around a lot of callbacks, essentially forcing the developer to manually translate sequential code into continuation passing style.

Parallelism: In the Cloud, Cluster and Client
Software and digital experience is now becoming ubiquitous in our lives. Software is everywhere – it is stocking supermarket shelves, delivering electricity and water to our homes, storing our personal information in computers around the world, running nuclear plants and even controlling doomsday weapons! Moreover, with the popularity of the Internet and mobile devices, and with the advances in graphical and natural user interfaces, the creative human minds and the innovative companies are making the digital age more personal, social and indispensible.
This is leading to three fundamental trends in the world today:
Data explosion – the phenomenal growth of raw digital data
Demand for computational power for transforming raw data into meaningful information
Exponential growth in the number of users getting on to the digital super highway

My Links

More links are coming.

Keep tuned!

Angel “Java” Lopez

Parallel Programming: Links, News and Resources (1)

Next Post

Parallel programming, leveraging multicore processor, or even distributed machines, is an actual topic. Some of the links I read/collected about this topic.

Patterns for Parallel Programming: Understanding and Applying Parallel Patterns with the .NET Framework 4

ztellman’s calx at master – GitHub
Calx is an idiomatic wrapper for OpenCL, which is an abstraction layer for parallel computation.

Workshop on Parallelism, and a “breakthrough” in combinatorial geometry

Back to (Parallel) Basics: Don’t Block Your Threads, Make Async I/O Work For You

The Renaissance Virtual Machine

smarr’s RoarVM at master – GitHub
RoarVM, formerly known as the Renaissance Virtual Machine (RVM) is developed as part of a IBM Research project to investigate programming paradigms and languages for manycore systems of the future.

YouTube – Clojure Demo

Thinking about massively parallel Smalltalk

InfoQ: Functional Approaches To Parallelism and Concurrency

Threading in C# – Free E-book
C# supports parallel execution of code through multithreading. A thread is an independent execution path, able to run simultaneously with other threads.

Helios: Heterogeneous Multiprocessing with Satellite Kernels

Threading in C# – Part 5 – Parallel Programming
In this section, we cover the multithreading APIs new to Framework 4.0 for leveraging multicore processors:These APIs are collectively known (loosely) as PFX (Parallel Framework).

CRDTs: Consistency without concurrency control

The Weekly Source Code 56 – Visual Studio 2010 and .NET Framework 4 Training Kit – Code Contracts, Parallel Framework and COM Interop

J-Machine Project Page
The J-Machine is a fine grained concurrent computer designed by the MIT Concurrent VLSI Architecture group (now located at Stanford University) in conjunction with Intel Corporation.

Parallel Programming with Hints
Wouldn’t it be nice to be able to write sequential programs and let the compiler or the runtime automatically find opportunities for parallel execution?

Beyond Locks and Messages: The Future of Concurrent Programming

Parallelism in .NET – Part 18, Task Continuations with Multiple Tasks

InfoQ: Multicore Programming in Haskell

Microsoft Biology Foundation 1.0 Released

Designing Parallel Algorithms: Part 1

Dr.Dobb’s Go Parallel

DataAllure: Hadoop for DNA sequence analysis

A Pattern Language for Parallel Programming
Ralph Johnson presents a pattern language that he and his colleagues are working on in an attempt to solve the hard issues of parallel programming through a set of design patterns: Structural Patterns, Computational Patterns, Parallel Algorithm Strategy Patterns, Implementation Strategy Patterns, and Concurrent Execution Patterns.

Back to (Parallel) Basics: Do you really want to do that? or Why doesn’t the new Parallel.For support BigInteger?

InfoQ: Billy Newport Discusses Parallel Programming in Java

Multi-core Parallelization in Clojure – a Case Study

Patterns of Parallel Programming: A Tutorial on Fundamental Patterns and Practices for Parallelism

The Design of a Task Parallel Library

My Links

Keep tuned!

Angel “Java” Lopez