A few days ago at UIUC, Ralph Johnson gave a very nice talk on “Parallel Programming Patterns.” It’s now online, and here’s the abstract:
Parallel programming is hard. One proposed solution is to provide a standard set of patterns. Learning the patterns would help people to become expert parallel programmers. The patterns would provide a vocabulary that would let programmers think about their programs at a higher level than the programming language. The patterns could steer programmers away from common errors and towards good design principles.
There have been a number of papers about parallel patterns, and one book Patterns for Parallel Programming. None of them have become popular. I think the problem is that parallel programming is diverse and requires more design expertise than traditional software design. Thus, parallel programming experts use more patterns than parallel programming expert. I’ll critique the existing patterns and explain what I think should be done to make a set of patterns that can be as effective for parallel programming as patterns have been for object-oriented design.
If Johnson’s name sounds familiar, it should: He’s one of the “Gang of Four” authors of the seminal book Design Patterns.
Recommended viewing.
Sutter’s Mill 
Hi,
I think this is a wrong place but didn’t know where to ask for. Is there any way i can download this talk for offline viewing? I tried but couldn’t find flv for this talk.
thanks a lot for any help on this.
David: It’s not as odd as it seems. If you think about it, C++ (and even C) have long had functional programming capabilities in the form of function pointers (and later functors). The new developments merely make the syntax tighter.
Functional programming and C++ seem rather an unlikely mix. My guess is that if functional programming becomes more widespread then it will be in the context of a language that is more suited.
Re functional programming: See also my comments about this in my ACM Queue article with Jim Larus, “Software and the Concurrency Revolution”, where page 5 has a section on “Imperative and functional languages.” Functional languages also have other concurrency benefits besides immutable state, but to speak to that one briefly: It has been said that there are two kinds of functional languages — “pure” functional languages where all state is immutable and you don’t need to worry about locks at all, and “impure” ones that people actually use. Immutability is a very important concept and great to apply where it makes sense, as Java and .NET do with Strings for example (the idea of immutability isn’t limited to functional languages after all), but you’re not going to take a copy of a million-element vector every time you update the value of one of its elements.
Thanks for the kind words about Exceptional C++. As you probably noticed from my recent columns, a book on Effective Concurrency is coming… in 2009 if I keep on schedule. :-) In the meantime do check out the EC columns.
Folks, could you please recommend any good book on concurrency stuff? Seriously, I can’t find any on Amazon, they are all either too specific(e.g pthreads usage) or too abstract. I wish there were books on concurrency of the same great value and quality as Exceptional C++ ones. Herb, do you have any plans on that? ;)
Yea, and boost::lambda, tr1::function et al. The key is, as you pointed out you, to be side-effect free (i.e., purely functional). In C++, this is all still very much explicit though.
I would prefer to see a runtime that can drop bits of logic where it sees fit dynamically based on run-time statistics. [This seems to me fairly easy to do in a language like Haskell although for some reason as yet undiscovered to me, it doesn’t do so. Haskell still requires a little ‘hint’ something akin to the parallel_for loop of C# and C++. It also seems to me to be possible to do this automatically in a language like Python, but it doesn’t do such tunings for you either (at least not without specifying additional syntax)].
When I think of the term “functional programming” I think of stateless programming. Are you talking about things like boost::bind?
It’s nice to have them named, I agree. But I think there’s going to be quite a bit fewer patterns for parallelism than for software architecture in general.
And, my second point still remains– I really see functional programming as the way forward. There really shouldn’t need to be so much explicitness with parallelism (as with the new syntaxes being developed for C++ in particular).
Jon, patterns aren’t meant to be ground breaking; they just give a name to the obvious, which makes it easier to talk about different designs.
In conversation, I’d like to be able to just say “It’s like the task pool pattern” rather than having to explain the inner details of a nameless idea.
If this is anything like ‘design patterns’, I wouldn’t expect anything all that groundbreaking. The design ‘patterns’ arise quite naturally, independently and spontaneously for those of us in the day-to-day practice of engineering. And further, I doubt that the problems of parallel programming are as suited to this sort of playbook as software architecture in general is. For parallellism, I would recommend readers learn functional programming techniques (even if they are stuck using imperative [or multi-paradigm] languages).
Thanks, this is something I have been exploring but its been hard to find lectures on. Appreciate you spreading the word.