What is a high-performance data structure? To answer that question, we’re used to applying normal considerations like Big-Oh complexity, and memory overhead, locality, and traversal order. All of those apply to both sequential and concurrent software.

But in concurrent code, we need to consider two additional things to help us pick a data structure that is also sufficiently concurrency-friendly:

• In parallel code, your performance needs likely include the ability to allow multiple threads to use the data at the same time. If this is (or may become) a high-contention data structure, does it allow for concurrent readers and/or writers in different parts of the data structure at the same time? If the answer is, “No,” then you may be designing an inherent bottleneck into your system and be just asking for lock convoys as threads wait, only one being able to use the data structure at a time.
• On parallel hardware, you may also care about minimizing the cost of memory synchronization. When one thread updates one part of the data structure, how much memory needs to be moved to make the change visible to another thread? If the answer is, “More than just the part that has ostensibly changed,” then again you’re asking for a potential performance penalty, this time due to cache sloshing as more data has to move from the core that performed the update to the core that is reading the result.

It turns out that both of these answers are directly influenced by whether the data structure allows truly localized updates. …

Want to kill your parallel application’s scalability? Easy: Just add a dash of contention.

Locality is no longer just about fitting well into cache and RAM, but also about avoiding scalability busters by keeping tightly coupled data physically close together and separately used data far, far apart. …

Part 2 of DevX’s interview with me about concurrency just went live on the web. From the article’s blurb:

What does the future hold for concurrency? What will happen to the tools and techniques around concurrent programming? In part two of our series, concurrency guru Herb Sutter talks about these issues and what developers need to be reading to understand concurrency. 

… In this final installment he looks into his crystal ball with an eye towards the future and gives developers hints for the resources they need to be better concurrent programmers.

This part touches on a variety of topics, from right-now items like delivering parallelism internally inside libraries to shield the programmer from knowing about concurrency and where to look for further reading, to future topics like transactional memory and upcoming homogeneous vs. heterogeneous manycore CPUs. I hope you enjoy it.

(March’s part 1 is here.)

I recently spent an hour on the phone to talk concurrency with DevX’s Alexa Weber Morales. Part 1 of that interview just went live on the web, and focuses mostly on what concurrency and parallelism are, how to take advantage of multicore chips, and whether concurrency will ever be really accessible to mainstream developers. The site seems to be having intermittent problems displaying the pages; just hit the link a few more times if it doesn’t work right away.

Disclaimer: I am not responsible for the article title (yikes! “whisperer”?! my goodness gracious) and Alexa’s intro blurb is way too kind. But it is true that it’s important for us-the-industry to bring concurrency to the mainstream in a grokkable way, as we have already successfully done with OO and GUIs in the past.

Many of you have kindly sent mail about my Effective Concurrency columns and asking when there’ll be a course. Well, I’m happy to announce that the answer is: May 19-21, 2008.

Here’s the brief information (more details below):

3-Day Seminar: Effective Concurrency

May 19-21, 2008
Bellevue, WA, USA
Developed and taught by Herb Sutter

This course covers the fundamental tools that software developers need to write effective concurrent software for both single-core and multi-core/many-core machines. To use concurrency effectively, we must identify and solve four key challenges:

• Leverage the ability to perform and manage work asynchronously
• Build applications that naturally run faster on new hardware having more and more cores
• Manage shared objects in memory effectively to avoid races and deadlocks
• Engineer specifically for high performance

This seminar will equip attendees to reason correctly about concurrency requirements and tradeoffs, to migrate existing code bases to be concurrency-enabled, and to achieve key success factors for a concurrent programming project. Most code examples in the course can be directly translated to popular platforms and concurrency libraries, including Linux, Windows, Java, .NET, pthreads, and the forthcoming ISO C++0x standard.

Note on class size limit and possible waitlist: There is a hard limit on attendance at this first one (really). But if the registration site says you’ll get waitlisted, don’t give up: Go ahead and sign up anyway because we may be able to put together a second installment of the seminar a week or two later if there’s enough interest.

Finally, here’s a summary of what we’ll cover during the three days.

Fundamentals

• Define basic concurrency goals and requirements
• Understand applications’ scalability needs
• Key concurrency patterns

Isolation: Keep Work Separate

• Running tasks in isolation and communicate via async messages
• Integrating multiple messaging systems, including GUIs and sockets
• Building responsive applications using background workers
• Threads vs. thread pools

Scalability: Re-enable the Free Lunch

• When and how to use more cores 
• Exploiting parallelism in algorithms 
• Exploiting parallelism in data structures 
• Breaking the scalability barrier

Consistency: Don’t Corrupt Shared State

• The many pitfalls of locks–deadlock, convoys, etc.
• Locking best practices
• Reducing the need for locking shared data
• Safe lock-free coding patterns
• Avoiding the pitfalls of general lock-free coding
• Races and race-related effects

Migrating Existing Code Bases to Use Concurrency

Near-Future Tools and Features

High Performance Concurrency

• Machine architecture and concurrency
• Costs of fundamental operations, including locks, context switches, and system calls
• Memory and cache effects
• Data structures that support and undermine concurrency
• Enabling linear and superlinear scaling

I hope to get to meet some of you here in the Seattle area!

There are two main ways to achieve superlinear scalability, or to use P processors to compute an answer more than P times faster…:

• Do disproportionately less work.
• Harness disproportionately more resources.

Last month, we focused on the first point by illustrating parallel search and how it naturally achieves superlinear speedups when matches are not distributed evenly because some workers get "rich" subranges and will find a match faster, which benefits the whole search because we can stop as soon as any worker finds a match.

This month, we’ll conclude examining the first point with a few more examples, and then consider how to achieve superlinear speedups by harnessing more resources—quite literally, running on a bigger machine without any change in the hardware. …