Effective Concurrency: Use Thread Pools Correctly – Keep Tasks Short and Nonblocking

This month’s Effective Concurrency column, “Use Thread Pools Correctly: Keep Tasks Short and Nonblocking”, is now live on DDJ’s website.

From the article:

… But the thread pool is a leaky abstraction. That is, the pool hides a lot of details from us, but to use it effectively we do need to be aware of some things a pool does under the covers so that we can avoid inadvertently hitting performance and correctness pitfalls. Here’s the summary up front:

1. Tasks should be small, but not too small, otherwise performance overheads will dominate.

2. Tasks should avoid blocking (waiting idly for other events, including inbound messages or contested locks), otherwise the pool won’t consistently utilize the hardware well — and, in the extreme worst case, the pool could even deadlock.

Let’s see why. …

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

Sharing Is the Root of All Contention (Mar 2009)

Use Threads Correctly = Isolation + Asynchronous Messages (Apr 2009)

“Use Thread Pools Correctly: Keep Tasks Short and Nonblocking” (Apr 2009)

2 thoughts on “Effective Concurrency: Use Thread Pools Correctly – Keep Tasks Short and Nonblocking

  1. 1. Task size:
    I agree that tasks should not be too small but IMO they don’t have to be small in general. As long as the thread pool is kept busy with enough tasks it does not matter if the tasks are small or large. Large tasks should remain large and should not be split into smaller pieces which would produce more overhead.

    It also doesn’t matter if the pool is running out of tasks because in this state the pool is inefficient anyway.
    If the remaining tasks are still divisible they can be split to get a better thread utilization. If not, it’s still the fastest way to run on a single thread.

    2. Blocking tasks
    If thread affinity is not an issue it’s possible to add blocking tasks to the thread pool if they are executed on fibers. But this requires a powerful Fiber API that provides a user mode scheduler and primitives for fiber synchronization.

    Coincidentally, I’ve already implemented such a fiber-based thread pool at http://www.fiberpool.de/en/index.html ;-) …but MS also announced a similar framework for Windows 7 and VS2010..


Comments are closed.