The Concurrency Land Rush: 2007-20??

Every time that we experience a "wave" in which the industry takes a programming paradigm that’s been growing in a niche, and brings it to the mainstream, we go through similar phases:

  • A land rush phase during which vendors try to stake out their turf. The market sees an explosive proliferation of products trying to enable mainstream developers to work in the new paradigm and at the same time to differentiate themselves from each other. This phase always takes multiple years, and multiple major releases of many products, consumed at first mainly by early adopters and then, as products mature, broadly across the marketplace.
  • A shakeout phase as, eventually, the market sifts through the products, selects major winners, and realigns itself around them.

We (the software industry) have done it all before for technologies like OO and GUIs, and now we’re going to do it for concurrency.

Starting in 2005, companies started to talk concurrency and to launch internal efforts. But only this year, in 2007, has our industry formally entered the concurrency wave’s land rush phase by starting to field real products and preview releases. We’re starting to see initial steps toward candidate mainstream-accessible products in releases like Intel’s Threading Building Blocks and Software Transactional Memory prototype compiler, and Microsoft’s just-announced and downloadable .NET Parallel Extensions (PFX) including CTP-level support for Parallel LINQ (PLINQ) and the Task Parallel Library.

True, the concurrency landscape has lots of existing tools. So did OO (e.g., Simula) and GUIs (e.g., the Win16 C API) long before they became widely accessible to developers in general. Concurrency’s "Simulas and Win16s" include Thread classes and OpenMPI for doing work asynchronously, to OpenMP for fast parallel computation, to some pre-fab lock-free container classes, to limited race detectors and very basic thread profilers. But today’s tools are still what you’d expect to see in a niche market: Good enough for highly-trained experts to deliver real products, but not easy enough and not integrated enough across the software stack and tool chains to be broadly adopted by mainstream developers.

Just as the OO mainstream land rush was fueled by new and updated tools like Smalltalk and Objective-C and C++ and Java offerings, and the GUI land rush was fueled by MFC and OWL and Cocoa and Visual Basic and Delphi, we’re starting to see new and existing companies bringing lots of new and updated offerings across the entire software stack and ecosystem. Expect to see lots of new and updated/extended offerings of:

  • Languages and frameworks
  • Runtimes and operating systems
  • Debuggers and profilers and program analysis tools
  • Component wizards and IDE integration
  • Books and courses and conferences

These are the just beginning. Much more will come, from those same companies and others. Expect at least dozens of major product announcements and releases across the industry, before the toolset expansion phase is fully underway and approaching some maturity. We the industry have undertaken to bring concurrency to the mainstream, and as with OO and GUIs it will take multiple years, and multiple major releases, across the industry on all platforms.

As I write this, Seattle’s first snowfall of the season is gently descending outside my window. It readily puts one in mind of Frost: We have promises to keep, and miles to go before we sleep.