“256 cores by 2013”?

I just saw a tweet that’s worth commenting on:


Almost right, and we have already reached that.

I said something similar to the above, but with two important differences:

  1. I said hardware “threads,” not only hardware “cores” – it was about the amount of hardware parallelism available on a mainstream system.
  2. What I gave was a min/max range estimate of roughly 16 to 256 (the latter being threads) under different sets of assumptions.

So: Was I was right about 2013 estimates?

Yes, pretty much, and in fact we already reached or exceeded that in 2011 and 2012:

  • Lower estimate line: In 2011 and 2012 parts, Intel Core i7 Sandy Bridge and Ivy Bridge are delivering almost the expected lower baseline, and offering 8-way and 12-way parallelism = 4-6 cores x 2 hardware threads per core.
  • Upper estimate line: In 2012, as mentioned in the article (which called it Larrabee, now known as MIC or Xeon Phi) is delivering 200-way to 256-way parallelism = 50-64 cores x 4 hardware threads per core. Also, in 2011 and 2012, GPUs have since emerged into more mainstream use for computation (GPGPU), and likewise offer massive compute parallelism, such as 1,536-way parallelism on a machine having a single NVidia Tesla card.

Yes, mainstream machines do in fact have examples of both ends of the “16 to 256 way parallelism” range. And beyond the upper end of the range, in fact, for those with higher-end graphics cards.

For more on these various kinds of compute cores and threads, see also my article Welcome to the Jungle.


Longer answer follows:

Here’s the main part from article, “Design for Manycore Systems” (August 11, 2009). Remember this was written over three years ago – in the Time Before iPad, when Android was under a year old:

How Much Scalability Does Your Application Need?

So how much parallel scalability should you aim to support in the application you‘re working on today, assuming that it’s compute-bound already or you can add killer features that are compute-bound and also amenable to parallel execution? The answer is that you want to match your application’s scalability to the amount of hardware parallelism in the target hardware that will be available during your application’s expected production or shelf lifetime. As shown in Figure 4, that equates to the number of hardware threads you expect to have on your end users’ machines.

Figure 4: How much concurrency does your program need in order to exploit given hardware?

Let’s say that YourCurrentApplication 1.0 will ship next year (mid-2010), and you expect that it’ll be another 18 months until you ship the 2.0 release (early 2012) and probably another 18 months after that before most users will have upgraded (mid-2013). Then you’d be interested in judging what will be the likely mainstream hardware target up to mid-2013.

If we stick with "just more of the same" as in Figure 2’s extrapolation, we’d expect aggressive early hardware adopters to be running 16-core machines (possibly double that if they’re aggressive enough to run dual-CPU workstations with two sockets), and we’d likely expect most general mainstream users to have 4-, 8- or maybe a smattering of 16-core machines (accounting for the time for new chips to be adopted in the marketplace). [[Note: I often get lazy and say “core” to mean all hardware parallelism. In context above and below, it’s clear we’re talking about “cores and threads.”]]

But what if the gating factor, parallel-ready software, goes away? Then CPU vendors would be free to take advantage of options like the one-time 16-fold hardware parallelism jump illustrated in Figure 3, and we get an envelope like that shown in Figure 5.

Figure 5: Extrapolation of “more of the same big cores” and “possible one-time switch to 4x smaller cores plus 4x threads per core” (not counting some transistors being used for other things like on-chip GPUs).

First, let’s look at the lower baseline, ‘most general mainstream users to have [4-16 way parallelism] machines in 2013’? So where are were in 2012 today for mainstream CPU hardware parallelism? Well, Intel Core i7 (e.g., Sandy Bridge, Ivy Bridge) are typically in the 4 to 6 core range – which, with hyperthreading == hardware threads, means 8 to 12 hardware threads.

Second, what about the higher potential line for 2013? As noted above:

  • Intel’s Xeon Phi (then Larrabee) is now delivering 50-64 cores x 4 threads = 200 to 256-way parallelism. That’s no surprise, because this article’s upper line was based on exactly the Larrabee data point (see quote below).
  • GPUs already blow the 256 upper bound away – any machine with a two-year-old Tesla has 1,536-way parallelism for programs (including mainstream programs like DVD encoders) that can harness the GPU.

So not only did we already reach the 2013 upper line early, in 2012, but we already exceeded it for applications that can harness the GPU for computation.

As I said in the article:

I don’t believe either the bottom line or the top line is the exact truth, but as long as sufficient parallel-capable software comes along, the truth will probably be somewhere in between, especially if we have processors that offer a mix of large- and small-core chips, or that use some chip real estate to bring GPUs or other devices on-die. That’s more hardware parallelism, and sooner, than most mainstream developers I’ve encountered expect.

Interestingly, though, we already noted two current examples: Sun’s Niagara, and Intel’s Larrabee, already provide double-digit parallelism in mainstream hardware via smaller cores with four or eight hardware threads each. "Manycore" chips, or perhaps more correctly "manythread" chips, are just waiting to enter the mainstream. Intel could have built a nice 100-core part in 2006. The gating factor is the software that can exploit the hardware parallelism; that is, the gating factor is you and me.

Podcast: Interview on Hanselminutes

imageA few weeks ago at the Build conference, Scott Hanselman and I sat down to talk about C++ and modern UI/UX. The podcast is now live here:

The Hanselminutes Podcast, Show #346
“Why C++” with Herb Sutter

Topics Scott raises include:

  • 2:00 Scott mentions he has used C++ in the past. C++ has changed. We still call it C++, but it’s a very different language now.
  • 5:30 (Why) do we care about performance any more?
  • 10:00 What’s this GPGPU thing? Think of your GPU as your modern 80387.
  • 13:45 C++ is having a resurgence. Where is C++ big?
  • 18:00 Why not just use one language? or, What is C++ good at? Efficient abstraction and portability.
  • 21:45 Programmers have a responsibility to support the business. Avoid the pitfall of speeds & feeds.
  • 24:00 My experience with my iPad, my iPhone, and my Slate 7 with Win8.
  • 28:45 We’re in two election seasons – (a) political and (b) technology (Nexus, iPad Mini, Surface, …). Everyone is wallpapering the media with ads (some of them attack ads), and vying for customer votes/$$, and seeing who’s going to be the winner.
  • 35:00 Natural user interfaces – we get so easily used to touch that we paw all screen, and Scott’s son gets so used to saying “Xbox pause” that anything that doesn’t respond is “broken.”

Reader Q&A: A good book to learn C++11?

Last night a reader asked one of the questions that helped motivate the creation of isocpp.org:

I am trying to learn the new C++. I am wondering if you are aware of resources or courses that can help me learn a little. I was not able to find any books for C++11. Any help would be greatly appreciated.

By the way, the isocpp website is great :)

Thanks! And you beat me to the punchline. :)

A good place to start is isocpp.org/get-started. It recommends four books, three of which have been updated for C++11 (and two of those three are available now). C++ Primer 5e is a good choice.

We also maintain a list of new articles and books under isocpp.org/blog/category/articles-books.

As for courses, there’s also a growing list of good courses there under isocpp.org/blog/category/training, including two C++11 overview courses by Scott Meyers and Dave Abrahams respectively.

As with all blog categories, you have three ways to find the most recent items under Articles & Books, Training, and other categories:

  • Home page: The most recent items in each category are always listed right on the site’s newspaper-inspired home page.
  • RSS: You can subscribe to RSS feeds for all posts, or for specific categories to get notifications of new articles, books, training, events, and more as they become available.
  • Twitter: Follow @isocpp for notifications of new stuff.

The site and blog are curated, aiming for higher quality and lower volume. Even so, we know you won’t be able to read, watch, or listen to all of the content. So we’re doing our best to make sure that when you do pick something recommended by the site, it’s likely to be of high quality and address what you’re looking to find.

Enjoy! This is version 1 of the site… I hope that it’s already useful, and you’ll see quite a bit more over the coming year.

Friday’s Q&A session now online

imageMy live Q&A after Friday’s The Future of C++ talk is now online on Channel 9. The topics revolved around…

… recent progress and near-future directions for C++, both at Microsoft and across the industry, and talks about some announcements related to C++11 support in VC++ 2012 and the formation of the Standard C++ Foundation.

Herb takes questions from a live virtual audience and demos the new http://isocpp.org site on an 82 inch Perceptive Pixel display attached to a Windows 8 machine.

Thanks to everyone who tuned in.

Our industry is young again, and it’s all about UI

Jeff Atwood’s post two days ago inspired me to write this down. Thanks, Jeff.

“I can’t even remember the last time I was this excited about a computer.”

Jeff Atwood, November 1, 2012

Our industry is young again, full of the bliss and sense of wonder and promise of adventure that comes with youth.

Computing feels young and fresh in a way that it hasn’t felt for years, and that has only happened to this degree at two other times in its history. Many old-timers, including myself, have said “this feels like 1980 again.”

It does indeed. And the reason why is all about user interfaces (UI).

Wave 1: Late 1950s through 60s

First, computing felt young in the late 1950s through the 60s because it was young, and made computers personally available to a select few people. Having computers at all was new, and the ability to make a machine do things opened up a whole new world for a band of pioneers like Dijkstra and Hoare, and Russell (Spacewar!) and Engelbart (Mother of All Demos) who made these computers personal for at least a few people.

The machines were useful. But the excitement came from personally interacting with the machine.

Wave 2: Late 1970s through 80s

Second, computing felt young again in the late 1970s and 80s. Then, truly personal single-user computers were new. They opened up to a far wider audience the sense of wonder that came with having a computer of our very own, and often even with a colorful graphical interface to draw us into its new worlds. I’ll include Woods and Crowther (ADVENT) as an example, because they used a PDP as a personal computer (smile) and their game and many more like it took off on the earliest true PCs – Exidy Sorcerers and TRS-80s, Ataris and Apples. This was the second and much bigger wave of delivering computers we could personally interact with.

The machines were somewhat useful; people kept trying to justify paying $1,000 for one “to organize recipes.” (Really.) But the real reason people wanted them was that they were more intimate – the excitement once again came from personally interacting with the machine.

Non-wave: 1990s through mid-2000s

Although WIMP interfaces proliferated in the 1990s and did deliver benefits and usability, they were never as exciting to the degree computers were in the 80s. Why not? Because they weren’t nearly as transformative in making computers more personal, more fun. And then, to add insult to injury, once we shipped WIMPiness throughout the industry, we called it good for a decade and innovation in user interfaces stagnated.

I heard many people wonder whether computing was done, whether this was all there would be. Thanks, Apple, for once again taking the lead in proving them wrong.

Wave 3: Late 2000s through the 10s

Now, starting in the late 2000s and through the 10s, modern mobile computers are new and more personal than ever, and they’re just getting started. But what makes them so much more personal? There are three components of the new age of computing, and they’re all about UI (user interfaces)… count ’em:

  1. Touch.
  2. Speech.
  3. Gestures.

Now don’t get me wrong, these are in addition to keyboards and accurate pointing (mice, trackpads) and writing (pens), not instead of them. I don’t believe for a minute that keyboards and mice and pens are going away, because they’re incredibly useful – I agree with Joey Hess (HT to @codinghorror):

“If it doesn’t have a keyboard, I feel that my thoughts are being forced out through a straw.”

Nevertheless, touch, speech, and gestures are clearly important. Why? Because interacting with touch and speech and gestures is how we’re made, and that’s what lets these interactions power a new wave of making computers more personal. All three are coming to the mainstream in about that order…

Four predictions

… and all three aren’t done, they’re just getting started, and we can now see that at least the first two are inevitable. Consider:

Touchable screens on smartphones and tablets is just the beginning. Once we taste the ability to touch any screen, we immediately want and expect all screens to respond to touch. One year from now, when more people have had a taste of it, no one will question whether notebooks and monitors should respond to touch – though maybe a few will still question touch televisions. Two years from now, we’ll just assume that every screen should be touchable, and soon we’ll forget it was ever any other way. Anyone set on building non-touch mainstream screens of any size is on the wrong side of history.

Speech recognition on phones and in the living room is just the beginning. This week I recorded a podcast with Scott Hanselman which will air in another week or two, when Scott shared something he observed firsthand in his son: Once a child experiences saying “Xbox Pause,” he will expect all entertainment devices to respond to speech commands, and if they don’t they’re “broken.” Two years from now, speech will probably be the norm as one way to deliver primary commands. (Insert Scotty joke here.)

Likewise, gestures to control entertainment and games in the living room is just the beginning. Over the past year or two, when giving talks I’ve sometimes enjoyed messing with audiences by “changing” a PowerPoint slide by gesturing in the air in front of the screen while really changing the slide with the remote in my pocket. I immediately share the joke, of course, and we all have a laugh together, but the audience members more and more often just think it’s a new product and expect it to work. Gestures aren’t just for John Anderton any more.

Bringing touch and speech and gestures to all devices is a thrilling experience. They are just the beginning of the new wave that’s still growing. And this is the most personal wave so far.

This is an exciting and wonderful time to be part of our industry.

Computing is being reborn, again; we are young again.

Talk now online: The Future of C++ (VC++, ISO C++)

imageYesterday, many thousands of you were in the room or live online for my talk on The Future of C++. The talk is now available online.

This has been a phenomenal year for C++, since C++11’s publication just 12 months ago. And yesterday was a great day for C++.

Yesterday I had the privilege of announcing much of what Microsoft and the industry have been working on over the past year.

(minor) C++ at Microsoft

On September 12, we shipped VC++ 2012 with the complete C++11 standard library, and adding support for C++11 range-for, enum class, override and final. Less than two months later, yesterday we announced and shipped the November 2012 CTP, a compiler add-in to VC++ 2012 adding C++11 variadic templates, uniform initialization and initializer_lists, delegating constructors, function template default arguments, explicit conversion operators, and raw string literals. Details here, and download here.

Note that this is just the first batch of additional C++11 features. Expect further announcements and deliveries in the first half of 2013.

(major) C++ across the industry

Interest and investment in C++ continues to accelerate across the software world.

  • ISO C++ standardization is accelerating. Major companies are dedicating more people and resources to C++ standardization than they have in years. Over the next 24 months, we plan to ship three Technical Specifications and a new C++ International Standard.
  • C++ now has a home on the web at isocpp.org. Launched yesterday, it both aggregates the best C++ content and hosts new content itself, including Bjarne Stroustrup’s new Tour of C++ and Scott Meyers’ new Universal References article.
  • We now have a Standard C++ Foundation. Announced yesterday, it is already funded by the largest companies in the industry down to startups, financial institutions to universities, book publishers to other consortia, with more members joining weekly. For the first time in C++’s history since AT&T relinquished control of the language, we have an entity – a trade organization – that exists exclusively to promote Standard C++ on all compilers and platforms, and companies are funding it because the world runs on C++, and investing in Standard C++ is good business.

This is an exciting time to be part of our industry, on any OS and using any language. It’s especially an exciting time to be involved with C++ on all compilers and platforms.

Thank you all, whatever platform and language you use, for being part of it.