Guru of the Week and the Exceptional C++ Series

It’s time for me to pick up Guru of the Week (GotW) again in earnest, as part of work on revising my three Exceptional C++ books for today’s C++. Most Exceptional C++ Items are enhanced versions of GotW issues, after all, so the simplest and best place to start is with GotW. It’s also much easier to write (and read) one short piece at a time.

Last spring I wrote a few GotWs, but life got busy with the Standard C++ Foundation, isocpp.org, and C++14. I was also trying to figure out what reasonable tools to use to write new Items, since I wanted to publish the same Item on this blog and then in e-book and dead tree editions, all with a single source while maintaining a pleasing format. I’ve found a provisional promising tool (Word!) and format for that I’ll try out for now, and you’ll see it in the next GotW issue to be posted soon.

But a major issue was figuring out the right balance between updating existing GotW issues and writing new ones.

Here’s where I landed…

 

First, I’ve decided to keep and revise nearly all of my existing Items. This is not an obvious choice: As a counterexample, some authors have decided to not revise their existing books, but to write new books about C++11, for various reasons including that they expect the advice in the earlier books is still current. I think the latter is partly true, because of course the topics are still current and the code still compiles – C++11 has a great backward compatibility story with C++98.

However, when I looked through my own articles and Items, every single one I looked at had two qualities:

  • The topic was still current and useful, so it should still be covered. (With very rare exceptions like material discussing the now-removed export template feature.)
  • The code and discussion were addressed with C++98-era advice that is now dated and/or incomplete.

For example, nearly every GotW’s code should just naturally use auto and { } initialization. Most should use or actively discuss move semantics, lambdas, and other C++11 features. And it’s hard to write about existing features like virtual function overriding (as in GotW #5) without feeling something important is missing unless there’s also discussion of the way C++ has extended the existing feature, such as with override and final.

 

Second, I’ve decided not to target C++11 – rather, I’m going to write for C++14. Why would I do that, when C++ compilers [preemptive snarky remark: including my own team’s] are still catching up to C++11? For several good reasons:

  • C++14 is going to be current C++ soon. As of April, it’s now feature-complete, and the detailed technical specification should be frozen this fall or winter (leaving time for ISO ballots and “’14” publication). Compilers and libraries are already aggressively implementing it. I wouldn’t be surprised if we had multiple fully-conforming C++14 implementations by the end of next year.
  • C++14 is a small delta, a minor release aimed mainly at “completing C++11.” And it does complete C++11 well, to the point where as an author I really want to teach the complete story. I want to teach the complete “avoid writing new and delete” guidance that C++14 make_unique fully enables so that I can stop mumbling “except when you create a unique_ptr” as a special case. I want to write generic lambdas with C++14’s terse auto parameters that makes lambdas shorter to write and more flexible, and stop repeating parameter type names. I want to use C++14 move-capture in lambdas that makes them complete, and not feel obliged to talk about artificially putting objects on the heap and capturing shared_ptrs to them by value as a workaround for lack of move capture. And, in a few places where appropriate, I want to be free to use a few smaller C++14 tidbits like optional<T> that can affect interface design guidance now that they’re in the draft standard.

In addition, I’ve found that C++11/14 is changing my coding style, including my basic variable declaration style. I now think auto should be used consistently to declare local variables, even when you want to commit to a specific type(!) – this certainly surprised me, and likely will surprise you too, so I’ll cover that this year in a GotW and a recorded talk, no later than at C++ and Beyond later this year and possibly sooner.

 

I’ve converged on the above by updating the first dozen Exceptional C++ Items (in GotW order), trying out alternatives and seeing how well each one worked out. Those first dozen Items are now done and I’m pleased with the result, showing off auto and move and generic lambdas and mutexes/atomics and the new meanings of const and mutable and everything else, most of their code tested against various of the latest compilers, their advice updated to be current with current C++. And in every single GotW, no matter how short, it just felt right to:

  • keep the topic, because each topic is still as relevant and useful as ever;
  • revise the C++98 code and advice, because in every case both the code examples and the discussion was dated and/or incomplete, usually both; and
  • skip C++11 to write directly for the complete story in C++14 without apology, because it let me avoid digressions for workarounds not needed in C++14, which made the text cleaner and simpler.

As I’ve revised Items, I’ve found that some updates are small but pervasive, such as using auto. Others are extensive, with entire subsections completely rewritten, thrown out as no longer relevant, or newly added to cover essential new sides of the story in modern C++. For example, GotW #6 about const-correctness is now a two-parter, covering the new meanings of const and mutable. Watch for it soon.

 

A word about numbering and sequence: My plan is to update and post most GotWs in numerical order starting with GotW #1, but every so often we’ll jump forward to new high numbers (starting with #89) as I create ones that weren’t originally a GotW – a Sutter’s Mill magazine article that wasn’t originally published as a GotW, or a brand-new C++14 topic. So for example you might see a sequence like “… 23, 24, 91, 25, 26, …”. All GotWs will have substantial new material, but the newly minted high numbers will be entirely on brand-new topics that didn’t arise in C++98 or just weren’t covered in GotW form before. (The handful of GotWs I wrote last year will be renumbered as they’re folded into the original series, either as updates or with new numbers in the original sequence if they’re on completely new topics. )

 

I’ll start posting fresh C++11/14 GotWs in the next few days. As usual, I’ll first post a problem by itself to invite discussion about possible solutions, then after there’s been time for comments I’ll post my own solution and the next problem.

It’s kind of exciting to write “GotW14” – today’s C++ really does feel like a fresh new language.

I hope you enjoy them as much as I enjoy writing them.

Trip Report: ISO C++ Spring 2013 Meeting

wg21-attendanceThe Bristol meeting concluded a few hours ago, and I just posted my trip report on isocpp.org:

This afternoon in Bristol, UK, the ISO C++ standards committee adopted generic lambdas, dynamic arrays (an improved version of C99 VLAs), variable templates, reader/writer locks, make_unique, optional<T>, standard library user-defined literals, and a number of other language and library improvements – and approved the result as the feature-complete Committee Draft (CD) of Standard C++14 to be distributed for its primary international review ballot.

In addition to completing the C++14 CD document, the committee also made progress on three additional important parallel specifications that are on track to be published around the same time as C++14:

  • File system library (draft), based on Boost.FileSystem version 3.
  • Networking library, small at first and regularly extended.
  • “Concepts Lite” language extensions (draft), to express template constraints and improve template usability and error messages.

Together these mark the C++ committee’s main planned deliverables for 2014. …

To provide just a sampling [of C++14], here are a few quick examples of some of the newly added features…

Continue reading…

Complex initialization for a const variable

On std-discussion, Shakti Misra asked:

I have seen in a lot of places code like
int i;
if(someConditionIstrue)
{
Do some operations and calculate the value of i;
i = some calculated value;
}
use i; //Note this value is only used not changed. It should not be changed.
But unfortunately in this case there is no way to guarantee it.
so now if some one comes and does

i = 10;// This is valid

What i was thinking: is there a way to tell that "i" can be set only once?
After that it will be a constant. Something like

const once int i;
if(someConditionIstrue)
{
Do some operations and calculate the value of i;
i = calculated value;
}
use i; //Note this value is only used not changed.
i = 10;// Compiler error

Olaf van der Spek replied:

A lambda?

I answered as follows:

Bingo. Olaf nailed it: The way to do it is with a lambda. This is one of the examples I give in my talk Lambdas, Lambdas Everywhere (here are the slides from C++ & Beyond 2010).

In your example, do this:

 

const int i = [&]{

    int i = some_default_value;

    if(someConditionIstrue)
    {
        Do some operations and calculate the value of i;
        i = some calculated value;
    }

    return i;

} (); // note: () invokes the lambda!

 

That’s the C++11 idiom for this. It’s still being absorbed though – not everyone knows about it yet as we’re still all learning C++11 styles together.

Words of wisdom: Bjarne Stroustrup

Bjarne Stroustrup wrote the following a few minutes ago on the concepts mailing list:

Let me take this opportunity to remind people that

  • "being able to do something is not sufficient reason for doing it" and
  • "being able to do every trick is not a feature but a bug"

For the latter, remember Dijkstra’s famous "Goto considered harmful" paper. The point was not that the "new features" (loop constructs) could do every goto trick better/simpler, but that some of those tricks should be avoided to simplify good programming.

Concepts and concepts lite are meant to make good generic programming simpler. They are not meant to be a drop-in substitute for every metaprogramming and macroprogramming trick. If you are an expert, and if in your expert opinion you and your users really need those tricks, you can still use them, but we need to make many (most) uses of templates easier to get right, so that they can become more mainstream. That where concepts and concept lite fits in.

Some of you may find this hard to believe, but "back then" there was quite serious opposition to function declarations because "they restricted the way functions could be used and the way separate compilation could be used" and also serious opposition to virtual functions "because pointers to functions are so much more flexible." I see concepts lite (and concepts) in the same light as goto/for, unchecked-function-arguments/function-declarations, pointers-to-functions/abstract-classes.

Reminds me of the related antipattern: “Something must be done. This is something. Therefore we must do it!”

atomic Weapons: The C++ Memory Model and Modern Hardware

[ETA: Updated OneDrive slides link]

Most of the talks I gave at C++ and Beyond 2012 last summer are already online at Channel 9. Here are two more.

This is a two-part talk that covers the C++ memory model, how locks and atomics and fences interact and map to hardware, and more. Even though we’re talking about C++, much of this is also applicable to Java and .NET which have similar memory models, but not all the features of C++ (such as relaxed atomics).

Note: This is about the basic structure and tools, not how to write lock-free algorithms using atomics. That next-level topic may be on deck for this year’s C++ and Beyond in December, we’ll see…

atomic<> Weapons: The C++ Memory Model and Modern Hardware

This session in one word: Deep.

It’s a session that includes topics I’ve publicly said for years is Stuff You Shouldn’t Need To Know and I Just Won’t Teach, but it’s becoming achingly clear that people do need to know about it. Achingly, heartbreakingly clear, because some hardware incents you to pull out the big guns to achieve top performance, and C++ programmers just are so addicted to full performance that they’ll reach for the big red levers with the flashing warning lights. Since we can’t keep people from pulling the big red levers, we’d better document the A to Z of what the levers actually do, so that people don’t SCRAM unless they really, really, really meant to.

Topics Covered:

  • The facts: The C++11 memory model and what it requires you to do to make sure your code is correct and stays correct. We’ll include clear answers to several FAQs: “how do the compiler and hardware cooperate to remember how to respect these rules?”, “what is a race condition?”, and the ageless one-hand-clapping question “how is a race condition like a debugger?”
  • The tools: The deep interrelationships and fundamental tradeoffs among mutexes, atomics, and fences/barriers. I’ll try to convince you why standalone memory barriers are bad, and why barriers should always be associated with a specific load or store.
  • The unspeakables: I’ll grudgingly and reluctantly talk about the Thing I Said I’d Never Teach That Programmers Should Never Need To Now: relaxed atomics. Don’t use them! If you can avoid it. But here’s what you need to know, even though it would be nice if you didn’t need to know it.
  • The rapidly-changing hardware reality: How locks and atomics map to hardware instructions on ARM and x86/x64, and throw in POWER and Itanium for good measure – and I’ll cover how and why the answers are actually different last year and this year, and how they will likely be different again a few years from now. We’ll cover how the latest CPU and GPU hardware memory models are rapidly evolving, and how this directly affects C++ programmers.

Videos: Panel, and C++ Concurrency

I’m about two weeks late posting this, but two more C++ and Beyond 2012 videos are now available online.

The first is my concurrency talk:

C++ and Beyond 2012: C++ Concurrency (Herb Sutter)

I’ve spoken and written on these topics before. Here’s what’s different about this talk:

  • Brand new: This material goes beyond what I’ve written and taught about before in my Effective Concurrency articles and courses.
  • Cutting-edge current: It covers the best-practices state of the art techniques and shipping tools, and what parts of that are standardized in C++11 already (the answer to that one may surprise you!) and what’s en route to near-term standardization and why, with coverage of the latest discussions.
  • Blocking vs. non-blocking: What’s the difference between blocking and non-blocking styles, why on earth would you care, which kinds does C++11 support, and how are we looking at rounding it out in C++1y?

The answers all matter to you – even the ones not yet in the C++ standard – because they are real, available in shipping products, and affect how you design your software today.

The second is one of the panels:

imageC++ and Beyond 2012: Panel – Convincing your Colleagues

From C++ and Beyond 2012, Andrei, Herb and Scott present Convincing Your Colleagues – an interactive panel.

Abstract:

You can’t do a better job if you don’t change what you’re doing, but change is hard.  It’s especially hard when what needs to change is your colleagues’ approach to software development. Moving your team forward often requires persuading your peers to change their behavior, sometimes to do something they’re not doing, other times to stop doing something they’ve become accustomed to.  Whether the issue is to embrace or avoid C++ language features, to adopt new development tools or abandon old ones, to increase use of or scale back on overuse of design patterns, to adhere to coding standards, or any of the plethora of other matters that affect software creation, moving things forward typically requires getting your colleagues to buy into the change you’re proposing.  But how can you do that?

In this panel session, Andrei, Herb, and Scott share how they go about convincing their colleagues to change and take questions from the audience.

Truth be told, the panel ranged widely and probably most of the time was on other topics!

I hope you find them useful.

Java vulnerabilities

With the help of friends Robert Seacord and David Svoboda of CERT in particular, I posted a note and link to their CERT post today because people have been misunderstanding the recent Java vulnerabilities, thinking they’re somehow really C or C++ vulnerabilities because Java is implemented in C and C++.

From the post:

Are the Java vulnerabilities actually C and C++ vulnerabilities?

by Herb Sutter

 

You’ve probably seen the headlines:

[US-CERT] Java in Web Browser: Disable Now!

We’ve been telling people to disable Java for years. … We have confirmed that VU#625617 can be used to reliably execute code on Windows, OS X, and Linux platforms. And the exploit code for the vulnerability is publicly available and already incorporated into exploit kits. This should be enough motivation for you to turn Java off.

Firefox and Apple have blocked Java while U.S. Homeland Security recommends everyone disable it, because of vulnerabilities
Homeland Security still advises disabling Java, even after update

Some people have asked whether last week’s and similar recent Java vulnerabilities are actually C or C++ vulnerabilities – because, like virtually all modern systems software, Java is implemented in C and C++.

The answer is no, these particular exploits are pure Java. Some other exploits have indeed used vulnerabilities in Java’s native C code implementation, but the major vulnerabilities in the news lately are in Java itself, and they enable portable exploits on any operating system with a single program. …

Some other C++ experts who have better sense than I do won’t add the following bit publicly, but I can’t help myself: Insert “write once, pwn everywhere” joke here…

Video: You Don’t Know const and mutable

imageAt C++ and Beyond in August, I gave a 30 min talk on the changed meaning of const and mutable. The talk video is now online:

You Don’t Know [keyword] and [keyword]

const means const.

Bonus: mutable is useful and continues to mean ‘already as good as const.’

This is another way C++ has become simpler: const now means what people always thought it meant, and the four-line code example doesn’t need deep analysis at all – it just works.

But we analyze the four-line example anyway as a motivating case to see why and how it works, so we can fully appreciate this new simplification in C++11…

An implementation of generic lambdas is now available

For those interested in C++ standardization and not already following along at isocpp.org, here’s an item of likely interest:

An implementation of generic lambdas (request for feedback)—Faisal Vali

This week, Faisal Vali shared an initial “alpha” implementation of generic lambdas in Clang. Faisal is the lead author of the proposal (N3418), with Herb Sutter and Dave Abrahams.

To read and participate in the active discussion, see the message thread on std-proposals.

Here is a copy of Faisal’s announcement…

Read more at isocpp.org…

Compatibility

On yesterday’s thread, I just wrote in a comment:

@Jon: Yes, C++ is complex and the complexity is largely because of C compatibility. I agree with Bjarne that there’s a small language struggling to get out — I’ve participated in private experiments to specify such a language, and you can do it in well under half the complexity of C++ without losing the expressivity and flexibility of C++. However, it does require changes to syntax (mainly) and semantics (secondarily) that amount to making it a new language, which makes it a massive breaking change for programs (where existing code doesn’t compile and would need to be rewritten manually or with a tool) and programmers (where existing developers need retraining). That’s a very big tradeoff. So just as C++ ‘won’ in the 90s because of its own strengths plus its C compatibility, C++11 is being adopted because of its own strengths plus its C++98 compatibility. At the end of the day compatibility continues to be a very strong force and advantage that isn’t lightly tossed aside to “improve” things. However, it’s still worthwhile and some interesting things may come of it. Stay tuned, but expect (possible) news in years rather than months.

That reminded me of a snippet from the September 2012 issues of CACM. When I first read it, I thought it was so worthwhile that I made a magnified copy and pasted it on the window outside my office at work – it’s still there. The lightly annotated conclusion is shown here.

imageThe article itself was about a different technology, but the Lessons Learned conclusion remind us of the paramount importance of two things:

  • Backward compatibility with simple migration path (“compat”). That’s what keeps you “in the ecosystem.” If you don’t have such compatibility, don’t expect wide adoption unless there are hugely compelling reasons to put up with the breaking change. It’s roughly the same kind of ecosystem change for programmers to go from Language X to Language Y as for users to go from Platform X to Platform Y (e.g., Windows to Mac, iPhone to Android) – in each case you have a lot of relearning, and you lose understood tools and services some of which have replacements and some of which don’t.
  • Focusing on complete end-to-end solutions (“SFE” or scenario-focused engineering). This is why it’s important not to ship just a bunch of individual features, because that can leave holes where Steps 1, 2, and 4 of an end-to-end experience are wonderful but Step 3 is awkward, unreliable, or incomplete, which renders much of the good work in the other steps unuseful since they can’t be used as much or possibly at all.

Enjoy.