Trip report: Winter ISO C++ standards meeting

On March 5, the ISO C++ committee completed its winter meeting in Jacksonville, FL, USA. We had record-tying attendance, with over 110 experts officially representing eight national bodies. As usual, we met for six days Monday through Saturday, and around the clock from 8:30am till 10pm most days, after which many people still went back to hang out in the lobby or their rooms to update papers. — The hotel had a baby grand piano outside the main meeting room lobby, and so late at night you could often walk by and find one of several committee members playing a tune, while as usual people collaborated on their proposals, perched on couches and tables clustered around glowing rectangles, incanting standardese to the soft strains of Russian folk ballads and arena rock.

Here’s a summary of what happened, with some details about the current ISO C++ process so you can see just how the work is progressing and getting released. I’ve tried to add some links to the relevant feature design papers, or to the papers that summarize what was done which in turn usually carry more links to the design papers.

Note: The features listed below are those approved for C++17 at this meeting only. Previous meetings added other features that are already in the working draft.

What we did: Merging into C++17

This meeting was a major milestone in two ways.

First, we largely completed choosing the target set of features the committee felt were ready for C++17. Some had their specification wording adopted last week into the working draft for the C++ international standard (IS), and a few more are targeted to be adopted at our next meeting in June in Oulu, Finland, following which C++17 is intended to be sent out for its major international comment ballot.

Second, and perhaps even more importantly, we had the first merge of the new Technical Specification (TS) “beta feature branches” into the C++ international standard “trunk.” The committee considered five candidates, and adopted the first four as ready for C++17, with reports from implementers and users that the features were baked and unlikely to need future breaking design changes. The criteria for whether the committee thinks a given feature is ready to merge will vary by feature; this time, the committee ended up merging all the ones that had been completed and in beta testing for a year or more.

The four accepted branches merged into C++17 were:

  • The Parallelism TS, a.k.a. “Parallel STL.” This includes parallelized versions of most STL algorithms, which get a new execution policy where you can opt into running them in parallel on multiple cores and/or on multiple vector lanes—hardware that is now ubiquitous in the mainstream, as most medium- and high-end smartphones are both multicore and have vector units built in. For example, C++17 now allows: for_each(std::par, first, last, [](auto& x){ process(x); });.
  • The Library Fundamentals 1 TS (most parts), adding a new round of library components for C++17 including any, optional, string_view, shared_ptr for arrays, memory pools, search and sampling algorithms, and more. (An example of “more”: Have you ever wanted to pass a tuple that contains a function call’s argument list and expand it into the individual arguments? Now you can, using built-in standard functionality.)
  • The File System TS, which allows portable handling of files, paths, symbolic links, and more.
  • The Mathematical Special Functions IS, which was originally part of the first Library TR back in the 2000s and has now finally gained approval to be added to the standard for C++17.

We also voted the following other features directly into C++17 without going through a TS beta first:

  • Lambdas are now allowed inside constexpr functions.
  • Lambdas can now capture a copy of *this object by value, using the notation [*this]. It also works in combination with default capture, so [=, *this] is also allowed.
  • The range-for loop can now deal with generalized ranges where the “end” type is different from the “begin” type, such as sentinel-based and counted ranges, which will work well with the Ranges TS work.
  • The [[fallthrough]] attribute lets you designate that you are intentionally letting a switch’s case block fall through to the next block without a break.
  • The [[nodiscard]] attribute lets you annotate a return value to say that it should not be ignored by the caller, or annotate a type to apply to all uses of that type as a returned value. Canonical examples include the return value of std::async where ignoring the return value would mean being not async at all, and the return value of a container’s empty() function where people sometimes confuse empty() and clear() and we don’t want them to think calling empty() has side effects if they call it by mistake.
  • The [[maybe_unused]] attribute lets you annotate an intentionally unused variable or parameter.
  • Various other minor improvements and fixes.

Evolution also approved the following features that were not voted in at this meeting, but are currently on track for possible approval for C++17 at our next meeting in June for inclusion in the C++17 international comment ballot (note the links are to the pre-meeting paper versions; for example, in the paper that proposed constexpr_if the committee changed the syntax to if constexpr at the meeting):

A few more, such as structured bindings, might make it too but have not yet received Evolution design approval, so we’ll see in June if there’s support in Evolution and Core to call them ready.

C++17 will change the way we write C++ code, just as C++11 and C++14 did. For example, string_view and optional are expected to be heavily used in writing interfaces. And with parallel STL often you can just add std::par or std::par_vec, and your algorithm will speed up by a factor of 2-4 on ordinary hardware; we had a compelling story with C++11 move semantics where we could say “just recompile your code and it’ll often be noticeably faster,” and this is likely to be an even bigger improvement.

Other progress

We also made significant progress on other work, including that we now have opened five TSes to launch our next round of beta feature branches, most of which we expect to be sent out for their “debutante” international comment ballots in the next handful of meetings:

  • Ranges, Networking, Library Fundamentals 2, and Parallelism 2: We approved creating these at our last meeting, and officially adopted the first working drafts for all four of these TSes at this meeting. These form the “first four” nucleus of the next round of TSes to be developed by the committee and are now being officially specified in detail.
  • Modules: Recall that at the last meeting we reached design agreement to start work on Modules. At this meeting, we approved the initial wording to officially start active work on a Modules specification as a TS. That brings us to five active TSes in this second round.

And there are more to come:

  • Coroutines: These are being targeted for a new TS (possibly Concurrency 2 or their own TS) so that we have additional time to finish investigating an alternative proposal. I expect this to settle out within another couple of years and then make progress in some form (either the current proposal, or the alternative one if it pans out equally well, or possibly both) into the following C++ standard.
  • Concurrency 2: The concurrency group reports that they hope to launch a second Concurrency TS later this year at the Issaquah meeting. This might include coroutines as above. It might include features like synchronic, and there is still hope that the group might yet get design agreement on executors (a way to control where to execute work, such as where to run a parallel task, or where to execute a callback or continuation); if so, it would naturally go into the second Concurrency TS.
  • 2D Graphics: The 2D Graphics proposal that specifies a portable C++ library to wrap the Cairo C API has progressed to design and wording review. It’s big, so it will take a couple of meetings to process it, but it’s looking good that we could get an initial TS draft started this year.

At this meeting we also reached a design agreement on contracts, so that we now think we have a design that the competing proposers agree on and that the committee as a whole likes, and we hope to get initial draft specification wording later this year if this trend continues. Once we see the work, the committee can decide on a preferred target vehicle (TS or straight into the IS).

The committee also continued to spend time at the meeting incubating other work, including reflection APIs.

The main surprise last week was that the “part 1” unified call syntax proposal made it to full committee but we there discovered that there wasn’t yet consensus to adopt it. The authors (Bjarne Stroustrup and myself with the help of a number of experts) are continuing to work between meetings to see if consensus can be achieved by the June meeting.

Finally, we retired a TS that was not completed: We ended up not getting consensus on the experimental material intended for the Arrays TS, so last week we made the decision to retire that one.

Status of concepts and modules

Here are some specific notes about two particular TSes that everyone agrees are important and game-changing for how we will write C++ code: Concepts and modules.

Concepts was published as a TS less than a year ago, with the first production compiler implementation to ship soon as part of GCC 6.0. Despite its relative newness and lack of field experience, the committee seriously considered adopting it already for C++17 at this meeting. It was right on the edge: There was already a strong minority who felt it was ready to add to C++17, and a modest majority preferred to wait another year or two and get more experience with shipping implementations and field use of the feature.

Virtually everyone who spoke praised the feature as important, and either ready or getting close to ready, for adopting into the C++ standard. The only major distinction of opinion was whether or not they supported putting it specifically in C++17; those who spoke against that generally expressed that this feature was so important that we want to be sure we get it right and approve it once we have more experience using it in real world users’ code. My impression is that concepts is now one of the strongest feature candidates moving along well in the pipeline and adoption in the trunk IS working draft post-C++17.

At the October meeting, modules for the first time achieved a design consensus in the Evolution working group (EWG), with a “phase 1” that everyone agreed on and a “phase 2” that could be added. The most notable part of “phase 2” is determining whether and how to allow a module to carry and export macros. In October, EWG decided to aim modules for a TS covering “phase 1” and asked the proposers to come back with detailed specification wording, which they did for this meeting.

Last week, the whole committee voted to start a TS project and turn that document into the first working draft. Here’s a subtle but important note: This milestone of “create an initial working draft” is important, but usually not all that impactful to the community. The reason this one is different, and is actually very important to users, is that we knew that there are likely two major vendor implementation efforts waiting to start work on implementing support for modules in major compilers, but who were waiting until there was some draft specification that had been approved as a basis for development by the committee. We knew that having “at least an initial working draft” was a sufficient bar for the implementers to get started, and so despite our heavy workload this week focusing on C++17-targeted items (which this is not), I asked the Core working group to make a special effort to prioritize getting this project to this stage because we knew it would unblock new implementation efforts. We are very pleased to see this succeed, and look forward to active work beginning soon on more implementations of modules in several major compilers.

The pipeline

There are members who are very disappointed that we didn’t include more in C++17, especially concepts which was very close. To understand where we’re at, consider the following snapshot of our current “trunk” and “beta feature branches” status, shown in the context of three eras of C++ standardization laid side by side: C++9x, C++0x, and C++1x.

wg21-timeline

The C++ international standard (IS) is the “trunk” containing everything that is formally part of C++. Every time it is published, we have a new official international legal specification for C++.

A Technical Specification (TS) is a “beta feature branch” shipped as a separate published document. The contents of a TS are the committee’s best guess at what C++ users need, with a lower confidence bar on that guess because it is a “beta” and we do get a chance to make corrections before setting it in stone in the standard. In 2015, we completed our first round of six published major TSes, and are launching a second round.

This is a turning point: We just finished filling the pipeline, and this meeting saw the first merge from the beta branches. The disappointment came from that some would have liked to merge another branch or two. The good news is that, those branches are already shipped beta specifications that vendors can implement—in fact, most of them already have at least one supported implementation in a production compiler or library product. The branches that did not quite land in time to merge for C++17 can be expected to merge early in the next cycle—and in the meantime we’re still shipping the “trunk” and its ready features too. Now that the pipeline is full, we can expect features to leave the pipeline and merge regularly at high quality, which is essential with over four million C++ users worldwide relying on our work.

Notes:

  • Each bar starts at the meeting where the detailed specification work officially began (usually “adopt initial draft”) and ends at the meeting that technical work was completed (usually “start final ballot / send for publication”).
  • Arrowheads join the IS at the meeting at which a TS feature branch was merged into the IS trunk.
  • In the current decade, the smaller boxes show full ISO C++ meetings—sometimes two per year, sometimes three per year—and highlights the meetings where we launch a C++ CD (ISO comment ballot, light blue) or DIS (ISO approval ballot, dark blue).
  • I’ve omitted the 1998-2001 hiatus because no new features were under active development in that period. The committee was voluntarily taking a break, and also catching up with C++98 bug reports while compilers caught up with the standard.

From “what” to “when”: C++0x to C++1x

During C++0x, the best and most solid parts of C++0x were the parts that shipped first as “beta” in the Library Extensions TR. That’s the model we’ve now embraced as the norm, rather than as the exception.

C++0x followed the “what” or “feature” model: We picked “what” features were in the standard and released whenever they turned out to be ready. That means we giving up on being able to predict “when” we’d ship, and even while C++0x was repeatedly delayed and the “0x is hex” jokes started, we released nothing in the standard the meantime; even features that had been approved into the C++ draft standard in 2004 had to wait until 2011 before they appeared in an official final standard. C++11 was a good release, but we underestimated “when” it would be ready by as much as five years.

C++1x follows the “when” or “train” model: We’re picking “when” to release the standard regularly, currently every three years, and including whatever features are ready. This still allows big long-pole features to be worked on; they’re just worked on concurrently until ready to merge. I’m pleased that the committee seems to be generally quite happy with the result. In Jacksonville, several experts came up to me to say that they had noticed how we had shipped more work in six years than in any other six-year period in the history of ISO C++. This would not be possible without the committee’s help and support; thank you again, everyone!

Although the train model has benefits, any model is also a tradeoff. The train model means we can’t slip a few meetings to try to “squeak in” another feature, and in fact we didn’t ship everything we hoped to ship in C++17, notably concepts… but neither did we at the same point in the C++0x cycle, in 2007. Now as in 2007, we are at a point where we have some good new features ready to ship in the standard, and at the same time we have several large desirable features in flight that are “almost but not quite” ready yet. In 2007, items on the edge or “just over the horizon” included work like threads and C++0x concepts, and in 2016 they include concepts “lite” and ranges. In both cases, those features are actually 3-4 years away (whether we knew it then or not). The difference is that in 2016, we’re shipping what is ready while still continuing development at full speed on the others.

Also, we initially thought a train model would let us alternate “minor” and “major” releases. The idea was that C++14 could be a “minor” release, and C++17 a “major” release. What actually happened was that we learned the pipeline+train model leads to regular “medium” releases as the pipeline moves at a steady pace and we regularly release what’s ready. As it turns out, C++14 was more than a fine-tuning release; it included powerful features like generic lambdas. Conversely, C++17 won’t be a “major release” on the scale of C++11, but neither could it be because C++11 took nine years to develop.

Never before in the history of C++ has the standard maintained such high specification quality, never before has it released on a regular predictable schedule… and as we removed that quality and schedule uncertainty, it’s no accident that never before have all the major compilers tracked the standard so closely, which benefits the whole worldwide C++ community.

How to define “progress”: Being unblocked to move to the next stage

How do we know we’re making progress? There is a tendency to emphasize “what features ship in C++.next,” a specific version; this was the committee’s mental model during C++0x.

In C++1x, the key is to keep the pipeline moving. What I monitor is that that major proposals are unblocked so the proposers can do work toward the next stage between meetings. As long as each proposal is not blocked, it’s moving forward toward the IS and “what ships in C++.next” will take care of itself as soon as it’s baked and ready.

Take a few minutes to read The life of an ISO proposal: From “cool idea” to “international standard.” Here are how some major work moved along at this meeting, in the context of those Stages:

  • Ranges, Networking, Library Fundamentals 3, and Parallelism 2: Stage 6 -> 7. All of these reached Stage 6 at our last meeting, and reached Stage 7 at this meeting. They’re all moving, and they’re well along in their development cycles and getting ready to debut in their international TS comment ballots in the near future.
  • Modules: Stage 3 -> 6. This reached Stage 3 at our previous meeting in November, and is already at Stage 6 at this meeting—that’s very rapid progress in one meeting!
  • Contracts: Stage 2 -> 3. This was stalled for a long time in Stage 2, and thanks to lots of work by the proposers (brokered personally by Bjarne Stroustrup) at the last meeting in November and then between meetings, it finally broke the logjam and reached Stage 3 at this meeting. Now it’s finally unblocked and on track again.
  • Coroutines, and 2D Graphics: Stage 3. These have both been at Stage 3 for a couple of meetings, and should reach Stages 4 and 5 over the next meeting or three.

If we focus primarily on each proposal moving incrementally to Stage N+1 and identifying and removing impediments, getting great new work in each new C++.next release of the standard becomes very nearly automatic.

What’s next

Our next meeting is this June in Oulu, Finland, where we plan to send a feature-complete C++17 out for its major international comment ballot. After that, we’ll address ballot comments at the November 2016 and February-March 2017 meetings and plan to send C++17 out for its final approval ballot then.

What’s after C++17? The default is to continue on the 3-year cycle (aim for C++20 then C++23), but we’re also thinking about whether to try out going to a 2-year cycle (aim for C++19, and if that works then C++21 and C++23). We’ll discuss that later this year as we see how C++17 and current/future TS plans are coming together, and should have more definite plans to report in a meeting or two.

Thank you again to the over 110 experts at Jacksonville last week, and the many more who participate in standardization through their national bodies, without whose efforts and willing collaborative spirit these results would not be achievable. The over four million C++ users worldwide benefit, and appreciate it very much too. Thank you, all, for your many contributions to Standard C++.

Acknowledgments

Thank you to everyone who provided input data and/or review of drafts of this trip report, including but not limited to: JF Bastien, Steve Clamage, Marshall Clow, Gabriel Dos Reis, William M. (Mike) Miller, Nathan Myers, Clark Nelson, Gor Nishanov, Peter Sommerlad, John Spicer, Bjarne Stroustrup, Ville Voutilainen, Jonathan Wakely, Nathan Wilson, and Jeffrey Yasskin.