Feeds:
Posts
Comments

On June 25, the ISO C++ committee completed its summer meeting in Oulu, Finland, hosted by Symbio and the Finnish national body.

We again had some 100 experts officially representing nine national bodies. As usual, we met for six days Monday through Saturday, and around the clock from 8:30am till 11pm most days – evening sessions regularly went a little later than the usual 10pm because it was Midsummer and there was literally no full night all week long at that short distance from the Arctic Circle, and people commonly did a double take when they looked at their watches and were surprised to find that it was already nearly midnight.

Here’s a summary of what happened, with some details about the current ISO C++ process so you can see just how the work is progressing and getting released. I’ve tried to add some links to the relevant feature design papers, or to the papers that summarize what was done which in turn usually carry more links to the design papers.

C++17 is feature-complete, enters review period

The big news is that C++ is feature-complete, and on time! We added several more features to the C++17 working paper (see next section), then approved the result to be sent out this summer for its major ISO international comment ballot, which is the Committee Draft or “CD” ballot.

Between now and our November meeting, national bodies around the world will be reviewing the draft and reporting any concerns about feature design details and wording correctness or consistency. Then for probably the next two meetings we’ll be addressing every national body comment and recording its resolution, as well as continuing to process our own known issues lists to fine-tune the text of the standard. That usually takes two meetings, and if that’s the case again this time then we’ll be putting the finishing touches on C++17 in our November and March meetings and then, we hope, sending C++17 out for its possibly-final ballot in the spring. If we need an extra meeting, then that would extend to the July meeting next year and the possibly-final ballot in late summer.

So C++17 is and tracking to ship on schedule next year. In my March trip report, I mentioned that after completing C++11 we switched to a “train model” where we have been shipping the standard consistently every three years. So far, we have been running the trains on time: C++14 was actually the first C++ standard ever that shipped when expected, and now C++17 is on track to do it again. This is excellent news for the community because it enables implementers to track the standard closely, whereas in previous releases they often held back from implementing new features aggressively because with an open-ended schedule the committee could (and did) change its mind again about the design of some feature before the standard actually shipped. The ISO C++ committee is now sticking to a high quality bar, voting features in when they’re stable and releasing on time, which removes uncertainty and is a major reason why compilers are more in sync than ever before: After C++98 shipped, it took 5 years before we saw the first complete conforming compiler that implemented all language features (modulo bugs of course); after C++11 shipped, it took 3 years; when C++14 shipped, it took months. Now, many C++17 features are already available in major compilers, and I wouldn’t be surprised if C++17 repeated C++14’s synchronization with the community so that we see at least one major implementation that has all C++17 features in the same year that C++17 is published. This is great news for the community, because it means we have to play less of the “which compilers actually implement which features” game; true, we still need to deal with older compilers, and some compilers are still not as quick as others to ship the latest features, but all of the major compilers’ current releases are in closer sync with the standard than they’ve ever been before and even the delta between them continues to shrink, which is good news for all C++ users.

Note: Last time I mentioned that there was some thought of moving to a two-year cadence after C++17, but for now we’re staying with three years, so the next standard after C++17 will be C++20.

More features added to C++17

At this meeting, we added several more features into C++17; you can find a good summary of those features in Bryce Lelbach’s Reddit post. Again, note that these are just the ones we added at this meeting; the complete list of C++17 features is longer. For example, see my March trip report for what was added at the spring meeting.

Language features

Here are some of the more noticeable language features we added to C++17 in Oulu. I’ll merge from Bryce’s nice list (thanks Bryce!), my own comments in the previous trip report, and some new comments and examples:

Dynamic memory allocation for over-aligned data: Extending the C++11 alignof/alignas for heap allocated memory control.

Template argument deduction for constructors, so that you can write just pair p(2, 4.5); instead of pair<int,double> p(2, 4.5); or auto p = make_pair(2, 4.5);. This is pretty sweet, including that it obsoletes many “make” helpers.

template <auto>: Recall that templates can take “non-type parameters,” or ordinary compile-time values; a familiar example is std::array<widget,10> which is an array of 10 widget objects, passing 10 as a compile-time number. Well, as of C++11 we allow “auto”matically deducing the type of local variables:

auto i = 10;           // deduces int

and in the Concepts TS we allow “auto”matically deducing the type of parameters:

void f(auto value) { } // same as template <class T> void f(T value);

f(10);                 // deduces int

Now, with the template<auto> extension, C++17 will additionally allow the same to happen when you pass a value as a template parameter, which is just another place you can pass a value:

template <auto value> void f() { }

f<10>();               // deduces int

Guaranteed copy elision: When you call a function that returns an object by value, and you use the copy to initialize a local variable, the language has always said that you copy (or move) twice. You may know that, since forever, C++ has also added, “but by the way, the compiler is allowed to elide (disappear) the extra copy.” With guaranteed copy elision, in many cases the C++17 language now says you copy (or move) once; that is, compiler is now required not to perform an extra copy or move, so that copy elision is no longer an optimization but a guaranteed language feature.

Richard Smith, the proposal author, provided the following additional notes and examples that show how the copy elision guarantee depends on what happens inside the function. In some cases, such as when the function returns a temporary of the right type, the new rules guarantee that zero copies will be performed, and the return type is not even required to be copyable or movable any more:

T f() {
  return T{}; // no copy here (C++17)
}

T x = f();    // no copy here either (C++17)

In other cases, where the named return value optimization (NRVO) would apply today, there is one copy and the compiler may elide the copy, but is not guaranteed to do so because there are cases where that could not be guaranteed:

T g() {
  T t;
  return t;   // one copy, can be elided but elision is not guaranteed
}

T y = g();    // no copy here (C++17)

And if the function returns something that really requires a copy, such as returning a reference parameter, you get exactly one copy:

T h(T &t) {
  return t;   // one guaranteed copy (by necessity)
}

T z = h(x);   // no copy here (C++17)

Order of expression evaluation guarantees: This removes portability and usability bugs. In a nutshell, a number of code examples that you thought worked, now actually do work. In particular, this solves the long-standing f(new T, new T) bug that I wrote about nearly 20 years ago, fixes some examples in published books, and makes the brand-new future<T>::then chaining actually work correctly.

Inline variablesThis feature makes it easier to define global variables (that’s the bad news) correctly (that’s the good news), including in header files. The net effect is that some code people already write in practice, but shouldn’t have because it had subtle pitfalls, now works.

if constexpr: This is a very powerful feature that allows branches that are evaluated at compile time. Note that this is a disciplined compile-time “if”, not just text substitution; the code in a false branch needs to be syntactically well-formed, but doesn’t need to be semantically valid.

This lets you express some pretty powerful stuff, including that a function template can write all the special cases of its algorithm right within its body (e.g., a fast algorithm for random-access iterators, and a fallback for less powerful iterators), without writing a set of template specializations.

It also works very nicely with C++14’s “auto” return type deduction for inline functions: It’s perfectly fine for two mutually exclusive if constexpr branches to return unrelated types since only one of the branches can be taken for a given set of inputs, and the correct return type will be deduced. Of course, within the same if constexpr branch the return types have to be compatible according to the current rules. In fact, we’ll see an example of this in the next topic…

Structured bindings: This allows taking a value that contains multiple elements, such as a tuple or a struct, and binding convenient names to the individual elements – much like std::tie, except without having to have variables of the correct type already available. I was pleasantly surprised to see this one make it into C++17 even though it came in late in the cycle. Here’s a simple example of what it enables:

tuple<T1,T2,T3> f();
auto [x,y,z] = f(); // types are: T1, T2, T3

map<int,string> mymap;
auto [iter, success] = mymap.insert(value); // types are: iterator, bool

struct mystruct { int i; string s; double d; };
mystruct s = { 1, “xyzzy”s, 3.14 };
auto [x,y,z] = s; // types are: int, string, double

What if you want to add structured bindings for your own type that isn’t a struct or a tuple? I’m glad you asked, because it’s an excuse to also show off also how you can use if constexpr. Let’s say you’re the author of a type S:

class S {
   int i;
   char c[27];
   double d;
public:
   // ...
};

Now you want to allow structured bindings for your type S, and for extra marks you don’t want to bind directly to the private variables but perhaps want member c to be bound as a string_view (also adopted for C++17; see below). Here’s how do add this ability; it’s a few lines, but you only have to do it once for all users of type S:

// 1. Add tuple_size and tuple_element support
namespace std {
   template<> struct tuple_element<0,S> { using type = int; };
   template<> struct tuple_element<1,S> { using type = string_view; };
   template<> struct tuple_element<2,S> { using type = double; };
   template<> struct tuple_size<S>: public integral_constant<size_t,3> {};
}

// 2. Now add get<> support (using C++17, because why not; it’s better
// than =delete’ing the primary get<> template and adding specializations)
template<int I>
auto get(const S&) {
   if      constexpr(I == 0) return x.i;
   else if constexpr(I == 1) return string_view{x.c}; }
   else if constexpr(I == 2) return x.d;
}

Now all users of S can write code like this:

S f();  // some function that returns an S

auto [ n, s, val ] = f(); // types are int, std::string_view, and double

if (init; condition) and switch (init; condition): Just like the venerable for (int i = 0; /*…*/) loop has always allowed declaring a variable (in that example, i) that exists for the scope of the loop, we will now be able to do the same for if and switch without having to resort to today’s workaround of declaring the variable before the if or switch and then wrapping it up in { } to restrict its scope. Here’s an example of the new cleaner syntax, which allows more tightly-scoped variables:

map<int,string> mymap;

if (auto result = mymap.insert(value); result.second) {
    // insert succeeded, and result is valid for this block
    use(result.first);  // ok
    // ...
} // result is destroyed here

In fact, you might have noticed that I used a similar example for both structured bindings and for if/switch initialization. It took Reddit about 30 minutes to discover that they’re great together, like peanut butter and jelly:

// better together: structured bindings + if initializer

if (auto [iter, succeeded] = mymap.insert(value); succeeded) {
    use(iter);  // ok
    // ...
} // iter and succeeded are destroyed here

Two language features that didn’t make it (yet)

Of the new language features I mentioned last time, only two didn’t make it into C++17:

  • operator. (dot) to allow smart references (in parallel with smart pointers) and much more. During final core language wording review, the authors discovered a problem that couldn’t be fixed in real time at this meeting, and was pulled back while the authors fix the proposal. This proposal is still active and is expected to come back with a fix to that problem very soon, possibly as soon as in the next week or two for the post-meeting mailing; I’m hearing some rumblings that national bodies might mention this in their summer ballot comments, so we might see it again during C++17 ballot resolution, else I’d expect it to be one of the first things added to the draft standard once C++17 ships.
  • Defaulted comparisons, to generate ==, !=, <, <=, >, >= for types that don’t write them by hand. The current version of the proposal is opt-out, and would generate comparisons for existing types. When the proposal got to the full committee, we discovered that the committee didn’t have consensus to approve the paper’s opt-out model; a number of people spoke up for an opt-in model instead. This proposal might eventually come back but probably not soon.

Standard library features

Here are some of the library features that made it into C++17 at this meeting:

variant<>: Don’t use raw unions any more. This addition completes the trifecta of std::any and std::optional (which were already added to C++17). The new std::variant is a type-safe union; for more background see also my fall meeting trip report. Here’s an example adapted from the paper that shows how you can use it:

variant<int, float, string> v, w;
v = “xyzzy”;         // now v contains a string
v = 12;              // now v contains an int

int i = get<int>(v); // ok, because it contains an int

w = get<int>(v);     // ok, assign to another variant
w = get<0>(v);       // same effect as the previous line
w = v;               // same effect as the previous line

get<double>(v);      // compile-time error: v can’t contain a double
get<3>(v);           // compile-time error: v doesn’t have 4 types

try {
  get<float>(w);     // will throw: w contains an int, not a float
}
catch (bad_variant_access&) {}

Moving nodes between map<>, unordered_map<>, set<>, and unordered_set<>: You will now be able to directly move internal nodes from one node-based container directly into another container of the same type (differing at most in the comparator template parameter), either one node at a time or the whole container. Why is that important? Because it guarantees no memory allocation overhead, no copying of keys or values, and even no exceptions if the container’s comparison function doesn’t throw. The mechanics are provided by new functions .extract and .move, and corresponding new .insert overloads. Here’s an example adapted from the paper:

map<int, string> src {{1,”one”}, {2,”two”}, {3,”buckle my shoe”}};
map<int, string> dst {{3,”three”}};

dst.insert(src.extract(src.find(1))); // iterator version
dst.insert(src.extract(2));           // key type version
auto r = dst.insert(src.extract(3));  // key type version
    // note: this last one will fail because dst already contains 3;
    // the node is owned by r, the returned struct

// At this point we have the following state:
//   src == {}
//   dst == {{1,“one”}, {2,“two”}, {3,“three”}}
//   r.position == dst.begin() + 2
//   r.inserted == false
//   r.node == {3,“buckle my shoe”}

// We can go back and retry the failed insert with a different key:
r.node.key() = 4;                      // use a non-conflicting value
dst.insert(r.position, std::move(r.node));  // now succeeds

Extended memory management tools for in-place construction and destruction: These include destroy(_at|_n), uninitialized_move(_n), uninitialized_value_construct(_n), and uninitialized_default_construct(_n).

In case you missed it: string_view also in C++17

C++17 also contains a number of other useful features. I’ve mentioned these before, including “small” but widely-used features like optional and any, right up to Parallel STL. But let me just take a moment to remark again on the wonderfulness of string_view, which is a non-owning view of a contiguous string owned by someone else.

If you have a const string& parameter, consider just changing it to string_view. As Mark Isaacson mentioned in his NDC talk earlier this month, it’s pretty rare that you can get a performance boost by just doing a global search-and-replace, but Mark points out you can pretty much do that by globally replacing const string& with string_view (with one exception noted in the next paragraph). Why is this often more efficient? In particular, when the caller passes something like a string literal as an argument, if your function takes it via a const string& parameter that means the caller has to convert the string literal to a temporary string object, which typically incurs a heap allocation and deallocation, whereas passing it to a string_view incurs no allocation overhead at all.

However, beware one potential pitfall, especially until more of the world has migrated to string_view: If your function f(/*some string*/ x) internally calls one or more other functions that still take a string& (either const or non-const), then let your function f continue to take x by const string& and don’t change it to string_view. The reason is that if f is called with an actual string object, then f(const string& x) can pass it along painlessly by reference, whereas f(string_view x) would avoid a copy when entering f but would need to create a new string internally just to pass it along to the other function that needs a string. Bottom line: If you know your function needs to call other functions that you don’t control and that take (possibly const) string&, then you should probably leave your function taking const string& too.

C++17: Recognizably new

C++17 will pervasively change the way we write C++ code, just as C++11 did. As these and other new features become available, we’re going to see new code using structured bindings, if/switch scope variables, string_view, optional, any, variant, Parallel STL, and more all over the place.

Here’s my personal litmus test for whether we’ve changed the way people program: Just as you can take a look at a screenful of code and tell that it’s C++11 (not C++98), if you can look at a screenful of code and tell that it’s C++17, then we’ve changed the way we program C++. I think C++17 will meet that bar.

Here’s a little “C++17 demo” put together by committee member Thomas Köppe, who is the author of the if/switch initializer proposal. It shows off a few of the features that will soon be available as part of C++17, and how they work together. Note this uses some other new C++17 features I didn’t specifically mention. I’ll let the comments speak for themselves; thanks, Thomas, for providing this demo.

unordered_map<string, unique_ptr<Foo>> items;
vector<unique_ptr<Foo>> standby;

// Desired semantics of f: If there is currently no item 'id', install
// 'foo' as item 'id'. Otherwise, put 'foo' on the standby list for later.

//------------------------------------------------
// Before C++17: 7 lines + 4 pitfalls
//
void f(string id, unique_ptr<Foo> foo) {
   auto it = items.find(id);
   if (it == items.end()) {
      auto p = items.emplace(move(id), move(foo));
      p.first->second->launch();
   } else {
      standby.push_back(move(foo));
      standby.back()->wait_for_notification();
   }

   // Notes:  
   // * Variable 'id' can no longer be used (moved-from); or...  
   // * ...would need to pass by 'const string& id' and force copying.
   // * Map lookup performed twice. Ordered map could use lower_bound +
   //   hint, but unordered map cannot.  
   // * (Cannot emplace unconditionally, because it might destroy *foo.)
}

//------------------------------------------------
// With C++17: 4 lines + none of those pitfalls
//
void f(string_view id, unique_ptr<Foo> foo) {
   if (auto [pos, inserted] = items.try_emplace(id, move(foo)); inserted){
      pos->second->launch();
   } else {
      standby.emplace_back(move(foo))->wait_for_notification();
   }
}

Other progress

Because this time our focus was on completing C++17, we didn’t spend as much time on the other TSes as we did at a normal meeting, but we made progress on some of the TSes and will continue with all of them next time. In particular, the Reflection and 2D Graphics proposals received several hours of review each and are making great progress; at our next meeting we’ll continue with Ranges, Networking, Library Fundamentals 2, Parallelism 2, Modules, Coroutines, Contracts, and more.

What’s next

This summer, we’re going to send out the feature-complete C++17 out for its major international comment ballot. After that, we’ll address ballot comments at the November 2016 and February-March 2017 meetings and plan to send C++17 out for its final approval ballot then.

Once C++17 ships next year, I expect we’ll start looking at merging more of the TS “beta branches” into the C++ “trunk.” Here’s an updated snapshot of our status:

wg21-timeline

Thank you again to our host Symbio, to the over 100 experts at Oulu last week, and to the many more who participate in standardization through their national bodies, without whose efforts and willing collaborative spirit these results would not be achievable. The over four million C++ users worldwide benefit, and appreciate it very much too. Thank you, all, for your many contributions to Standard C++.

logo2xOn Saturday afternoon, at the ISO C++ meeting in Oulu, Finland, we completed the feature set of C++17 and approved sending out the feature-complete document for its primary international comment ballot (aka “CD” or Committee Draft ballot).

An hour later, I sat down (via Skype) with Rob and Jason to do a CppCast interview about C++17 and what happened at the just-concluded meeting. I hope you enjoy it. Also check out the two related links:

Yesterday was a (long) travel day as many of us made our way back home — and had ad-hoc standards discussions in several different airport departure lounges. Now that I’m back in the office with my usual coffeemaker nearby, I’ll soon get around to writing up a written trip report. In the meantime, I hope you find the CppCast and other links useful.

(Interestingly, it just now occurs to me as I write this that one of the things we just adopted for C++17 make the CppCast subtitle needlessly verbose… the “<C++>” part can now be inferred! Cute. C++ without lots of angle brackets… who knew?)

On March 5, the ISO C++ committee completed its winter meeting in Jacksonville, FL, USA. We had record-tying attendance, with over 110 experts officially representing eight national bodies. As usual, we met for six days Monday through Saturday, and around the clock from 8:30am till 10pm most days, after which many people still went back to hang out in the lobby or their rooms to update papers. — The hotel had a baby grand piano outside the main meeting room lobby, and so late at night you could often walk by and find one of several committee members playing a tune, while as usual people collaborated on their proposals, perched on couches and tables clustered around glowing rectangles, incanting standardese to the soft strains of Russian folk ballads and arena rock.

Here’s a summary of what happened, with some details about the current ISO C++ process so you can see just how the work is progressing and getting released. I’ve tried to add some links to the relevant feature design papers, or to the papers that summarize what was done which in turn usually carry more links to the design papers.

Note: The features listed below are those approved for C++17 at this meeting only. Previous meetings added other features that are already in the working draft.

What we did: Merging into C++17

This meeting was a major milestone in two ways.

First, we largely completed choosing the target set of features the committee felt were ready for C++17. Some had their specification wording adopted last week into the working draft for the C++ international standard (IS), and a few more are targeted to be adopted at our next meeting in June in Oulu, Finland, following which C++17 is intended to be sent out for its major international comment ballot.

Second, and perhaps even more importantly, we had the first merge of the new Technical Specification (TS) “beta feature branches” into the C++ international standard “trunk.” The committee considered five candidates, and adopted the first four as ready for C++17, with reports from implementers and users that the features were baked and unlikely to need future breaking design changes. The criteria for whether the committee thinks a given feature is ready to merge will vary by feature; this time, the committee ended up merging all the ones that had been completed and in beta testing for a year or more.

The four accepted branches merged into C++17 were:

  • The Parallelism TS, a.k.a. “Parallel STL.” This includes parallelized versions of most STL algorithms, which get a new execution policy where you can opt into running them in parallel on multiple cores and/or on multiple vector lanes—hardware that is now ubiquitous in the mainstream, as most medium- and high-end smartphones are both multicore and have vector units built in. For example, C++17 now allows: for_each(std::par, first, last, [](auto& x){ process(x); });.
  • The Library Fundamentals 1 TS (most parts), adding a new round of library components for C++17 including any, optional, string_view, shared_ptr for arrays, memory pools, search and sampling algorithms, and more. (An example of “more”: Have you ever wanted to pass a tuple that contains a function call’s argument list and expand it into the individual arguments? Now you can, using built-in standard functionality.)
  • The File System TS, which allows portable handling of files, paths, symbolic links, and more.
  • The Mathematical Special Functions IS, which was originally part of the first Library TR back in the 2000s and has now finally gained approval to be added to the standard for C++17.

We also voted the following other features directly into C++17 without going through a TS beta first:

  • Lambdas are now allowed inside constexpr functions.
  • Lambdas can now capture a copy of *this object by value, using the notation [*this]. It also works in combination with default capture, so [=, *this] is also allowed.
  • The range-for loop can now deal with generalized ranges where the “end” type is different from the “begin” type, such as sentinel-based and counted ranges, which will work well with the Ranges TS work.
  • The [[fallthrough]] attribute lets you designate that you are intentionally letting a switch’s case block fall through to the next block without a break.
  • The [[nodiscard]] attribute lets you annotate a return value to say that it should not be ignored by the caller, or annotate a type to apply to all uses of that type as a returned value. Canonical examples include the return value of std::async where ignoring the return value would mean being not async at all, and the return value of a container’s empty() function where people sometimes confuse empty() and clear() and we don’t want them to think calling empty() has side effects if they call it by mistake.
  • The [[maybe_unused]] attribute lets you annotate an intentionally unused variable or parameter.
  • Various other minor improvements and fixes.

Evolution also approved the following features that were not voted in at this meeting, but are currently on track for possible approval for C++17 at our next meeting in June for inclusion in the C++17 international comment ballot (note the links are to the pre-meeting paper versions; for example, in the paper that proposed constexpr_if the committee changed the syntax to if constexpr at the meeting):

A few more, such as structured bindings, might make it too but have not yet received Evolution design approval, so we’ll see in June if there’s support in Evolution and Core to call them ready.

C++17 will change the way we write C++ code, just as C++11 and C++14 did. For example, string_view and optional are expected to be heavily used in writing interfaces. And with parallel STL often you can just add std::par or std::par_vec, and your algorithm will speed up by a factor of 2-4 on ordinary hardware; we had a compelling story with C++11 move semantics where we could say “just recompile your code and it’ll often be noticeably faster,” and this is likely to be an even bigger improvement.

Other progress

We also made significant progress on other work, including that we now have opened five TSes to launch our next round of beta feature branches, most of which we expect to be sent out for their “debutante” international comment ballots in the next handful of meetings:

  • Ranges, Networking, Library Fundamentals 2, and Parallelism 2: We approved creating these at our last meeting, and officially adopted the first working drafts for all four of these TSes at this meeting. These form the “first four” nucleus of the next round of TSes to be developed by the committee and are now being officially specified in detail.
  • Modules: Recall that at the last meeting we reached design agreement to start work on Modules. At this meeting, we approved the initial wording to officially start active work on a Modules specification as a TS. That brings us to five active TSes in this second round.

And there are more to come:

  • Coroutines: These are being targeted for a new TS (possibly Concurrency 2 or their own TS) so that we have additional time to finish investigating an alternative proposal. I expect this to settle out within another couple of years and then make progress in some form (either the current proposal, or the alternative one if it pans out equally well, or possibly both) into the following C++ standard.
  • Concurrency 2: The concurrency group reports that they hope to launch a second Concurrency TS later this year at the Issaquah meeting. This might include coroutines as above. It might include features like synchronic, and there is still hope that the group might yet get design agreement on executors (a way to control where to execute work, such as where to run a parallel task, or where to execute a callback or continuation); if so, it would naturally go into the second Concurrency TS.
  • 2D Graphics: The 2D Graphics proposal that specifies a portable C++ library to wrap the Cairo C API has progressed to design and wording review. It’s big, so it will take a couple of meetings to process it, but it’s looking good that we could get an initial TS draft started this year.

At this meeting we also reached a design agreement on contracts, so that we now think we have a design that the competing proposers agree on and that the committee as a whole likes, and we hope to get initial draft specification wording later this year if this trend continues. Once we see the work, the committee can decide on a preferred target vehicle (TS or straight into the IS).

The committee also continued to spend time at the meeting incubating other work, including reflection APIs.

The main surprise last week was that the “part 1” unified call syntax proposal made it to full committee but we there discovered that there wasn’t yet consensus to adopt it. The authors (Bjarne Stroustrup and myself with the help of a number of experts) are continuing to work between meetings to see if consensus can be achieved by the June meeting.

Finally, we retired a TS that was not completed: We ended up not getting consensus on the experimental material intended for the Arrays TS, so last week we made the decision to retire that one.

Status of concepts and modules

Here are some specific notes about two particular TSes that everyone agrees are important and game-changing for how we will write C++ code: Concepts and modules.

Concepts was published as a TS less than a year ago, with the first production compiler implementation to ship soon as part of GCC 6.0. Despite its relative newness and lack of field experience, the committee seriously considered adopting it already for C++17 at this meeting. It was right on the edge: There was already a strong minority who felt it was ready to add to C++17, and a modest majority preferred to wait another year or two and get more experience with shipping implementations and field use of the feature.

Virtually everyone who spoke praised the feature as important, and either ready or getting close to ready, for adopting into the C++ standard. The only major distinction of opinion was whether or not they supported putting it specifically in C++17; those who spoke against that generally expressed that this feature was so important that we want to be sure we get it right and approve it once we have more experience using it in real world users’ code. My impression is that concepts is now one of the strongest feature candidates moving along well in the pipeline and adoption in the trunk IS working draft post-C++17.

At the October meeting, modules for the first time achieved a design consensus in the Evolution working group (EWG), with a “phase 1” that everyone agreed on and a “phase 2” that could be added. The most notable part of “phase 2” is determining whether and how to allow a module to carry and export macros. In October, EWG decided to aim modules for a TS covering “phase 1” and asked the proposers to come back with detailed specification wording, which they did for this meeting.

Last week, the whole committee voted to start a TS project and turn that document into the first working draft. Here’s a subtle but important note: This milestone of “create an initial working draft” is important, but usually not all that impactful to the community. The reason this one is different, and is actually very important to users, is that we knew that there are likely two major vendor implementation efforts waiting to start work on implementing support for modules in major compilers, but who were waiting until there was some draft specification that had been approved as a basis for development by the committee. We knew that having “at least an initial working draft” was a sufficient bar for the implementers to get started, and so despite our heavy workload this week focusing on C++17-targeted items (which this is not), I asked the Core working group to make a special effort to prioritize getting this project to this stage because we knew it would unblock new implementation efforts. We are very pleased to see this succeed, and look forward to active work beginning soon on more implementations of modules in several major compilers.

The pipeline

There are members who are very disappointed that we didn’t include more in C++17, especially concepts which was very close. To understand where we’re at, consider the following snapshot of our current “trunk” and “beta feature branches” status, shown in the context of three eras of C++ standardization laid side by side: C++9x, C++0x, and C++1x.

wg21-timeline

The C++ international standard (IS) is the “trunk” containing everything that is formally part of C++. Every time it is published, we have a new official international legal specification for C++.

A Technical Specification (TS) is a “beta feature branch” shipped as a separate published document. The contents of a TS are the committee’s best guess at what C++ users need, with a lower confidence bar on that guess because it is a “beta” and we do get a chance to make corrections before setting it in stone in the standard. In 2015, we completed our first round of six published major TSes, and are launching a second round.

This is a turning point: We just finished filling the pipeline, and this meeting saw the first merge from the beta branches. The disappointment came from that some would have liked to merge another branch or two. The good news is that, those branches are already shipped beta specifications that vendors can implement—in fact, most of them already have at least one supported implementation in a production compiler or library product. The branches that did not quite land in time to merge for C++17 can be expected to merge early in the next cycle—and in the meantime we’re still shipping the “trunk” and its ready features too. Now that the pipeline is full, we can expect features to leave the pipeline and merge regularly at high quality, which is essential with over four million C++ users worldwide relying on our work.

Notes:

  • Each bar starts at the meeting where the detailed specification work officially began (usually “adopt initial draft”) and ends at the meeting that technical work was completed (usually “start final ballot / send for publication”).
  • Arrowheads join the IS at the meeting at which a TS feature branch was merged into the IS trunk.
  • In the current decade, the smaller boxes show full ISO C++ meetings—sometimes two per year, sometimes three per year—and highlights the meetings where we launch a C++ CD (ISO comment ballot, light blue) or DIS (ISO approval ballot, dark blue).
  • I’ve omitted the 1998-2001 hiatus because no new features were under active development in that period. The committee was voluntarily taking a break, and also catching up with C++98 bug reports while compilers caught up with the standard.

From “what” to “when”: C++0x to C++1x

During C++0x, the best and most solid parts of C++0x were the parts that shipped first as “beta” in the Library Extensions TR. That’s the model we’ve now embraced as the norm, rather than as the exception.

C++0x followed the “what” or “feature” model: We picked “what” features were in the standard and released whenever they turned out to be ready. That means we giving up on being able to predict “when” we’d ship, and even while C++0x was repeatedly delayed and the “0x is hex” jokes started, we released nothing in the standard the meantime; even features that had been approved into the C++ draft standard in 2004 had to wait until 2011 before they appeared in an official final standard. C++11 was a good release, but we underestimated “when” it would be ready by as much as five years.

C++1x follows the “when” or “train” model: We’re picking “when” to release the standard regularly, currently every three years, and including whatever features are ready. This still allows big long-pole features to be worked on; they’re just worked on concurrently until ready to merge. I’m pleased that the committee seems to be generally quite happy with the result. In Jacksonville, several experts came up to me to say that they had noticed how we had shipped more work in six years than in any other six-year period in the history of ISO C++. This would not be possible without the committee’s help and support; thank you again, everyone!

Although the train model has benefits, any model is also a tradeoff. The train model means we can’t slip a few meetings to try to “squeak in” another feature, and in fact we didn’t ship everything we hoped to ship in C++17, notably concepts… but neither did we at the same point in the C++0x cycle, in 2007. Now as in 2007, we are at a point where we have some good new features ready to ship in the standard, and at the same time we have several large desirable features in flight that are “almost but not quite” ready yet. In 2007, items on the edge or “just over the horizon” included work like threads and C++0x concepts, and in 2016 they include concepts “lite” and ranges. In both cases, those features are actually 3-4 years away (whether we knew it then or not). The difference is that in 2016, we’re shipping what is ready while still continuing development at full speed on the others.

Also, we initially thought a train model would let us alternate “minor” and “major” releases. The idea was that C++14 could be a “minor” release, and C++17 a “major” release. What actually happened was that we learned the pipeline+train model leads to regular “medium” releases as the pipeline moves at a steady pace and we regularly release what’s ready. As it turns out, C++14 was more than a fine-tuning release; it included powerful features like generic lambdas. Conversely, C++17 won’t be a “major release” on the scale of C++11, but neither could it be because C++11 took nine years to develop.

Never before in the history of C++ has the standard maintained such high specification quality, never before has it released on a regular predictable schedule… and as we removed that quality and schedule uncertainty, it’s no accident that never before have all the major compilers tracked the standard so closely, which benefits the whole worldwide C++ community.

How to define “progress”: Being unblocked to move to the next stage

How do we know we’re making progress? There is a tendency to emphasize “what features ship in C++.next,” a specific version; this was the committee’s mental model during C++0x.

In C++1x, the key is to keep the pipeline moving. What I monitor is that that major proposals are unblocked so the proposers can do work toward the next stage between meetings. As long as each proposal is not blocked, it’s moving forward toward the IS and “what ships in C++.next” will take care of itself as soon as it’s baked and ready.

Take a few minutes to read The life of an ISO proposal: From “cool idea” to “international standard.” Here are how some major work moved along at this meeting, in the context of those Stages:

  • Ranges, Networking, Library Fundamentals 3, and Parallelism 2: Stage 6 -> 7. All of these reached Stage 6 at our last meeting, and reached Stage 7 at this meeting. They’re all moving, and they’re well along in their development cycles and getting ready to debut in their international TS comment ballots in the near future.
  • Modules: Stage 3 -> 6. This reached Stage 3 at our previous meeting in November, and is already at Stage 6 at this meeting—that’s very rapid progress in one meeting!
  • Contracts: Stage 2 -> 3. This was stalled for a long time in Stage 2, and thanks to lots of work by the proposers (brokered personally by Bjarne Stroustrup) at the last meeting in November and then between meetings, it finally broke the logjam and reached Stage 3 at this meeting. Now it’s finally unblocked and on track again.
  • Coroutines, and 2D Graphics: Stage 3. These have both been at Stage 3 for a couple of meetings, and should reach Stages 4 and 5 over the next meeting or three.

If we focus primarily on each proposal moving incrementally to Stage N+1 and identifying and removing impediments, getting great new work in each new C++.next release of the standard becomes very nearly automatic.

What’s next

Our next meeting is this June in Oulu, Finland, where we plan to send a feature-complete C++17 out for its major international comment ballot. After that, we’ll address ballot comments at the November 2016 and February-March 2017 meetings and plan to send C++17 out for its final approval ballot then.

What’s after C++17? The default is to continue on the 3-year cycle (aim for C++20 then C++23), but we’re also thinking about whether to try out going to a 2-year cycle (aim for C++19, and if that works then C++21 and C++23). We’ll discuss that later this year as we see how C++17 and current/future TS plans are coming together, and should have more definite plans to report in a meeting or two.

Thank you again to the over 110 experts at Jacksonville last week, and the many more who participate in standardization through their national bodies, without whose efforts and willing collaborative spirit these results would not be achievable. The over four million C++ users worldwide benefit, and appreciate it very much too. Thank you, all, for your many contributions to Standard C++.

Acknowledgments

Thank you to everyone who provided input data and/or review of drafts of this trip report, including but not limited to: JF Bastien, Steve Clamage, Marshall Clow, Gabriel Dos Reis, William M. (Mike) Miller, Nathan Myers, Clark Nelson, Gor Nishanov, Peter Sommerlad, John Spicer, Bjarne Stroustrup, Ville Voutilainen, Jonathan Wakely, Nathan Wilson, and Jeffrey Yasskin.

20151024_012338818_iOS

Taking polls during the Friday afternoon plenary session — the room is hotter than it looks

Yesterday we just wrapped up our fall ISO C++ committee meeting in Kona, HI, USA. We normally meet in windowless rooms all week, but because of the committee’s current size we had to use the hotel’s largest space which was open-air (though technically still windowless).

It was a busy week. During the days from 8:00am-5:00pm, usually six sessions ran concurrently: Core (CWG), Evolution (EWG), Library (LWG), Library Evolution (LEWG), Concurrency (SG1), and usually at least one more subgroup such as Numerics. In the evenings, from 7:30-10:00pm, there were more evening sessions than evenings, and sometimes three would run concurrently. Daily sessions started at 8:00am, and most of us attended the evening sessions until 10:00pm each day, after which many people still went back to their rooms to update papers until midnight. (This schedule is pretty usual for ISO C++ meetings.)

What we did

We approved a number of actions — see STL’s great Reddit post for a summary, and some details on the library parts. Here are a few more notes from my own perspective.

First, actual major deliverables:

Concepts TS: It’s published! We did that between meetings, but it’s worth mentioning again here. There was initial discussion about putting it directly into C++17; we expect to seriously consider that at our next meeting.

Concurrency TS: We approved the final Concurrency TS for publication. It’s done!

Parallelism TS: The Concurrency group decided to pursue airlifting version 1 of this TS (which has already been published) into the draft for C++17, and we expect to see that proposed to the full committee at our next meeting. This week, we went ahead and also approved starting official work on version 2 of the Parallelism TS.

Ranges TS: We approved starting official work on a Ranges TS based on Eric Niebler’s great working text. The goal is to send it out for its main ballot in March following our next meeting.

Networking TS: We approved starting official work on a Networking TS based on Chris Kohlhoff’s excellent wording, evolved from his Boost.ASIO library.

Next, other major work:

Modules has reached a milestone: design agreement! This was done by separating “phase 1” and “phase 2” of modules, and greenlighting phase 1 to proceed to wordsmithing for adoption (hopefully at our next meeting) into a new Modules TS, while in parallel continuing work on an expanded phase 2 which can follow.

Variant has reached a milestone: design agreement! This consumed our main Monday evening session. For the first time, the broader set of committee members agreed on its design and we expect to see (and hopefully approve) actual wording at our next meeting. Its initial target will likely be Library Fundamentals v2. (Spoiler for those following along at home: It’s the same variant that LEWG approved at our last meeting in Lenexa, but after an exception renders the variant invalid, attempting to access the invalid variant causes defined behavior, such as throwing an exception, instead of undefined behavior. This was acceptable to three major constituencies: people who wanted a zero-overhead variant, people who wanted a variant without an omnipresent empty state, and people who wanted a variant without undefined behavior.)

Contracts seem to have made a breakthrough in getting the various parties to agree on the nitty-gritty details of a design. This consumed two three-hour evening sessions on Tuesday and Wednesday nights. If this holds, we expect to see a combined proposal that for the first time achieves broad agreement in time for our next meeting.

Reflection continues to make slow but steady progress. Chandler Carruth reminded us that several initial proposals have already started progressing through the committee.

There’s more, but those are some of the highlights.

Next meetings

Our next meetings (always listed here) will be Jacksonville, FL, USA on Feb 29 – Mar 5, 2016 and Oulu, Finland on Jun 20-25, 2016. Note that the Oulu meeting is when we plan to feature-complete C++17 and send it out for its main comment ballot, so that will be a big meeting. (It will also be within a few dozen kilometers of the Arctic Circle and is deliberately scheduled so that the summer solstice occurs during the meeting, so the joke is that there will likely again be lots of post-dinner sessions in Oulu, but for once we can guarantee there will be no evening sessions!)

My talk at CppCon is now available online: “Writing Good C++14… By Default” (slides)  It’s about type and memory safety for C++ — not a small target.

Definitely watch Bjarne’s keynote first. This talk is largely designed to be “part 2” of his keynote.

I’m very excited about the C++ Core Guidelines to promote modern C++14 style and this effort to achieve type and memory safety dovetails with that. Bjarne and I merged our efforts last winter, and with the help of many people brought it to this point where it’s ready to open up, and we hope this work can continue making progress.

The CppCon 2015 program is up! If anything this is an even stronger program than last year, which is saying something.

My name isn’t on it yet, but yes, I am giving a talk at CppCon. It should be announced this week.

I just posted my Lenexa ISO C++ trip report over on isocpp.org covering our recent meeting.

The ISO C++ committee is shipping more work sooner via concurrent Technical Specifications, but it’s still fairly new to find ourselves doing so much work that the “new normal” is to issue an international ballot from every ISO C++ meeting. This time, we completed three Technical Specifications (Transactional Memory, Library Fundamentals, and Parallelism) and sent out another for its comment ballot (Concurrency). We also did quite a bit of forward-looking planning for C++17, which is much closer than one would think since we’re aiming for a comment draft in mid/late 2016.

And check out the CppCon angle… it’s great to see the committee/community interaction and collaboration, and I can’t wait for CppCon in September and the fall ISO C++ meeting in October.

Follow

Get every new post delivered to your Inbox.

Join 3,002 other followers