Month: November 2018

Trip report: Fall ISO C++ standards meeting (San Diego)

On Saturday November 10, the ISO C++ committee completed its fall meeting in San Diego, California, USA, hosted with thanks by Qualcomm. This was the biggest ISO C++ meeting in our 29-year history, with some 180 people at the meeting, representing 12 nations. For more details about our size increase, including how we adapted organizationally to handle the load, see my “pre-trip report” posted before the meeting began.

Because this is one of the last meetings for adding features to C++20, we gave priority to proposals that might make C++20, and we adopted a number of them for C++20. Thank you to all of the hundreds of people who participate in ISO C++, those who came to the meeting and still more who participated electronically, and who all helped with the design refinement and specification wording and organization. I want to at least try to recognize by name many of the authors of the proposals we adopted, but nobody succeeds with a proposal on their own. C++ is a team effort – this wouldn’t be possible without all of your help. So, thank you, and apologies for not being able to acknowledge everyone by name.

Notes:

  • Some of the links below are to papers that will not be published until the post-meeting mailing at the end of this month, and so the links will become public at that time.
  • For more details, see also the Reddit trip report.
  • You can find a brief summary of ISO procedures here.

Prelude: Focus

The committee is actively working to keep coherence and direction in the face of a tsunami of proposals and a huge number of enthusiastic people. Perhaps the most impactful record-setting number was the size of the pre-meeting mailing: 274 papers. For comparison, even excluding the biggest paper which was the updated C++ standard working draft which appears in every mailing, the pre-meeting mailing was enormous:

  • By word count, it exceeded Shakespeare’s complete published works.
  • By paper count, it exceeded the previous pre-meeting papers record by about 65%, and started to approach the total number of technical papers to produce the first C++ standard (total of pre-meeting mailings from 1990-1997).

We appreciate all the input, including that many of the papers are about bug fixes (always welcome) and rounding out existing features. However, a large number were proposals for new “good” features. And the trouble is that we can’t say yes to every feature that is “good” that benefits some users; we have to decide on a focused set, of at least coordinated features and ideally of general composable features, that fulfills the aim and mission of C++ and keeps the language adoptable and usable.

As I reminded in my pre-trip report, focus means saying no more often, and so we’ve taken several steps in recent meetings and at this meeting:

  • About a year ago, we created the Direction Group (Bjarne Stroustrup, Daveed Vandevoorde, Michael Wong, Howard Hinnant, Roger Orr, and the recently retired Beman Dawes) which has created and maintains an advisory document Direction for ISO C++ (P0939).
  • In 2018, Pearson Education made free electronic copies of The Design and Evolution of C++ (Bjarne Stroustrup) available to all committee members. Not only does this help ensure that the newer people have access to this expected-to-be-read foundational work, which continues to be very current, but also having it in machine-readable and -searchable form makes it much easier to reference and quote from in our own papers.
  • At this meeting, we let EWG and LEWG continue to focus on near-term papers, and created SG17 (EWG Incubator, JF Bastien) and SG18 LEWG Incubator (Bryce Adelstein Lelbach) to help review and improve the papers that EWG and LEWG could not handle, to improve them and to prune them before they reach the main subgroup. Thank you to JF and Bryce for being willing to chair these new groups!

Two major features adopted for C++20: Ranges, and Concepts convenience notation

Ranges (Eric Niebler, Casey Carter, Christopher Di Bella) was adopted for C++20. This was a tremendous amount of work by Casey Carter in particular (witness the recurring 3:00am editing update emails during the week). As Eric Niebler put it: “If you liked the Ranges TS, you’ll love C++20.”

Concepts “convenience” notation for constrained templates (Ville Voutilainen, Thomas Köppe, Andrew Sutton, Herb Sutter, Gabriel Dos Reis, Bjarne Stroustrup, Jason Merrill, Hubert Tong, Eric Niebler, Casey Carter, Tom Honermann, Erich Keane, Walter E. Brown, Michael Spertus) passed unanimously on Saturday. I highlighted this paper as a “we may have a winner here” in my last trip report, and indeed it sailed through and was adopted for C++20. Recall that we already added the concepts core feature to C++20 back in 2017, but without the convenience notation to write templates without the “template” or “requires” keywords; at this meeting we finally converged on a convenience syntax to write constrained templates that both addressed all the major problems people had identified in the Concepts TS convenience notation design, and was also acceptable to the primary concepts designers (hence the long list of coauthors). For the first time, besides the special case of generic lambdas, C++ will now let you write lots of generic functions without “template” or angle brackets, and that are concept-constrained and therefore much easier to use correctly than function templates have ever been before.

More constexpr adopted for C++20: Ongoing concerted push toward general compile-time programming

In the first part of my CppCon 2017 talk, I emphasized that C++ is serious about first-class compile-time programming. That is a general theme to current C++ evolution, and is particularly important for being able to make effective use of compile-time reflection, and for building on that further in the future for compile-time code generation such as my metaclasses proposal relies upon.

First-class compile-time programming in C++ has been building since we allowed simple one-liner constexpr functions in C++11, to constexpr functions with loops in C++14, to constexpr lambdas and “if constexpr” in C++17. This week, we have added still more as a coordinated set of additions to C++20:

We’re on track to making most “normal” C++ code available to run at compile time — and although C++20 won’t get quite all the way there, C++20 is a landmark release and a turning point where start to permanently leave behind the angle brackets and workarounds we’ve been using since the 1990s, with the near-complete birth of fully “natural” compile-time C++ code. Recall that we have already been adding support for user-defined types as template parameter types; soon (either before or after C++20’s feature freeze) we may be able to use strings as template arguments, and use containers like std::vector in compile-time code. Of course, there likely will be some limits; for example, supporting compile-time std::thread is possible, but less likely to be worth the effort.

Looking ahead to C++23 for a moment, where we expect still more of that plus (we hope) full static reflection in the standard, this marks a difficult-to-overstate landmark shift in C++ programming — not a course change, but really taking all the things that programmers have already been trying to do indirectly and giving it first-class natural support. The long-term results are likely to exceed our expectations in ways that we can’t fully anticipate yet. So fasten your seat belts, and stay tuned. C++ programming is likely to evolve more, and in better ways, in the upcoming 5 years than it already has in the past 20.

Other changes approved for C++20

A number of other smaller changes were adopted as well.

Other progress and decisions

Modules (Richard Smith; and Gabriel Dos Reis) for the first time had a unified design approved targeting C++20. Wording specification work will continue over the holidays, and we expect to consider modules for C++20 at our next meeting in February.

Executors: Thanks to progress between meetings and special meeting in September, we now are hopeful that an initial Executors design can make it for C++20. The feature was not merged at this meeting, but the design was approved for C++20 and we expect to consider adding the wording specification to C++20 at our next meeting.

Coroutines: We continued to make progress on coroutines. At this meeting, EWG again recommended merging the Coroutines TS (Gor Nishanov) into C++20, and this time EWG additionally explicitly included plans to incorporate features from the competing Core Coroutines proposal (Geoff Romer, James Dennett, Chandler Carruth). As in Rapperswil, the vote to merge the Coroutines TS into C++20 fell just short numerically and was not adopted for C++20 at this meeting. The proposers, and new collaborators from Facebook and other companies, will continue to work on improving consensus over the winter by addressing remaining concerns, including doing further work to merge features from Core Coroutines into the TS approach, such as an upcoming paper “A unifying design for Executors, Sender/Receiver, coroutines, parallel algorithms and networking” by Lewis Baker of Facebook which is expected to appear in the post-meeting mailing. We expect coroutines to be proposed again for C++20 at our February meeting in Kona with this additional information, and with the national bodies having more time to absorb the large amount of new information that was presented at this meeting.

Networking: This depends on Executors, and despite some discussion about decoupling the non-Executor parts, at this meeting we decided to target merging Networking into C++ for soon post-C++20 (i.e., targeting C++23). It also might depend on Coroutines, because some experts are still working through whether there is integration work to be done to merge Networking with Coroutines.

Reflection TS v1 (David Sankel) ISO ballot continues: The Reflection TS international comment ballot was already in progress during the meeting and will conclude next month. As I mentioned in my last trip report, note again that the TS’s current template metaprogramming-based syntax is just a placeholder; the feedback being requested is on the core “guts” of the design, and the committee already knows it intends to replace the surface syntax with a simpler programming model that uses ordinary compile-time code and not <>-style metaprogramming. In San Diego, we began looking at a “next-generation” reflection proposal  P1240 (Andrew Sutton, Faisal Vali, Daveed Vandevoorde).

2D Graphics (Michael B. McLaughlin, Herb Sutter, Jason Zink, Guy Davidson, Michael Kazakov) sent back to SG13: The SG13 HMI (human-machine interface) study group was reopened with Roger Orr as chair and will be taking another look at next steps for the Graphics proposal in the coming months.

Upcoming new work: Machine learning and education

In addition to the two Incubator Study Groups I mentioned in my pre-trip report, we also formed two new domain-specific study groups:

SG19: Machine Learning (Michael Wong). We feel we can leverage C++’s strengths in generic programming, optimization and acceleration, as well as code portability, for the specific domain of Machine Learning. The aim of SG19 is to address and improve on C++’s ability to support fast iteration, better support for array, matrix, linear algebra, in memory passing of data for computation, scaling, and graphing, as well as optimization for graph programming.

SG20: Education (JC van Winkel). We feel we have an opportunity to improve the quality of C++ education, to help software developers correctly use our language and ecosystem to write correct, maintainable, and performing software. SG20 aims to create curriculum guidelines for various levels of expertise and application domains, and to stimulate WG21 paper writers to include advise on how to teach the new feature they are proposing to add to the standard.

Thank you to Michael and JC for volunteering as chairs!

What’s next

Whew! Here is a cheat-sheet summary of our current reasonable expectations for some of the major pieces of work. Note that this is an estimate only, and progress can end up being different than expected.

wg21-proposal-progress

And here is an updated snapshot of where we are on the schedule for C++20, which can always be found in paper P1000:

wg21-schedule

Thank you again to the approximately 180 experts who attended this meeting, and the many more who participate in standardization through their national bodies! Have a good winter… we look forward now to several interim telecons and potentially side meetings, and then our next regular WG21 meeting in February (Kona, HI, USA).

Pre-trip report: Fall ISO C++ standards meeting (San Diego)

In one hour, our fall meeting will begin. I’ll still write a trip report at the end with the results of the meeting, but because this is an unusually (and historically) large meeting we’ve had to make a few adjustments.

This post is combined from a couple of administrative emails I sent to the committee over the past few weeks. Note that they’re written with that audience in mind, so they end with guidance about how to work effectively in the new subgroups, but I thought that others who follow C++ standardization would find them interesting and useful too.

New Study Groups: EWG Incubator and LEWG Incubator

As you may have noticed, we’ve been growing:

  • Record #people: 173 unique names known so far for San Diego.
  • Record #papers: 274 in pre-San Diego.

In the past we’ve had five main subgroups working during the week, but that doesn’t scale well to either the number of people (it’s not ideal to have ~170 in just five huge subgroups) or the number of papers (five subgroups will leave many papers unhandled). The following are some adjustments we’re making in order to scale organizationally.

Effective [October 19], we have formed two new Study Groups with the chairs shown below, both of which will meet in San Diego for three days each:

  • SG17 “EWG Incubator SG” (JF Bastien)
  • SG18 “LEWG Incubator SG” (Bryce Adelstein Lelbach)

Thank you very much to JF and Bryce for making themselves available to serve as chairs!

Notes:

  • The “(L)EWG Incubator” SGs are not additional (L)EWG groups, they are (like all SGs) a pipeline stage in front of EWG and LEWG respectively.
  • As advertised in our agenda, in San Diego (L)EWG will again prioritize C++20-targetable material first, and the Incubator SGs’ primary agendas will be to schedule those papers that are not likely to fit in the main (L)EWG groups. The chairs are coordinating their agendas. Please consult the daily agendas that will be maintained on the wiki to see what topics are scheduled for each session, so that you don’t miss your favorite topic.
  • Although SGs typically forward work to (L)EWG, they can as needed forward work to each other too if they believe the proposal needs domain expert input from that SG before entering (L)EWG. Papers that are primarily about a topic that is owned by an active topic-specific SG should still go to that SG; for example, all other SGs normally redirect any stray concurrency-related papers they receive to SG1 instead.
  • Like most SGs, we should expect the Incubator SGs will forward only a subset of what they receive to (L)EWG, sometimes after first asking for significant changes. As with all SGs, any papers that these SGs refine and forward need to pass through (L)EWG as usual, including to meet the IS ship deadlines described in P1000. As with all SGs[*], SG17 and SG18 never “pre-bless” proposals; if SG17/18 approves forwarding a proposal to (L)EWG, that just means it is ready to have an initial presentation in (L)EWG and begin its journey in those groups, not that it has any default positive standing in (L)EWG. There should never be an expectation of “well the SG blessed it, if we’d wanted to comment we should have been in the SG.”
    • [*] except for SG1 uniquely; proposals coming from SG1 do often have some advanced standing and trust because of SG1’s demonstrated expertise and history, but even SG1’s proposals get looked at by (L)EWG first and don’t skip (L)EWG entirely
  • The quantity of work that we process into the standard is still gated by CWG and LWG bandwidth. Therefore a major function of the Incubator SGs is act as a selection pipeline stage to turn higher input volume into higher average quality, and avoid the status quo default of papers backing up in front of EWG/LEWG/CWG/LWG.

We also have two expanded/reactivated Study Groups, both of which will meet in San Diego:

  • SG12 “UB, Undefined/Unspecifed Behavior” (Gabriel Dos Reis) is now regularly meeting for ~3 days, co-located again with our sister WG23 (Vulnerabilities) in San Diego. – So San Diego is again a combined WG21/WG23 meeting, not just WG21.
  • SG13 “HMI, I/O” (Roger Orr) is now reactivated to handle the Graphics proposal that does not currently have consensus to proceed in LEWG but has a critical mass of people actively interested in working on it, so it makes sense to reactivate the SG. Also, that proposal is actually a group of some 5-6 sub-proposals, so it will be up to SG13 to decide whether to incubate them as a group or individually, and then feed them to appropriate other subgroups for consideration when ready. Thank you to Roger for being willing to chair the group!

And we have a request for at least one new Study Group, though this will not be created until after San Diego:

  • Several companies have approached me to actively pursue work in Machine Learning. No concrete announcement yet, but I’m penciling in an ‘SG19 (chair tbd)’ for that.

So here’s an updated look at our current organization:

wg21-organization

The subgroups with heavy outlines are the ones that, starting in San Diego, will meet for multiple full days during our face-to-face meetings. You’ll notice there are now eight (8) of those, and so starting in San Diego our new standard meeting space requirement will be 8 concurrent breakout rooms (was previously 6, with the 6th shared among SGs and usually lightly utilized). Here’s a summary of the breakout tracks in San Diego (and of course we’ll have plenary first thing Monday and on Saturday – a few subgroups may choose to also meet a little longer on Saturday after plenary, as usual, and that will be announced on Saturday):

wg21-breakouts

(Insert your favorite “8-track” joke here, with de rigueur comment about C and C++ starting in the 1970s.)

Perspective: Our history and trajectory

I counted the #attendees (solid) and #papers (dashed) all the way back to my personal first meeting in July 1997. Here is the resulting graph.*

wg21-graph

Please go look at the quaint 1997 papers page, when we completed C++98 standardization. Fun facts:

  • The highest paper number was N1148. So we had 1148 papers in the entire 8-year cycle to produce C++98.
  • Pre- and post-meeting mailings were typically of a similar size. So we had ~600 pre-meeting papers in the entire 8-year cycle to produce C++98.
  • Many of those were issues lists, a higher proportion than today. And the technical papers were typically shorter than today’s.

So our San Diego technical paper volume is within a close order of all the technical papers to produce C++98. And, as already noted, it’s more than Shakespeare’s lifetime published word count, even excluding the C++ working paper.

Also in 1997, two months before the first meeting shown in the above graph, there was a famous Q&A session at WWDC that included the following quote:

So we had to decide what are the fundamental directions we’re going in and what makes sense and what doesn’t. There were a bunch of things that didn’t, and microcosmically they might have made sense but macrocosmically they made no sense…

Focus is about saying no, and the result of that focus is going to be some really great products where the total is much greater than the sum of the parts.

— Some Guy, 1997  (and Bjarne Stroustrup, 2018, paraphrasing)

What does “focus means saying no” mean to WG21?

We are a standards committee, not a design team. A design team can assign work (because it has employees), can make decisions without democratic vote (because it has a hopefully-benevolent dictator), and can just refuse to listen to ideas. A standards committee isn’t like that: In WG21, we are volunteers and we always review as many papers as we can (of course we assume the author/champion is present etc., see SD-4). For us, “saying no” doesn’t mean ignoring by fiat, it means deciding as a group to decline to go in that direction after first listening to it.

The way I think of this mailing’s papers is in five “buckets”, illustrated here:

wg21-focus.png

Yes, I can count. There’s a bucket 0 that’s not on the slide:

0. Thank you for your papers that used to be missing! In the past we’ve sometime had presentations and reflector mails without papers – aka “please write a paper,” and you did. Thank you for those.

1. Thank you for your bug-fix papers! We appreciate the many eyes to help our standard be better. We expect to take most of these (modulo agreeing on the right fix).

2. Thank you for your papers to improve an existing feature! We appreciate ways to remove limitations on our features (e.g., enable move-capture for lambdas). We expect to take some percentage of these (modulo agreeing on the right improvement).

3. Thank you for your new overlapping feature ideas! Sometimes there is a group of related narrower features where there is a lovable general feature struggling to hatch. We will try to merge-via-generalization N features into 1 general feature that can be orthogonal and composable with the rest of the language and library. Note that this frequently takes a number of meetings. Also, ideally it results in the ability to deprecate existing features by replacement-via-generalization (think how “using” subsumed “typedef”).

4. Thank you for still more new solo feature ideas! We will consider all of them, but for we expect to have a high bar for language proposals and require that they are composable with the rest of the language and clearly aligned with (a step in the direction of) where we want C++ to go in the next 10-30 years. And, over time, expect to see fewer of them, and to have them increasingly make existing features redundant (deprecate existing features by replacement-via-generalization, again).

Both #3’s merging and #4’s filtering procedures are aiming at the same goal: Producing a strong proposal that solves more than one problem.

What you can do (message to new subgroup participants)

Especially in #3 and #4, it’s unavoidable that the Incubator SGs will break hearts by saying “no.” So, please keep in mind the following tips.

When you are an Incubator SG participant, you will participate in polls that break hearts – so be kind, and especially watch for features that are #3’s (related to others and potentially generalizable) or look for ways to help them become #3’s (invite specific overlapping proposals that may not exist yet, to help complement and round out this proposal into something general), don’t just assume they’re #4’s (facing a difficult bar as a partly-fledged solo feature).

When you are an Incubator SG presenter, be prepared that not everyone may love your feature as much as you do, at least at first – but find ways to frame, and to evolve, your proposal to make it #3 instead of #4, especially by finding like-minded proposers of similar features that you can together turn into a #3 feature group and refine/complete/integrate together. Because it’ll be easier to succeed in #3 than in #4.

In general, we encourage all participants to consult:

  • P0939, the Direction Group’s suggestions – people are not expected to necessarily agree with every part but we should be open to the well-informed advice therein.
  • D&E, which is available for free to committee members – contact Bjarne to get a copy.

Although we can’t read all the mailing, we can read these two items.

 

Thank you to everyone, especially to the new subgroup chairs and the existing officers and chairs and upcoming meeting hosts, who have worked very hard at short notice, especially over the past few weeks, to help organize for our current growth.