WG21 has a strict schedule (see P1000) by which we ship the standard every three years. We don’t delay it.
Around this time of each cycle, we regularly get questions about “but why so strict?”, especially because we have many new committee members who aren’t as familiar with our history and the reasons why we do things this way now. And so, on the pre-Cologne admin telecon last Friday, several of the chairs encouraged me to write down the reasons why we do it this way, and some of the history behind the decision to adopt this schedule.
I’ve now done that by adding a FAQ section to the next draft of P1000, and I’ve sent a copy of it to the committee members now en route to Cologne. That FAQ material will appear in the next public revision of P1000 which will be in the post-meeting mailing a few weeks from now.
In the meantime, because the draft FAQ might be of general public interest too, here is a copy of that material. I hope you find it largely useful, occasionally illuminating, and maybe even a bit entertaining.
Now off to the airport… see (many of) you in Cologne. We are expecting about 220 people… by far our largest meeting ever. More about that in the post-meeting trip report after the meeting is over…
(As of pre-Cologne, July 2019) There are bugs in the standard, so should we delay C++20?
Of course, and no.
We are on our planned course and speed: Fixing bugs is the purpose of this final year, and it’s why this schedule set the feature freeze deadline for C++“20” in early “19” (Kona), to leave a year to fix bugs including to get a round of international comments this summer. We have until early 2020 (three meetings: Cologne, Belfast, and Prague) to apply that review feedback and any other issue resolutions and bug fixes.
If we had just another meeting or two, we could add <feature> which is almost ready, so should we delay C++20?
Of course, and no.
Just wait a couple more meetings (post-Prague) and C++23 will be open for business and <feature> can be the first thing voted into the C++23 working draft. For example, that’s what we did with concepts; it was not quite ready to be rushed from its TS straight into C++17, so the core feature was voted into draft C++20 at the first meeting of C++20 (Toronto), leaving plenty of time to refine and adopt the remaining controversial part of the TS that needed a little more bake time (the non-“template” syntax) which was adopted the following year (San Diego). Now we have the whole thing.
This feels overly strict. Why do we ship releases of the IS at fixed time intervals (3 years)?
Because it’s one of only two basic project management options to release the C++ IS, and experience has demonstrated that it’s better than the other option.
What are the two project management options to release the C++ IS?
I’m glad you asked.
There are two basic release target choices: Pick the features, or pick the release time, and whichever you pick means relinquishing control over determining the other. It is not possible to control both at once. They can be summarized as follows:
(1) “What”: Pick the features, and ship when they’re ready; you don’t get to pick the release time. If you discover that a feature in the draft standard needs more bake time, you delay the world until it’s ready. You work on big long-pole features that require multiple years of development by making a release big enough to cover the necessary development time, then try to stop working on new features entirely while stabilizing the release (a big join point).
This was the model for C++98 (originally expected to ship around 1994; Bjarne originally said if it didn’t ship by about then it would be a failure) and C++11 (called 0x because x was expected to be around 7). This model “left the patient open” for indeterminate periods and led to delayed integration testing and release. It led to great uncertainty in the marketplace wondering when the committee would ship the next standard, or even if it would ever ship (yes, among the community, the implementers, and even within the committee, some had serious doubts in both 1996 and 2009 whether we would ever ship the respective release). During this time, most compilers were routinely several years behind implementing the standard, because who knew how many more incompatible changes the committee would make while tinkering with the release, or when it would even ship? This led to wide variation and fragmentation in the C++ support of compilers available to the community.
Why did we do that, were we stupid? Not exactly, just inexperienced and… let’s say “optimistic,” for (1) is the road paved with the best of intentions. In 1994/5/6, and again in 2007/8/9, we really believed that if we just slipped another meeting or three we’d be done, and each time we ended up slipping up to four years. We learned the hard way that there’s really no such thing as slipping by one year, or even two.
Fortunately, this has changed, with option (2)…
(2) “When”: Pick the release time, and ship what features are ready; you don’t get to pick the feature set. If you discover that a feature in the draft standard needs more bake time, you yank it and ship what’s ready. You can still work on big long-pole features that require multiple releases’ worth of development time, by simply doing that work off to the side in “branches,” and merging them to the trunk/master IS when they’re ready, and you are constantly working on features because every feature’s development is nicely decoupled from an actual ship vehicle until it’s ready (no big join point).
This has been the model since 2012, and we don’t want to go back. It “closes the patient” regularly and leads to sustaining higher quality by forcing regular integration and not merging work into the IS draft until it has reached a decent level of stability, usually in a feature branch. It also creates a predictable ship cycle for the industry to rely on and plan for. During this time, compilers have been shipping conforming implementations sooner and sooner after each standard (which had never happened before), and in 2020 we expect multiple fully conforming implementations the same year the standard is published (which has never happened before). This is nothing but goodness for the whole market – implementers, users, educators, everyone.
Also, note that since we went to (2), we’ve also been shipping more work (as measured by big/medium/small feature counts) at higher quality (as measured by a sharp reduction in defect reports and comments on review drafts of each standard), while shipping whatever is ready (and if anything isn’t, deferring just that).
How serious are we about (2)? What if a major feature by a prominent committee member was “almost ready”… we’d be tempted to wait then, wouldn’t we?
Very serious, and no.
We have historical data: In Jacksonville 2016, at the feature cutoff for C++17, Bjarne Stroustrup made a plea in plenary for including concepts in C++17. When it failed to get consensus, Stroustrup was directly asked if he would like to delay C++17 for a year to get concepts in. Stroustrup said No without any hesitation or hedging, and added that C++17 without concepts was more important than a C++18 or possibly C++19 with concepts, even though Stroustrup had worked on concepts for about 15 years. The real choice then was between: (2) shipping C++17 without concepts and then C++20 with concepts (which we did), or (1) renaming C++17 to C++20 which is isomorphic to (2) except for skipping C++17 and not shipping what was already ready for C++17.
What about something between (1) and (2), say do basically (2) but with “a little” schedule flexibility to take “a little” extra time when we feel we need to stabilize a feature?
No, because that would be (1).
The ‘mythical small slip’ was explained by Fred Brooks in The Mythical Man-Month, with the conclusion: ”Take no small slips.”
For a moment, imagine we did slip C++20. The reality is that we would be switching from (2) back to (1), no matter how much we might try to deny it, and without any actual benefit. If we decided to delay C++20 for more fit-and-finish, we will delay the standard by at least two years. There is no such thing as a one-meeting or three-meeting slip, because during this time other people will continue to (rightly) say “well my feature only needs one more meeting too, since we’re slipping a meeting let’s add that too.” And once we slip at least two years, we’re saying that C++20 becomes C++22 or more likely C++23… but we’re already going to ship C++23! — So we’d still be shipping C++23 on either plan, and the only difference is that we’re not shipping C++20 in the meantime with the large amount of fully-baked work that’s ready to go, and making the world wait three more years for it. Gratuitously, because the delay will not benefit those baked features, which is most or all of them.
So the suggestion amounts to “let’s make C++20 be C++22 or C++23,” and the simple answer is “yes, we’re going to have C++23 too, but in addition to C++20 and not instead of it.” To delay C++20 actually means to skip C++20, instead of releasing the great good work that is stable and ready, and there’s no benefit to doing that.
But feature X is broken / needs more bake time than we have bugfix time left in C++20!
No problem! We can just pull it.
In that case, someone needs to write the paper aimed at EWG or LEWG (as appropriate) that shows the problem and/or the future doors we’re closing, and proposes removing it from the IS working draft. Those groups will consider it, and if they decide the feature is broken (and plenary agrees), that’s fine, the feature will be delayed to C++next. We have actually done this before, with C++0x concepts.
But under plan (1), we would be delaying, not only that feature, but the entire feature set of C++20 to C++23! That would be… excessive.
Does (2) mean “major/minor” releases?
No. We said that at first, before we understood that (2) really simply means you don’t get to pick the feature set, not even at a “major/minor” granularity.
Model (2) simply means “ship what’s ready.” That leads to releases that are:
- similarly sized (aka regular medium-sized) for “smaller” features because those tend to take shorter lead times (say, < 3 years each) and so generally we see similar numbers completed per release; and
- variable sized (aka lumpy) for “bigger” features that take longer lead times (say, > 3 years each) and each IS release gets whichever of those mature to the point of becoming ready to merge during that IS’s time window, so sometimes there will be more than others.
So C++14 and C++17 were relatively small, because a lot of the standardization work during that time was taking place in long-pole features that lived in proposal papers (e.g., contracts) and TS “feature branches” (e.g., concepts).
C++20 is a big release …
Yes. C++20 has a lot of major features. Three of the biggest all start with the letters “co” (concepts, contracts, coroutines) so perhaps we could call it co_cpp20. Or co_dependent. Wait, we’re digressing.
… and so aren’t we cramming a lot into a three-year cycle for C++20?
No, see “lumpy” above.
C++20 is big, not because we did more work in those three years, but because many long-pole items (including at least two that have been worked on in their current form since 2012, off to the side as P-proposals and TS “branches”) happened to mature and get consensus to merge into the IS draft in the same release cycle.
It has pretty much always been true that major features take many years. The main difference between plan (1) for C++98 and C++11 and plan (2) now is: In C++98 and C++11 we held the standard until they were all ready, now we still ship those big ones when they’re ready but we also ship other things that are ready in the meantime instead of going totally dark.
C++20 is the same 3-year cycle as C++14 and C++17; it’s not that we did more in these 3 years than in the previous two 3-year cycles, it’s just that more long-pole major features became ready to merge. And if any really are unready, fine, we can just pull them again and let them bake more for C++23. If there is, we need that to be explained in a paper that proposes pulling it, and why, for it to be actionable.
In fact, I think the right way to think about it is that C++14+17+20 taken as a whole is our third 9-year cycle (2011-2020), after C++98 (1989-1998) and C++11 (2002-2011), but because we were on plan (2) we also shipped the parts that were ready at the 3- and 6-year points.
Isn’t it better to catch bugs while the product is in development, vs. after it has been released to customers?
But if we’re talking about that as a reason to delay the C++ standard, the question implies two false premises: (a) it assumes the features haven’t been released and used before the standard ships (many already have production usage experience); and (b) it assumes all the features can be used together before the standard ships (they can’t).
Re (a): Most major C++20 features have been implemented in essentially their current draft standard form in at least one shipping compiler, and in most cases actually used in production code (i.e., has already been released to customers who are very happy with it). For example, coroutines (adopted only five months ago as of this writing) has been used in production in MSVC for two years and in Clang for at least a year with very happy customers at scale (e.g., Azure, Facebook).
Re (b): The reality is that we aren’t going to find many feature interaction problems until users are using them in production, which generally means until after the standard ships, because implementers will generally wait until the standard ships to implement most things. That’s why when we show any uncertainty about when we ship, what generally happens is that implementations wait – oh, they’ll implement a few things, but they will hit Pause on implementing the whole thing until they know we’re ready to set it in stone. Ask <favorite compiler> team what happened when they implemented <major feature before it was in a published standard>. In a number of cases, they had to implement it more than once, and break customers more than once. So it’s reasonable for implementers to wait for the committee to ship.
Finally, don’t forget the feature interaction problem. In addition to shipping when we are ready, we need time after we ship to find and fix problems with interactions among features and add support for such interactions that we on typically cannot know before widespread use of new features. No matter how long we delay the standard, there will be interactions we can’t discover until much later. The key is to manage that risk with design flexibility to adjust the features in a compatible way, not to wait until all risk is done.
The standard is never perfect… don’t we ship mistakes?
If we see a feature that’s not ready, yes we should pull it.
If we see a feature that could be better, but we know that the change can be done in a backward-compatible way, that’s not a reason to not ship it now; it can be done as an extension in C++next.
We do intentionally ship features we plan to further improve, as long as we have good confidence we can do so in a backward-compatible way.
But shouldn’t we aim to minimize shipping mistakes?
Yes. We do aim for that.
However, we don’t aim to eliminate all risk. There is also a risk and (opportunity) cost to not shipping something we think is ready. So far, we’ve been right most of the time.
Are we sure that our quality now is better than when were on plan (1)?
By objective metrics, notably national body comment volume and defect reports, C++14 and C++17 have been our most stable releases ever, each about 3-4 times better on those metrics than C++98 or C++11. And the reason is because we ship regularly, and put big items into TS branches first (including full wording on how they integrate with the trunk standard) and merge them later when we know they’re more baked.
In fact, since 2012 the core standard has always been maintained in a near-ship-ready state (so that even our working drafts are at least as high quality as the shipped C++98 and C++11 standards). That never happened before 2012, where we would often keep the patient open with long issues lists and organs lying around nearby that we meant to put back soon; now we know we can meet the schedule at high quality because we always stay close to a ship-ready state. If we wanted to, we could ship the CD right now without the Cologne meeting and still be way higher quality than C++98’s or C++11’s CDs (or, frankly, their published standards) ever were. Given that C++98 and C++11 were successful, recognizing that we’re now at strictly higher quality than that all the time means we’re in a pretty good place.
C++98 and C++11 each took about 9 years and were pretty good products …
Yes: 1989-1998, and 2002-2011.
… and C++14 and C++17 were minor releases, and C++20 is major?
Again, I think the right comparable is C++14+17+20 taken as a whole: That is our third 9-year cycle, but because we were on plan (2) we also shipped the parts that were ready at the 3- and 6-year points.
Does (2) allow making feature-based targets like P0592 for C++next?
Sure! As long as it doesn’t contain words like “must include these features,” because that would be (1).
Aiming for a specific set of features, and giving those ones priority over others, is fine – then it’s a prioritization question. We’ll still take only what’s ready, but we can definitely be more intentional about prioritizing what to work on first so it has the best chance of being ready as soon as possible.
9 thoughts on “Draft FAQ: Why does the C++ standard ship every three years?”
“Or co_dependent. Wait, we’re digressing.” Is that what’s called a co_tangent?
For several decennies, ITU-T (or CCITT, as it was named most of those years) had a strict 4-year cycle for releasing their “recommendations”/standards, such as X.21, X.25, T.400, X.500 etc. This received a lot of critisism: It was inflexible, couldn’t accomodate developments done throgh that 4 year period, development of different standards were forced to be synchronized to the same schedule, and so on. ISO, the International Standards Organization, never had any such regular update intervals.
So ITU abandoned their 4-year cycle. If my memory is correct, the 1992 recommendations were the last ones on the 4-year cycle. It is sort of funny that the C++ guys have returned to the practice that was abondoned 25+ years ago.
The distribution of the official standard must follow ISO rules. As much as I know, this is the reason it isn’t available for free.
Still, you always have access to the up-to-date draft online here: http://eel.is/c++draft/
You also have the LaTeX sources here: https://github.com/cplusplus/draft
You may find the additional documents in the following page useful: http://www.open-std.org/jtc1/sc22/wg21/
But above all, I don’t think many people should read the standard. It’s too technical text, that I suggest avoiding unless you must (e.g. if you are a compiler implementer). For any practical usage, sites like https://cppreference.com are much more relevant. BTW, the main blocker for using a new standard is the toolset availability with the implementation of it. Having the standard text handy doesn’t help if the compiler you use haven’t implemented the new standard yet.
Why is C++ standard (or C standard) not available free of charge for developers, academics and students to download and implement them? The only way to market these two languages is to make the relevant documents available for free download. when you have 500 million people downloading them, a proportion of them will start using the new standard and spread it in the community.
My 2 pence here.
Thanks for the FAQ but I’m still curious why it’s three-year cycle? Not a strict four-year cycle, or five-year cycle, etc?
@Buge: The next revision of P0592 has removed that particular phrasing (I pointed out the problem and Ville immediately recognized it and agreed).
Typos fixed, thanks. (The 3-4′ was 3-4x but I guess I used a Unicode “x” multiplication symbol and it didn’t survive the paste from Word to WordPress (it got ‘pressed?)…
(Would be nice if it’d be possible to subscribe to notifications on new comments without posting a new comment)
Thanks for writing it down!
I’m constantly surprised by the number of people who aren’t aware of this project management decision and the background for it, some of them are active committee members.
The point that was totally new for me was about the objective metrics for the quality of the standard. Good to hear that!
I don’t understand that P0592 point, it says “must-have-do-not-fail-to-ship items”, but without reading P0592 it seems like you’re saying P0592 is ok.
Couple small typos: “(Ask team” never closes the parenthesis, “3-4´ better” seems to have a weird character.
Comments are closed.