Feeds:
Posts
Comments

As I mentioned earlier, part of my fall schedule is to give a repeat of this spring’s sold-out seminar: “High-Performance and Low-Latency C++” (October 9-11, London, UK).

I am still getting mails about whether there are alternative/additional European dates for this seminar. Unfortunately, the answer is still no, but since I’m getting inquiries about it let me repeat that part of the earlier post:

On October 9-11, I’ll be in London giving a one-time repeat of “High-Performance and Low-Latency C++” (course details page). This is the same as the public course I gave in Stockholm [in April]; because that course sold out, and I was coming to Europe again for Qt World Summit anyway, we decided to do a single repeat that same week, this time in London.

Notes: (1) Some of you have emailed me asking if there will be other dates/cities, and the answer is no, sorry, I do seminars very rarely and this is the last one I have time to do for the foreseeable future. So if you are interested then this is the one to attend. (2) Some of you have also emailed me to ask whether the seminar will be recorded, and the answer is again no, sorry, the organizers are not set up for that. However, you can find all of my past Effective Concurrency writing (on which parts of this course are based) freely available via this blog, just search for that phrase or use the category tag — there’s a book’s worth of free material written by me in individual-article form.

So, if you’re interested, I hope you’ll be able to attend this October, and I look forward to seeing many of you there.

I’ve been working on an experimental new C++ language feature tentatively called “metaclasses” that aims to make C++ programming both more powerful and simpler. You can find out about it here:

  • Current proposal paper: P0707R1. I hope the first ten pages give a readable motivation and overview. (The best two pages to start with are 9 and 10, which probably means I need to reorder the paper…)
  • Initial intro talk video: ACCU 2017 (YouTube). This is the initial public presentation three months ago. Thank you to Roger Orr, Russel Winder, Julie Archer, and the other ACCU organizers for inviting me and for holding back the video until we could have the ISO C++ summer meeting about a week ago, so it could go live along with a report (herein) on the results of this feature’s first presentation to the ISO C++ committee. And special thanks to Ina and Arvid, the two audience volunteers who graciously agreed to come on-stage to participate in a live mini UX study. There’s a lot of subtle information in their nuanced reactions to the code examples; pay special attention when their responses are different or as their responses evolve.
  • “Incomplete and experimental” prototype compiler. The Clang-based prototype by Andrew Sutton is available as an online live compiler at cppx.godbolt.org, and as source at github.com/asutton/clang. It’s incomplete but can compile a number of the examples in the paper (see the paper for example code links). Thanks to Matt Godbolt for hosting it on godbolt.org!

Please see the paper and video to answer “what are metaclasses and why should I care?” If you’re the “show me code first, English later” kind of person, try the live compiler and these quick examples: interface, base_class, value (regular type), plain_struct (these links are also in the paper).

The rest of this post aims not to duplicate any information above, but to provide some context about the broader journey, and what I and others are attempting to accomplish.

A journey: Toward more powerful and simpler C++ programming

Phase 1: By using the existing language better

About five years ago, I started working on long-term effort toward making using C++ simpler and safer.

In the first phase, a small group of us—centered on Bjarne Stroustrup, Gabriel Dos Reis, Neil MacIntosh and Andrew Pardoe—pushed to see how far we could get with “C++ as it is” plus just a few well-chosen library-only extensions, with a particular goal of improving type and memory safety. Bjarne, Neil, and I first publicly reported on this effort in the two CppCon 2015 plenary sessions “Writing Good C++14” and “Writing Good C++14… By Default.” The results of that work so far have manifested as the C++ Core Guidelines and its support library GSL that adds a limited number of library types (e.g., span, now being standardized); and I led the Lifetime design in particular (available in the Guidelines /docs folder) which Neil and I and others continue to work on formalizing with the aim of sharing a “draft” static analysis spec later this year.

One of the goals of this phase was to answer the question: “How much progress can we make toward simplifying the existing C++ language with only a few key library extensions?” The answer as I see it turned out to be: “Some solid progress, but probably not a major simplification.” And so that answer led to phase two…

Phase 2: By evolving the language

Two years ago, I started to focus specifically on exploring ways that we might evolve the C++ language itself to make C++ programming both more powerful and simpler. The only way to accomplish both of those goals at the same time is by adding abstractions that let programmers directly express their intent—to elevate comments and documentation to testable code, and elevate coding patterns and idioms into compiler-checkable declarations. The work came up with several potential candidate features where judiciously adding some power to the language could simplify code dramatically.

Of those potential candidate features, metaclasses is the first major piece I picked to propose for ISO C++. [*] We presented it for the first time at the summer ISO C++ meeting earlier this month, and it received a warm reception. There was (rare) unanimous support for pursuing this kind of capability, but also some concern about how best to expose it and specific design change feedback the committee wants us to apply to improve the proposal. [**] We’ll work to include in a revision for the November standards meeting as we start the multi-year process of vetting and refining the proposal. So this is good progress, but note that it (only) means encouragement to continue the experiment and see where it leads; it’s far too early to talk about potential ship vehicles.

So do expect change: The proposal is still evolving, and it in turn assumes and builds on the static reflection proposal (P0578 et al.) and the compile-time programming proposal (P0633), both of which are actively evolving in their own right. Incidentally, one of the contributions of Andrew Sutton’s prototype metaclasses compiler is that it is implementing those other proposals too(!), since the metaclasses feature needs them. The aim is to keep the latest compiler and the latest P0707 paper in sync with each other and with those related proposals, but there will doubtless be occasional drift in between syncs.

What’s next

I’ll talk about metaclasses more in my upcoming CppCon 2017 talk this September, and Andrew Sutton will also be giving two CppCon talks about metaclasses—one about implementing them in Clang, and one about using them for a real project.

This is just the beginning, and we’ll see whether it all pans out and leads somewhere, but I hope you enjoy this exploration and I look forward to talking with many of you about it at CppCon this September.

 

———

Notes

[*] I actually brought a smaller piece from this same work to the committee at the previous meeting, the winter meeting in Kona: P0515 (consistent comparisons), which proposes adding the three-way <=> comparison operator. P0515 is only about a minor feature, and not one of the most important things that can help improve C++, so normally I wouldn’t have picked that piece to contribute first; but the committee was already continuing to actively discuss comparisons, so I cherry-picked it from my design work and contributed it since I had the design in my pocket anyway. Happily the committee liked what they saw and both EWG and LEWG accepted it, and it is now progressing well and on track to hopefully be voted into draft C++20 in the next meeting or two. Thanks to Jens Maurer and Walter Brown for the heavy lifting of writing the core language and library standardese wording, respectively, for that P0515 proposal.

[**] The committee’s design feedback was primarily about how to wrap up the transformation code: Instead of putting it inside a new “meta” class-like abstraction, how about wrapping the same code inside a compile-time function-like abstraction that takes an input meta::type parameter and returns a generated meta::type return value? This doesn’t affect the proposal’s basic engine, just the shape of its steering wheel—for example, we could change the first line of each example metaclass definition from the class-like syntax

$class interface {
    constexpr {
        // … basically same code …
    }
};

to the decorator-function-like syntax

meta::type interface(const meta::type source) {
    // … basically same code …
};

where the latter has the advantage that it’s easy to see that we’re reading one type and generating another type. Interestingly, I think this dovetails with the mini UX study in the video where most of the difficulty the UX participants seemed to encounter was in understanding the $class syntax, not the metaclass bodies and not later using the metaclasses to author new types.

But we’ll explore this and other options and validate/invalidate it with more experiments… and feel free to express your thoughts in the comments if you like one of these styles better, or perhaps another variation.

No, we didn't meet at city hall, but this shot was taken from the primary hotel which was across the street from city hall. Also, this building appears in Star Trek: http://spacing.ca/toronto/2013/10/01/star-trek-toronto-city-hall/[This post will be updated with additional details as mentioned in the comments section at bottom.]

A few minutes ago, the ISO C++ committee completed its summer meeting in Toronto, Ontario, Canada. We had some 120 people at the meeting, representing nine national bodies. As usual, we met for six days Monday through Saturday, including several evenings.

The following are some highlights of what we achieved this week. You can find a brief summary of ISO procedures here. The main things to note are:

  • “IS” means “international standard.” In our case, it’s the core C++ standard itself. Think of this as “trunk.”
  • “TS” means “technical specification,” a document separate from the main standard where we can gain experience with new features before putting them into the IS. We have several of these, summarized on the status page. Think of these as “beta branches.”

Note: As I reported in my previous trip report for the winter meeting (Kona 2017), we have already completed our work on C++17, which is now still going through its final ISO approval steps. Nothing we did this week affects C++17, but we did begin work on C++20…

First meeting for C++20

This was the first meeting where we could vote changes into Draft C++20. And we did!

First, the Concepts TS was merged into draft C++20. Yes, Concepts are back in the draft international standard and better than C++0x, which makes the famous concepts feature (principally driven by Bjarne Stroustrup, Gabriel Dos Reis, and Andrew Sutton) the first major language feature voted into draft C++20. Recall that Concepts was published as a separate TS two years ago, and available in GCC; today we voted to adopt nearly of it, except only the “introducer syntax” and “terse/natural syntax” for now because that syntax still has a handful of remaining issues that don’t have consensus, but it’s close and Evolution also voted unanimously to aggressively pursue adding the terse syntax as soon as possible once those design questions can be addressed. A few specific Concepts changes were also approved along with the merge, notably removing the need to write “bool” and removing function concepts until we see a need to overload concepts.

Here are some other features that were added to C++20 at this meeting. Note: These links currently find the most recent pre-meeting papers and so may not reflect the exact wording adopted at the meeting, but the links will light up to the post-meeting versions of the papers that were actually adopted as soon as those are available in the post-meeting mailing about three weeks from now.

One TS sent for its ISO ballot, three TSes completed to be published

We also spent quite a bit of the week working on three TSes that completed their review ballots, and sending out one more: The Modules TS is sent out for its comment and approval (PDTS) ballot. And the Coroutines TS, Networking TS, and Ranges TS are all done, as the groups worked hard to process and address all the comments from their ballots and we are sending them out for final publication. – It’s not every meeting that we publish three specifications! Some of them would have been handled at our last meeting, but we were busy finishing C++17 so they stacked up for this meeting.

Thank you very much again to all the volunteers who helped progress our proposals, send out our PDTSes, address our ballot comments and publish three (3) TSes, and get the C++20 cycle off to a roaring start.

Other progress

We considered merging one other published TS, the Concurrency TS. The feeling was that its three major features were in different stages of readiness, and some are advancing toward bring merged into C++20, possibly at our next meeting:

  • atomic_shared_ptr<T> is on track to be merged into C++20 with changes, in detailed standardese wording review and expected to be put up for approval at our next meeting. Interestingly, some of the changes are to make it more like the original design I proposed, including to name it atomic<shared_ptr<T>>.
  • Latches are also approved, essentially as-is, and the feature is in wording review for the next meeting.
  • Barriers are also potentially on track for C++20 but SG1 is still discussing open design problems. Its design can be expected to change substantially from the TS version.
  • Some minor pieces of the future extensions are moving ahead, but the extension has several unresolved design issues including to coordinate with the (we hope soon-coming) executors design.

Other items making progress for possible consideration to be adopted in the fall meeting:

The Library Evolution group has started discussing at how to integrate modules and contracts into the standard library. They expect to start integrating concepts next meeting, which is a big benefit of getting a major feature merged early in the C++20 cycle. They are also actively working on Unicode support, several new containers, and a date library, among other things.

We also continued incubating other work. In particular, the Reflection study group had presentations, and gave direction and feedback, on a few additions to the static reflection proposal that is already progressing in the main subgroups, the compile-time programming proposal, and my new metaclasses paper. (I’ll post a separate report on the metaclasses part soon, probably in the next week or so, with a link to the ACCU video.)

What’s next

In my trip report last fall, two meetings ago, I wrote: “Then, once C++17 ships next year and possibly as soon as our July meeting, I expect we’ll start looking at merging more of the TS ‘beta branches’ into the C++ ‘trunk.’”

Well, that’s exactly what happened: We did finish C++17 at the next meeting, and now at the meeting after that we did merge our first major TS into the C++ trunk.

Here’s an updated snapshot of our status (the latest is always on the isocpp.org/std/status page):

wg21-timeline-2017-07b

Thank you again to the 120 experts in Toronto this week, and the many more who participate in standardization through their national bodies! Have a good summer – and see you at CppCon!

[6/26: Updated to add ACAT]

I can’t remember the last time I’ve gone to Europe twice in one year, but this is the year… up first are ACAT in August and CppCon in September, then a week later in early October I’ll be heading to London to give a single repeat of the three-day High-Performance and Low Latency C++ course, in the same week that I’ll be in Europe already to give a keynote at Qt World Summit in Berlin.

Aug 21-25: ACAT

This year I’ll be giving a keynote at ACAT (topic and day tbd). Unfortunately I cannot be there in person due to conflicts, but the organizers are graciously making arrangements for a remote presentation.

Mon-Fri Sep 25-29*: CppCon (Seattle)

I’ll be giving a brand-new talk at CppCon. If you haven’t registered yet for CppCon but are thinking of coming, note that the Early Bird discount ends on July 7, the end of next week.

* Those are the dates of the 5-day core conference, but technically the full dates are Sep 23 – Oct 1 because CppCon has grown into, somewhat incredibly, a nine-day event this year. How it happened: Last year, we added two days of optional pre-conference tutorials on the weekend before the conference as an experiment; those were so successful that this year there are tutorials on both weekends, before and after the conference, again as an experiment… but I know that at least a few of you have already signed up for all 9 days (wow).

Mon-Wed Oct 9-11: High-Performance and Low-Latency C++ (London)

On October 9-11, I’ll be in London giving a one-time repeat of “High-Performance and Low-Latency C++” (course details page). This is the same as the public course I gave in Stockholm two months ago; because that course sold out, and I was coming to Europe again for Qt World Summit anyway, we decided to do a single repeat that same week, this time in London.

Notes: (1) Some of you have emailed me asking if there will be other dates/cities, and the answer is no, sorry, I do seminars very rarely and this is the last one I have time to do for the foreseeable future. So if you are interested then this is the one to attend. (2) Some of you have also emailed me to ask whether the seminar will be recorded, and the answer is again no, sorry, the organizers are not set up for that. However, you can find all of my past Effective Concurrency writing (on which parts of this course are based) freely available via this blog, just search for that phrase or use the category tag — there’s a book’s worth of free material written by me in individual-article form.

Thu Oct 12: Qt World Summit (Berlin)

The next day, I’ll be in Berlin giving one of the keynotes at Qt World Summit. I’m not sure yet whether it’ll be the closing keynote; for more details, please check their conference schedule once it’s posted.

I look forward to seeing many of you at these events.

The ISO C++ committee had its winter meeting in Kona, HI, USA from February 27 to March 4, hosted by Plum Hall and the Standard C++ Foundation. Over 100 people attended, officially representing 9 countries.

C++17 is done!

The big news is that we completed C++17, which dominated the work of the meeting: C++17 is now technically finished and being sent out for its final ISO balloting. All that remains for C++17 now is some ISO red tape and minor touch-up to get it officially published, which is expected to be just mechanical.

There’s not much exciting to report from a “new features added at this meeting” point of view, because the main work was to finish resolving national body review comments on C++17 and ship the product. Perhaps the most exciting thing was that we decided to add the std::byte type; the rest of the work was final pre-release bug fixes before casting C++17 in stone, the important-but-unsung work of getting a product to ship quality.

Thank you very much to everyone who contributed in person or through papers, email, telecons and otherwise to creating and shipping C++17 — as well as several TSes that we also worked on concurrently and will soon be considering to add early in the C++20 cycle! Your hard work is much appreciated by everyone. I especially want to call out the Library working group (LWG) members, who had the heaviest C++17 workload in Kona and worked around the clock every day to finish responding to the comments in their areas. Thanks again, everyone. Here’s a group photo we took just outside our main meeting room (directly behind us) right after approving C++17:

Kona2017 full committee - crop

Next, we’re already shifting gears to begin work on C++20 at our next meeting this July in Toronto.

Other progress

We also approved shipping the Coroutines TS out for its (expected-to-be-only) international comment ballot, aka PDTS (Proposed Draft Technical Specification).

We came oh-so-close to approve shipping the Modules TS out for PDTS as well. The Evolution working group (EWG) weighed in on whether a couple of final cases should be included before sending it out for comments, and we hope to finalize that feature set and send it out in July in Toronto.

We also did work on the Parallelism 2 TS, and on a number of pre-TS proposals including 2D Graphics. On a personal note, my comparisons proposal was positively reviewed by both EWG and the Library Evolution working group (LEWG) (only the core proposal was encouraged, none of the parts marked ‘optional’) and wouldn’t have got that far without standing on the shoulders of giants — the previous comparisons proposals listed in the paper, notably Bjarne Stroustrup’s efforts and again Jens Maurer’s writing the standardese wording for Bjarne and now for me — and the detailed comments of over two dozen reviewers many of whom did detailed reviews of multiple drafts (it was a busy winter) — thank you again to everyone who helped me with that, it is at least 2x better as a result. If people are interested, I might give a short talk about it at CppCon this fall.

The Reflection proposal has now progressed out of the incubator Study Group, and was presented to the main Evolution and Library Evolution subgroups for the first time. To handle related proposals, the Reflection Study Group’s charter is now being expanded to also include proposals toward a general first-class compile-time programming model. For anyone who loves the glorious compile-time magic that C++ template metaprogramming (TMP) is able to do, but wish it could both do much more and also not be template metaprogramming with angle brackets and inscrutable code and diagnostics, then you’ll love the work in this direction; think “constexpr on steroids.” I’m looking forward to seeing how this evolves.

Both the Networking TS and the Ranges TS successfully completed their only (PDTS) ballots before Kona. The only work that remains before finalizing those is to address their PDTS ballot comments, which will largely be done in LWG. LWG was fully booked all week long with C++17 CD comments, but on Saturday started processing the PDTS comments as well. We plan to finish resolving Networking and Ranges comments and finalize those TSes at our next meeting in July.

In EWG, we looked at some feedback on the Concepts TS with an eye toward hopefully merging the Concepts TS into the C++20 working paper early in the C++20 cycle — possibly as soon as this year. There was one design tweak that EWG approved, and a couple more that will likely be discussed again in July in Toronto.

Here is an updated snapshot of our status, also available on the status page:

wg21-timeline-2017-03.png

Thank you again to the over 100 experts in Kona, and the many more who participate in standardization through their national bodies! Have a good spring… next stop: Toronto, to start C++20 this July.
Correction, 3/25: The original text did not reflect that LWG additionally spent a lot of time on Saturday processing Ranges PDTS issues. LWG did (thanks!), so we actually have a head start on those. As originally stated, we expect to complete the Ranges TS and Networking TS in July, and the Kona head start on PDTS comment resolution makes doing so that much easier.

This discussion today on the Core Guidelines repo issues is probably of broad interest. It’s regarding why we chose to annotate not_null<T*> rather than the reverse in the Guidelines and the Guideline Support Library (GSL).

Pasting here:


I would take this interface reduction one step further and make an un-annotated T* implicitly “not null”.

I understand, and we considered that.

We decided against that for several reasons:

  • T*, smart_ptr<T>, span<T>, container<T>::iterator, range<T>, etc. are all non-owning indirections and should be consistent with each other — it would be strange for some to be nullable but not others. Iterators can be “null”, for example a default-constructed iterator is not referring to anything.
  • More generally, all of those can be default-constructed, and the only reasonable semantics for that are “doesn’t point to anything.” (This can be a springboard for a broader discussion about the situations where default-constructible types are important, Regular types, etc.)
  • A large fraction of existing of T* are deliberately intended to be null, because people by convention use references for not-null parameters in particular and so in modern C++ code the presence of a T*parameter often (not always) implies nullability by that convention. So trying to annotate the “nullable” case is a huge code churn, and not only unadoptable but actually against the intent of much existing code.
  • Even if we ignored that and changed the default for T*, then we’d need to invent yet another annotation wrapper such as nullable<T>, and have to teach and explain both not_null<T> and nullable<T> (inconsistently).

For these and other reasons, we think that pointers should be nullable by default unless annotated otherwise.

valid concerns that are being dismissed because of a failure to distinguish between best practices for new code, and pragmatic recommendations for updating old code

I hope that helps reassure you that the concerns were considered deeply and aren’t being dismissed, and apply both to new code and old code. Defaults are important, and should reflect the common case especially for new code, but also for old code much of which is “correct” but just expressed without enough information about the intent because the programmer didn’t have the option or tool to express the intent.

The key issue is to distinguish maybe-null and never-null in the type system, and both of our approaches agree on doing that. Tony Hoare called null pointers his “billion-dollar mistake,” but in my opinion, and I think yours, the mistake was not maybe-null pointers (which are necessary, unavoidable, and pervasively present in every language with pointer/reference indirections, including Java, C#, C, C++, etc.), but rather in not distinguishing maybe-null and never-null pointers in the type system. You and we are both trying to do that, and so in the above I think we’re largely agreeing and our discussion is narrowly just about which one should be the default.

 

I don’t get to Europe very often apart from ISO C++ standards meetings, but this spring I’ve been able to accept invitations for two English-language European events in the last week of April. If you’re interested in attending, please check out the links, and I look forward to meeting and re-meeting many of you there.

Tue-Thu Apr 25-27: High-Performance and Low-Latency C++ (Stockholm)

On April 25-27, I’ll be in Stockholm (Kista) giving a three-day seminar on “High-Performance and Low-Latency C++.” This contains updated and new material that reflects the latest C++ standards and compilers, with a focus to using modern C++11/14/17 effectively on modern hardware and memory architectures.

Note that the class size is limited to about 100, so that I’ll be able to interact with most attendees directly. Registration just opened recently and hasn’t been widely publicized yet, but today I was told that it’s already over 1/3 full. So if you or your colleagues might be interested in attending, please check out the link above; for group registrations, please contact Alfasoft directly.

Here is the summary, below; for a more detailed topic breakdown see the link above.

Description

Performance and efficiency are C++’s bread and butter, and they matter more than ever on modern hardware: In processors, single-threaded performance improvements are slowing down (unless your code is parallel); in Internet of Things, we are often asked to do more work with less hardware; and in cloud computing, processor/hardware time is often the major component of cost and so making code twice as efficient often means saving close to half the processing cost. Today, getting the highest performance and the lowest latency on modern hardware often means being aware of the hardware in ways that most other programming languages can’t – from hardware caches where simply arranging our data in the right order can give 50x speedups with otherwise identical code, to hardware parallelism where using parallel algorithms turns on high-performance parallel and vector processor hardware that otherwise sits idle.

Additionally, low latency increasingly matters at all scales: In user interfaces, low latency means responsive apps and productive users without the dreaded “wait…” donut; in financial trading, low latency regularly saves large amounts of cash; in embedded real-time systems, low latency is crucial to meeting deadlines and can even save lives. Today, this makes concurrency more important than ever, because it delivers two things: It hides latencies we have to deal with and cannot remove, from disk I/O latency to speed-of-light network latency; and it makes our code responsive by not introducing needless latencies of our own even when we’re not hiding someone else’s latency.

Goal

This intensive three day course will provide developers with the knowledge and skills required to write high-performance and low-latency code on today’s modern systems using modern C++11/14/17. During the training you’ll learn how to get the highest performance and the lowest latency on modern hardware in ways that are unique to C++, including how to arrange data to use hardware caches effectively, and how to use standard and your own custom-written parallel algorithms to harness high-performance parallel and vector processor hardware to compute results faster. You’ll also learn how to manage latency for responsive apps and for real-time systems, and techniques to hide the underlying latencies we have to deal with and cannot remove such as disk and network latency, and to make your own code responsive by not introducing needless latencies in your own code.

Sat Apr 29: ACCU closing keynote (Bristol)

Next, I’ll be heading to Bristol to catch the end of the ACCU 2017 conference, and give the closing talk on “Something(s) New in C++.” No, the title is not intentionally a tease; it’s just that I have several topics available, and I won’t be sure until about a month before the event which will be the best one to speak about. Here is the current abstract:

By the time the ACCU 2017 conference begins, C++17 is expected to be technically complete and in its final approval ballot. What comes next? Will C++ continue growing forever? Can C++ code be simplified? This is a brand-new talk of material I’ve never given before, in which I’ll present one (or more) of three proposals I’m personally working on to further improve C++ post-C++17. All follow a common theme – adding a strategic language and/or library feature to C++ that leads to significant, and sometimes dramatic, simplification of real-world C++ code. I’ll pick which one (or more) of those topics to present sometime in March.

What I can say is that, whichever topic it ends up being, it’ll be something you haven’t seen before that’s forward-looking and aimed directly toward making C++ code simpler and easier… and of course without compromising C++’s model of efficient machine-near abstraction.

I look forward to seeing many of you in Europe this spring.