Trip Report: February/March 2008 ISO C++ Standards Meeting

Visual C++ 2008 Feature Pack now available »

Trip Report: February/March 2008 ISO C++ Standards Meeting

2008-03-29 by Herb Sutter

[Updated Apr 3 to note automatic deduction of return type.]

The ISO C++ committee met in Bellevue, WA, USA on February 24 to March 1, 2008. Here’s a quick summary of what we did (with links to the relevant papers to read for more details), and information about upcoming meetings.

Lambda functions and closures (N2550)

For me, easily the biggest news of the meeting was that we voted lambda functions and closures into C++0x. I think this will make STL algorithms an order of magnitude more usable, and it will be a great boon to concurrent code where it’s important to be able to conveniently pass around a piece of code like an object, to be invoked wherever the program sees fit (e.g., on a worker thread).

C++ has always supported this via function objects, and lambdas/closures are merely syntactic sugar for writing function object. But, though “merely” a convenience, they are an incredibly powerful convenience for many reasons, including that they can be written right at the point of use instead of somewhere far away.

Example: Write collection to console

For example, let’s say you want to write each of a collection of Widgets to the console.

// Writing a collection to cout, in today’s C++, option 1:

for( vector<Widget>::iterator i = w.begin(); i != w.end(); ++i )
cout << *i << ” “;

Or we can leverage that C++ already has a special-purpose ostream_iterator type that does what we want:

// Writing a collection to cout, in today’s C++, option 2:

copy( w.begin(), w.end(),
ostream_iterator<const Widget>( cout, ” ” ) );

In C++0x, just use a lambda that writes the right function object on the fly:

// Writing a collection to cout, in C++0x:

for_each( w.begin(), w.end(),
[]( const Widget& w ) { cout << w << ” “; } );

(Usability note: The lambda version was the only one I wrote correctly the first time as I tried these examples on compilers to check them. ‘Nuff said. <tease type=”shameless”> Yes, that means I tried it on a compiler. No, I’m not making any product feature announcements about VC++ version 10. At least not right now. </tease>)

Example: Find element with Weight() > 100

For another example, let’s say you want to find an element of a collection of Widgets whose weight is greater than 100. Here’s what you might write today:

// Calling find_if using a functor, in today’s C++:

// outside the function, at namespace scope
class GreaterThan {
int weight;
public:
GreaterThan( int weight_ )
: weight(weight_) { }
bool operator()( const Widget& w ) {
return w.Weight() > weight;
}
};

// at point of use
find_if( w.begin(), w.end(), GreaterThan(100) );

At this point some people will point out that (a) we have C++98 standard binder helpers like bind2nd or (b) that we have Boost’s bind and lambda libraries. They don’t really help much here, at least not if you’re interested in having the code be readable and maintainable. If you doubt, try and see.

In C++0x, you can just write:

// Calling find_if using a lambda, in C++0x:

find_if( w.begin(), w.end(),
[]( Widget& w ) { return w.Weight() > 100; } );

Ah. Much better.

Most algorithms are loops… hmm…

In fact, every loop-like algorithm is now usable as a loop. Quick examples using std::for_each and std::transform:

for_each( v.begin(), v.end(), []( Widget& w )
{
…
… use or modify w …
…
} );

transform( v.begin(), v.end(), output.begin(), []( Widget& w )
{
…
return SomeResultCalculatedFrom( w );
} );

Hmm. Who knows: As C++0x lambdas start to be supported in upcoming compilers, we may start getting more used to seeing “});” as the end of a loop body.

Concurrency teaser

Finally, want to pass a piece of code to be executed on a thread pool without tediously having to define a functor class out at namespace scope? Do it directly:

// Passing work to a thread pool, in C++0x:

mypool.run( [] { cout << “Hello there (from the pool)”; } );

Gnarly.

Other approved features

N2535 Namespace associations (inline namespace)
N2540 Inheriting constructors
N2541 New function declarator syntax
N2543 STL singly linked lists (forward_list)
N2544 Unrestricted unions
N2546 Removal of auto as a storage-class specifier
N2551 Variadic template versions of std::min, std::max, and std::minmax
N2554 Scoped allocator model
N2525 Allocator-specific swap and move behavior
N2547 Allow lock-free atomic<T> in signal handlers
N2555 Extended variadic template template parameters
N2559 Nesting exceptions (aka wrapped exceptions)

Next Meetings

Here are the next meetings of the ISO C++ standards committee, with links to meeting information where available.

June 8-14, 2008: Sophia Antipolis, France
September 14-20, 2008: San Francisco Bay area, California, USA

The meetings are public, and if you’re in the area please feel free to drop by.

Posted in C++ | 34 Comments

34 Responses

on 2008-04-04 at 3:28 pm (no name)

Just found mistake in my previous post. It could be

find_if( w.begin(), w.end(),
[]( decltype(*w.begin()) w ) { return w.Weight() > 100; } );

It doesn’t look good and prove that [] (auto& w) would be nice to have :)
on 2008-04-04 at 12:12 pm (no name)

One nice improvement that should works for every iterator (just found here in Herb’s older post)

find_if( w.begin(), w.end(),
[]( decltype(*w) w ) { return w.Weight() > 100; } );
on 2008-04-04 at 9:58 am (no name)

I’m second (or nth) for [] (auto& w) addition. However without it one may use decltype to achieve almost the same for standard containers

find_if( w.begin(), w.end(),
[]( decltype(w)::value_type& w ) { return w.Weight() > 100; } );

However it raises one question. Will "decltype(w)::value_type& w" be compiled? We use the same identifier (w) for the container and for its element here. In Herb’s examples without decltype(w) this didn’t get any problem, but what is about this case?
on 2008-04-03 at 2:18 pm Herb Sutter

Several people said they’d like the feature to "deduce the return type if there’s only one return statement."

I have happy news: I was wrong when I said this feature didn’t make it in, because it turns out it did.

That means instead of:

find_if( w.begin(), w.end(),
             []( Widget& w ) -> bool { return w.Weight() > 100; } );

we can indeed write:

find_if( w.begin(), w.end(),
             []( Widget& w ) { return w.Weight() > 100; } );

And instead of:

transform( v.begin(), v.end(), output.begin(), []( Widget& w ) -> AnotherType
{
    …
return SomeResultCalculatedFrom( w );
} );

we can write:

transform( v.begin(), v.end(), output.begin(), []( Widget& w )
{
    …
return SomeResultCalculatedFrom( w );
} );

I’ve updated the blog entry to show that. Thanks to my colleague Jonathan Caves (one of our VC++ compiler engineers extraordinaire!) for pointing this out to me, and that he <cough> already implemented that part too <cough>.
on 2008-04-03 at 2:59 am Vivek Ragunathan

Hi Herb and All

I see my comments posted multiple times here. I guess my slow internet connection and impatience (multiple clicks on the Submit button) should have done that. Apologies for the ugliness.

Herb, If you feel the multiple repeatitive comments here look ugly and would annoy the readers, please delete them (except one).

Regards
on 2008-04-02 at 6:46 pm Alex

Herb, could you explain the new allocator model in plain English? It’s hard to undrstand N2554, but does it promise doubling the number of constructors for each library class? And there are concepts that tell the compiler which constructor to choose? All this sounds scary.
on 2008-04-02 at 9:04 am Tanveer Badar

This is cooler than COOL.

We can finally smack those C# coders in their faces.

*devil*

Can’t you do anything to everyone posting gazillions copies of same comment?
on 2008-03-31 at 3:18 am Vivek Ragunathan

Hi Herb

"What if you want a local copy? You say to pass it by value, but for safety reasons the current proposal says you get a read-only copy that you can’t modify:

    for_each( v.begin(), v.end(), [numWidgets]( Widget& w )
    {
        int i = numWidgets; // ok
        ++i;
        // "++numWidgets;" would be an error
    } );"

The numWidgets getting passed as a read-only copy and ++numWidgets giving an error seems ugly. Ok, I understand the intent of the lambda/syntax there.

Wouldn’t it be better if the name inside the [] just refer to the variable to be captured (Just like the argument and the parameter can have the same names but a parameter without & would mean only a copy). And in above case, any modifications are only local. Does that make sense ?

Anyways, C++ with lambdas and stuff, we are becoming Gods !

Regards

Vivek Ragunathan
on 2008-03-30 at 10:58 am Leo Sutic

(I was the one asking about captures.)

Ok, thanks for the explanation of captures – that makes a lot of sense.

Finally, I have to second the calls for "less boilerplate" – although I think a balance must be struck where we don’t have too much magic and too many things happening implicitly. I think the [](auto& w) has that balance – it allows you to name the input parameters (I think _1, _2 and so on may make it easier to write, but makes reading the code difficult), but avoid requiring the programmer to add redundant information.
on 2008-03-30 at 8:08 am Peter Dimov

Still no luck. :-) Sorry for the comment spam.
on 2008-03-30 at 8:05 am (no name)

Let’s see if this time I’m less unnamed.
on 2008-03-30 at 7:40 am Herb Sutter

(no name) wrote: I agree with Aleksey. "They don’t help much here" is an overstatement. The comparison with boost::bind mostly serves to highlight the "Widget const&" and "bool" redundant boilerplate instead of casting the lambdas in the favorable light they deserve. You could’ve chosen a better example.

Okay. But for what it’s worth, two extensions that were discussed but not approved (at least yet), but that would be conforming extensions in a compiler that implements this proposal, are:

– automatically deduce the return type (if there’s only one return statement)

– provide ‘auto’ type deduction on the parameters (note: not currently allowed by the general C++0x ‘auto’ feature)

Then you could write:

find_if( w.begin(), w.end(),
[]( auto& w ) { return w.Weight() > 100; } );

which I agree would be even slicker.

And then, if you allowed _1 and _2 etc. as shorthand for (auto _1, auto _2, etc.) you could get down to:

find_if( w.begin(), w.end(),
[] { return _1.Weight() > 100; } );

as a conforming extension, which I personally find attractive (it’s nearly the ideal minimal syntax; note that "_1.Weight() > 100" is the syntax everyone first tries with Boost.Lambda when a member function is involved, only to find it doesn’t work…) but I agree is controversial. So I do want to empahsize this is NOT currently supported in C++0x lambdas as they are in the draft now, it’s just a door that is left open in that there’s nothing to preclude such further extensions/improvements in the future without breaking compatibility.

If Aleksey, (no name), or others want to lobby for these further simplifications/extensions, now’s the time to send email to the committee reflectors with your reasons and use cases! :-) I’m not going to lobby hard, though; I’m just happy to get basic lambdas in.
on 2008-03-30 at 7:32 am Herb Sutter

Re binders: Okay, I give! I’ll use a better example next time.

(no name) asked: "How are local variables captured?" You have to specify whether it’s by copy or by reference. So this example is illegal because it tries to use a local variable:

    int numWidgets = 0;
    for_each( v.begin(), v.end(), []( Widget& w )
    {
      ++numWidgets; // error, numWidgets is not in scope
    } );

If you want to update numWidgets directly, capture it by reference:

    for_each( v.begin(), v.end(), [&numWidgets]( Widget& w )
    {
      ++numWidgets; // increments original numWidgets
    } );
    // numWidgets == v.size() here

Or use the shorthand [&] to take all captured variables implicitly by reference:

    for_each( v.begin(), v.end(), [&]( Widget& w )
    {
      ++numWidgets; // increments original numWidgets
    } );
    // numWidgets == v.size() here

What if you want a local copy? You say to pass it by value, but for safety reasons the current proposal says you get a read-only copy that you can’t modify:

    for_each( v.begin(), v.end(), [numWidgets]( Widget& w )
    {
        int i = numWidgets; // ok
        ++i;
        // "++numWidgets;" would be an error
    } );
    // numWidgets == 0 here

Or use the shorthand [=] to take all captured variables implicitly by copy:

    for_each( v.begin(), v.end(), [=]( Widget& w )
    {
        int i = numWidgets; // ok
        ++i;
        // "++numWidgets;" would be an error
    } );
    // numWidgets == 0 here

Similarly, for the question: "What will happen in the following case:"

    int flag = 0;
    mypool.run( [] { flag = 1; } );
    cout << flag << endl;

The answer is that the code is illegal, you have to say whether you capture flag by value or by reference, which can be as simple as replacing [] with [=] or [&]. And if you capture by value, you get a read-only copy so you couldn’t assign to it.
on 2008-03-30 at 6:35 am (no name)

I agree with Aleksey. "They don’t help much here" is an overstatement. The comparison with boost::bind mostly serves to highlight the "Widget const&" and "bool" redundant boilerplate instead of casting the lambdas in the favorable light they deserve. You could’ve chosen a better example.
on 2008-03-30 at 6:34 am (no name)

How are local variables captured? I read the section on capture lists in the spec, but couldn’t quite get my head around it. In particular, what happens if a local variable is modified within the closure:

    int numWidgets = 0;
    for_each( v.begin(), v.end(), []( Widget& w )
    {
      ++numWidgets;
…
… use or modify w …
…
    } );

Will numWidgets be 0 after this, or will it contain the number of widgets in v?

What will happen in the following case:

    int flag = 0;
    mypool.run( [] { flag = 1; } );
    cout << flag << endl;

What will be output?
    …always "0"
    …sometimes "0", sometimes "1", depending on thread scheduling
    …sometimes "0", followed by a crash as the closure references a local variable that has gone out of scope
on 2008-03-30 at 6:16 am Herb Sutter

And of course I forgot to mention you lose the natural function call syntax with binders.

BTW, Alexsey, you do have a point. You and I and a number of others do know how to use binders correctly, and especially in simple examples they’re more convenient than writing separate functors which is our main alternative today. But would you agree that neither of us is probably representative of typical C++ programmers? You’re one of the two authors of the book on template metaprogramming after all :-) (and that book is worth a plug — follow the link). It’s sometimes hard for experts to remember that what has become familiar to us isn’t generally readable/adoptable/usable for everyone, and I definitely include myself here as having that blind spot. Again, YMMV, though.
on 2008-03-30 at 5:10 am Herb Sutter

Alexsey: Binders are a matter of taste. In the GreaterThan example’s case I think it’s clear that using the C++98 binders is unreadable even for simple examples. The Boost/C++0x simplified binders are a big improvement, and you could argue that using them in simple examples like this one is still readable, but personally I find them hard to read and maintain even when (a) you write the code that uses them correctly the first time, and (b) in this simple only-need-one-binder-expression case.

If you don’t write them exactly correctly first time, even in this simple case you’re likely to get lengthy compiler errors that often aren’t much help in figuring out what you did wrong. For example, if instead of this:

find_if( w.begin(), w.end(), boost::bind( &Widget::Weight, _1 ) > 100 );

you forget the & and write this:

find_if( w.begin(), w.end(), boost::bind( Widget::Weight, _1 ) > 100 );

on several popular compilers you get screenfuls of errors, though in this case the first error tends to be helpful (one compiler, ahem, even says "use ‘&Widget::Weight’ to create a pointer to member — very helpful in this case, but notice how it’s now in the user’s face that he’s using pointers to members).

And if you got the & right but instead tried to write the expression the other "natural" way:

find_if( w.begin(), w.end(), 100 < boost::bind( &Widget::Weight, _1 ) );

the compiler errors on the compilers I tried are completely unhelpful. Library-based binders/lambdas are nearly always brittle, and particularly hard to use the minute member functions crop up, even in this simple example. So I don’t agree this was an ill-conceived example — I used a member function on purpose. :-)

YMMV, of course. I appreciate your comment and feedback, your point and my reply will definitely be helpful context for other readers.
on 2008-03-29 at 10:31 pm Tom Kirby-Green

I’m super happy to see lambdas and enclosures make it into C++. I’ve been using both heavily in C# since it’s 2.0 days and with 3.0 making them even more elegant it’s increasingly been a chore try to achieve the same elegance in C++ when ever I’ve had to drop back to using the old war horse for some legacy battle. My only real ‘concern’ (if that’s the right word) is how soon we might see this in Visual C++ (my shop just isn’t going to use gcc at this point). Visual Studio 2010? 2012? I’m just hoping it’ll still have the impact and audience it deserves by the time Microsoft are able deliver it. I’d hate it to be ‘the feature that never was’.
on 2008-03-29 at 10:06 pm Aleksey Gurtovoy

The find example in today C++ with Boost 1.34 is actually less verbose and at least as readable and maintainable as the C++0x version you provided:

find_if( w.begin(), w.end(), boost::bind( &Widget::Weight, _1 ) > 100 );

Don’t get me wrong, I’m looking forward to having lambda functions in the core language, and there are numerous, perhaps just slightly more complex examples where a built-in lambda facility would be a definite winner over any library-based implementation. Using an ill-conceived example and overstating your case at the same time ("They don’t really help much here"? In this case apparently they can help more than the proposed core functionality) hardly contributes to the quality of the post, though.
on 2008-03-29 at 9:35 pm (no name)

boost::lambda is already pretty damn cool. The fact that now it (the technique, not the library) will be natively supported is just so damn cool.
on 2008-03-29 at 8:41 pm Scott McMurray

I’m dissapointed that polymorphic lambdas didn’t make it. Having to type out map< string, pair< bool, vector<foo> > >::const_reference for the parameter type in the lambda means that I’m never going to use it when I could use iterators and auto instead ( for (auto i = m.begin(), e = m.end(); i != e; ++i) ) and type less and be more amenable to change later.

Especially since the paper suggests that the semantics are specified by conversion to a struct functor, in which templating operator() works great, especially now that there’s auto (…) -> decltype(…) { … }.

Maybe I just need to learn more about concepts, though, since the justification I read in one of the old lambda papers didn’t make much sense to me.
on 2008-04-05 at 7:37 pm Scott McMurray

Regarding noname #19:
Also, I wonder when the lookup for that decltype is done. IIRC, the semantics are defined by conversion to a struct, where w might be out of scope. (But the committe’s probably already thought of that.)

And yup, the example shows that even in a simple case, the for-loop-with-auto is shorter:
for (auto i = w.begin(); i != w.end(); ++i) {
if ( i->weight() > 100) {…}
}
Especially since the find_if will require even more code to check whether the return value is == w.end().

P.S. Yay, new blog host!
on 2008-04-06 at 7:19 am orcmid

Way off topic meta-matter: You didn’t post a note about moving the blog here, and comments are off there, so I am commenting on your most recent post on something I noticed.

I opened up a new feed, with its own folder, and I will then move posts from the folder from the old feed before I kill that one in my news reader.

In the new feed, synchronization with NewsGator on-line recovered a pile of stuff. Oddly, it is all dated 1969-12-31 which has to be some sort of clue. There is something weird about the RSS feed from the restored material.

Probably not a biggy (unless new posts show up that way too), but a definite curiosity.
on 2008-04-06 at 7:53 am Herb Sutter

Thanks Orcmid. Yeah, I did notice some date weirdness too, and it’s the more curious because the imported posts and comments all seem to have correct/reasonable dates when viewed in the usual blog interface. Hopefully it affects only the imported material, but I’ll keep an eye on it to see whether it affects new posts.
on 2008-04-09 at 1:13 am parallel roads » C++ is getting functional

[...] his blog, Herb Sutter recently posted notes from the latest C++ standards meeting: For me, easily the biggest news of the meeting was that we [...]
on 2008-04-09 at 7:58 am Lance Diduck

I agree — Lambdas are very cool. Soon someone will write a proof assisant in C++!!!
The more and more large and parallel-enabled applications I write, the more I find myself writing in a functional style. (the example mypool.run() given notwithstanding — it assumes that cout, etc also act in a reasonable way when called by many threads…)

Re Alex’s post: I would enjoy hearing a plain English explaination N2554 as well. May a few examples or something, demonstrating the problems all this addtional complexity solves.
on 2008-04-23 at 2:40 pm asdf

I think lambdas should also allow for auxiliary variables like:

[size_t line = 0] (int b) { display(b); if (++line % 10 == 0) flip_colors(); }
on 2008-06-20 at 3:37 pm Talking Lambdas With Bill Gates on BBC « Sutter’s Mill

[...] shot that includes yours truly at a whiteboard presenting an overview-plus-drilldown on C++0x lambda functions. It was a good review; Bill’s a sharp guy with a broad and deep background and incisive [...]
on 2008-06-30 at 3:54 am Next big programming language feature after closures | Mind Tree

[...] Closures are the current hot feature for programming languages. The inclusion of closures in Java appears to be around the corner, and the C++ committee has recently voted on closures in the upcoming C++ 0x standard. [...]
on 2008-07-08 at 10:12 am C++ joining Java to get closures » netjeff.com

[...] Herb Sutter writes on C++ closures, which have been accepted as part of the next C++0x standard. This means C++ is joining Java in supporting closures. My first introduction to closures was Javascript, and I think it will be great for the other mainstream languages to support them. Of course, it will probably be a few years until closures are widely used, as was the case for C++ and Java templates/generics. [...]
on 2008-07-09 at 5:54 pm DanielWaechter.com » Blog Archive » The Long Road Back to C++

[...] it out every time (though there are some limitations1)? Even better, we’re going to have real lambdas and closures! But who knows how long it’ll take to implement – we’re still waiting for C99. [...]
on 2008-10-03 at 10:40 am Tanveer Badar

One more question, why even go for this syntactic sugar when boost lambda does already provide this functionality?
on 2009-02-26 at 1:38 pm Free Training For Laid-Off Developers « Sutter’s Mill

[...] review. It’s quite interesting just how often using C++0x language and library features (from lambdas to shared_ptrs and concurrency) really help solve the old problems in even more elegant and robust [...]
on 2010-07-07 at 12:39 am Joe

“One more question, why even go for this syntactic sugar when boost lambda does already provide this functionality?”

Boost lambda is a great achivement, but it’s still limited by the lack of language support. Anything other than the simplest lambdas are unmaintainably complex.

Of course boost::lambda has the great advantage that it already works on many compilers, unlike the C++0x lambda which only gcc supports.