This special Guru of the Week series focuses on contracts. Postconditions are directly related to assertions (see GotW #97)… but how, exactly? And since we can already write postconditions using assertions, why would having language support benefit us more for writing postconditions more than for writing (ordinary) assertions?
JG Question
1. What is a postcondition, and how is it related to an assertion? Explain your answer using the following example, which uses a variation of a proposed post-C++20 syntax for postconditions. [1]
// A postcondition along the lines proposed in [1]
string combine_and_decorate( const string& x, const string& y )
[[post( _return_.size() > x.size() + y.size() )]]
{
if (x.empty()) {
return "[missing] " + y + optional_suffix();
} else {
return x + ' ' + y + something_computed_from(x);
}
}
Guru Questions
2. Rewrite the example in Question 1 to show how to approximate the same effect using assertions in today’s C++. Are there any drawbacks to your solution compared to having language support for postconditions?
3. Should a postcondition be expected to be true if the function throws an exception back to the caller? Justify your answer with example(s).
4. Should a postcondition be able to refer to both the initial (on entry) and final (on exit) value of a parameter, if those could be different? If so, give an example.
Notes
[1] G. Dos Reis, J. D. Garcia, J. Lakos, A. Meredith, N. Myers, and B. Stroustrup. “P0542: Support for contract based programming in C++” (WG21 paper, June 2018). Subsequent EWG discussion favored changing “expects” to “pre” and “ensures” to “post,” and to keep it as legal compilable (if unenforced) C++20 for this article I also modified the syntax from : to ( ), and to name the return value _return_ for postconditions. That’s not a statement of preference, it’s just so the examples can compile today to make them easier to check.
This special Guru of the Week series focuses on contracts. We covered basic assertions in GotW #97… but not all asserted conditions are created equal.
Given some assertion facility that can be used like this:
MyAssert( condition ); // expresses that ‘condition’ must be true
1. Give one example each of an asserted condition whose run-time evaluation is:
a) super cheap
Without resorting to constexpr expressions, it’s hard to find one cheaper than the one we saw in GotW #97 Example 3, which we can simplify down to this:
// Example 1(a): A dirt cheap assertion (from GotW #97 Example 3)
int min = /* some computation */;
int max = /* some other computation */;
MyAssert( min <= max );
This is always going to be dirt cheap. Not only is the integer comparison operation cheap, but min and max are already being accessed by this function and so they’re already “hot” in registers or cache.
b) arbitrarily expensive
“A condition that’s expensive? That sounds pretty easy,” you might say, and you’re right!
One commonly cited example is is_sorted. Just to emphasize how expensive it can be both in absolute terms and also relative to the program’s normal execution, let’s put it inside a well-known function… this is a precondition, but we’ll write it as an assertion in the function body for now which doesn’t affect this question: [1]
// Example 1(b): An arbitrarily expensive assertion
template <typename Iter, typename T>
bool binary_search( Iter begin, Iter end, const T& value ) {
MyAssert( is_sorted(begin, end) );
// ...
}
Checking that all the container’s elements are in ascending order requires visiting them all, and that O(N) linear complexity is arbitrarily expensive when the container’s size N can be arbitrarily large.
The icing on the cake: In this example, just evaluating the assertion requires doing more work than the entire function it appears in, which is only O(log N) complexity!
On a humorous note, O(N) remains O(N) no matter how hard we try to make it efficient:
MyAssert( std::is_sorted( std::execution::par, begin(s), end(s) ) );
// still O(N) arbitrarily expensive, but good try!
2. What does the answer to Question 1 imply for assertion checking? Explain.
We want a way to enable checking for some assertions but not others in the same code, because we may not always be able to afford expensive checks. That’s true whether we’re enabling checking at test time (e.g., on the developer’s machine) or at run time (e.g., in production).
Some conditions are so expensive that we may never check them without a good reason, even in testing. Example 1(b)’s is_sorted is a great example: You probably won’t ever enable it in production, and likely not by default in testing, except by explicitly turning it on during a bug hunt after enabling checking for cheaper assertions wasn’t enough or pointed at this data structure for deeper investigation. [2]
Other conditions are so cheap we’ll probably always check them absent a good reason not to, even in production. Example 1(a)’s min <= max is at this other end of the scale: It’s so dirt cheap to check that it’s unlikely we’ll ever have a performance reason to disable it. [2]
So it makes perfect sense that if Examples 1(a) and 1(b) appear in the same source file, the developer will want to enable checking for 1(b)’s assertion only by some kind of manual override to explicitly request it, and enable checking for 1(a)’s assertion all the time.
3. Give an example of an asserted condition that is in general impossible to evaluate, and so cannot be checked.
One common example is is_reachable for pointers or other iterators, to say that if we increment an iterator enough times we can make it equal to (refer to the same object as) a second iterator:
// Example 3: Very reasonable, but generally not checkable
auto first_iterator = /*...*/;
auto last_iterator = /*...*/;
MyAssert( is_reachable(first_iterator, last_iterator) );
std::find( first_iterator, last_iterator, value );
In general, there’s no way to write is_reachable. You could try to increment first_iterator repeatedly until it becomes equal to last_iterator, but when the latter is not reachable that might never happen and even just trying would often be undefined behavior.
You might be tempted to test is_reachable using std::distance:
… but that would be horribly wrong. Can you see why?
Take a moment to think about it before continuing…
…
…
… okay. The answer is that std::distance itself requires that last_iterator is reachable from first_iterator, otherwise it’s undefined behavior. So this maybe-tempting-looking alternative actually assumes what we want to prove, and so it’s not useful for this purpose. (GotW #100 will consider in detail the general question of preconditions of contract subexpressions, which covers examples like this one.)
Can these kinds of conditions still be useful?
Yes. In practice, these kinds of conditions spell out “this is a formal comment.” Static analyzers and other tools may be able to test such a condition in a subset of cases; for example, at some call sites an analyzer may be able to infer statically that two iterators point into different containers and so one isn’t reachable from the other. Alternatively, the tools might support special pseudofunction names that they recognize when you use them in assertion expressions to give information to the tool. So the conditions can still be useful, even if they can’t generally be checked the normal way, by just evaluating them and inspecting the result.
4. How do these questions help answer:
a) what “levels” of asserted conditions we should be able to express?
There’s a wide spectrum of “expensiveness” of assertion conditions, ranging from cheap to arbitrarily high to even impossible. In the post-C++20 contracts proposal at [3], this is partly captured by the proposed basic levels of default, audit, and axiom, roughly intended to represent “cheap,” “expensive,” and “impossible” respectively.
Because we need to check these with different frequencies (or not at all), we need a way to enable and disable checking for subsets of assertions independently, even when they’re in the same piece of code.
GUIDELINE: Distinguish between (at least) “cheap,” “expensive,” and “impossible” to evaluate assertions. If you develop your own assertion system for in-house use, support enabling/disabling at least these kinds of assertions independently. [1] I say “at least” because what’s “expensive” is subjective and will vary from program to program, from team to team… and even within a program from your code to your third-party library’s code that you’re calling. Having just two preset “cheap” and “expensive” levels is minimal, but useful.
b) why the assertions we can “practically” write are a subset of all the ones we might “ideally” like to write?
It can be useful to distinguish between all ideal assertions, meaning everything that has to be true at various points in the program for it to run correctly, and the practical assertions, meaning the subset of those that can be reasonably expressed as a C++ expression and checked. In GotW #97 question 3, part of the solution says that “if an assertion fails” then…
… there is a program bug, possibly in the assertion itself. The first place to look for the bug is in this same function, because if prior contracts were well tested then likely this function created the first unexpected state.
If we could write all ideal assertions, and exercise all control flow and data flow during testing, then a failed assertion would definitely mean a bug in the same function where it was written. Because we realistically can’t write and exercise them all, though, we could be observing a secondary effect from a bug that happened earlier. Still, this function is the first place to start looking for the bug.
Notes
[1] Upcoming GotWs will cover preconditions and violation handling. For handlers, we’ll cover additional distinctions such as categories of violations (e.g., to distinguish safety-related checks vs. other checks).
[2] As always, any checks left on in production would often install a different handler, such as a log-and-continue handler rather than a terminating handler; see GotW #97 Question 4, and note [1].
Thank you to the following for their feedback on this material: Joshua Berne, Guy Davidson, J. Daniel Garcia, Gábor Horváth, Maciej J., Andrzej Krzemieński.
This special Guru of the Week series focuses on contracts. We covered basic assertions in GotW #97… but not all asserted conditions are created equal.
JG Questions
Given some assertion syntax:
SomeAssert( condition ); // expresses that ‘condition’ must be true
1. Give one example each of an asserted condition whose run-time evaluation is:
a) super cheap
b) arbitrarily expensive
Guru Questions
2. What does the answer to Question 1 imply for assertion checking? Explain.
3. Give an example of an asserted condition that is in general impossible to evaluate, and so cannot be checked. Can these kinds of conditions still be useful?
4. How do these questions help answer:
a) what “levels” of asserted conditions we should be able to express?
b) why the assertions we can “practically” write are a subset of all the ones we might “ideally” like to write?
Assertions have been a foundational tool for writing understandable computer code since we could write computer code… far older than C’s assert() macro, they go back to at least John von Neumann and Herman Goldstine (1947) and Alan Turing (1949). [1,2] How well do we understand them… exactly?
1. What is an assertion, and what is it used for?
An assertion documents the expected state of specific program variables at the point where the assertion is written, in a testable way so that we can find program bugs — logic errors that have led to corrupted program state. An assertion is always about finding bugs, because something the programmer thought would always be true was found to actually be false (oops).
For example, this line states that the program does not expect min to exceed max, and if it does the code has a bug somewhere:
// Example 1: A sample assertion
assert( min <= max );
If in this example min did exceed max, that would mean we have found a bug and we need to go fix this code.
GUIDELINE: Assert liberally. [3] The more sure you are that an assertion can’t be false, the more valuable it is if it is ever found to be false. And in addition to finding bugs today, assertions verify that what you believe is “obviously true” and wrote correctly today actually stays true as the code is maintained in the future.
GUIDELINE: Asserted conditions should never have side effects on normal execution. Assertions are only about finding bugs, not doing program work. And asserted conditions only evaluated if they’re enabled, so any side effects won’t happen when they’re not enabled; they might sometimes perform local side effects, such as to do logging or allocate memory, but the program should never rely on them happening or not happening. For example, adding an assertion to your code should never make a logically “pure” function into an impure function. (Note that “no side effects on normal execution” is always automatically true for violation handlers even when an assertion system such as proposed in [4] allows arbitrary custom violation handlers to be installed, because those are executed only if we discover that we’re in a corrupted state and so are already outside of normal execution. [5] For conditions, it’s up to us to make sure it’s true.)
2. C++20 supports two main assertion facilities… For each one, briefly summarize how it works, when it is evaluated, and whether it is possible for the programmer to specify a message to be displayed if the assertion fails.
assert
The C-style assert is a macro that is evaluated at execution time (hopefully enabled at least during your unit testing! see question 4) if NDEBUG is not set. The condition we pass it must be a compilable boolean expression, something that can be evaluated and converted to bool.
It doesn’t directly support a separate message string, but implementations will print the failed condition itself, and so a common technique is to embed the information in the condition itself in a way that doesn’t affect the result. For example, a common idiom is to append &&"message" (a fancy way of saying &&true):
// Example 2(a): A sample assert() with a message
assert( min <= max
&& "BUG: argh, miscalculated min and max again" );
static_assert
The C++11 static_assert is evaluated at compile time, and so the condition has to be a “boolean constant expression” that only refers to compile-time known values. For example:
// Example 2(b): A sample static_assert() with a message
static_assert( sizeof(int) >= 4,
"UNSUPPORTED PLATFORM: int must be at least 4 bytes" );
It has always supported a message string, and C++17 made the message optional.
Bonus: [[assert: ?
Looking forward, a proposed post-C++20 syntax for assertions would support it as a language feature, which has a number advantages including that it’s not a macro. [4] This version would be evaluated at execution time if checking is enabled. Currently that proposal does not have an explicit provision for a message and so programmers would use the && "message" idiom to add a message. For example:
// Example 2(c): An assertion along the lines proposed in [4]
[[assert( min <= max
&& "BUG: argh, miscalculated min and max again" )]] ;
3. If an assertion fails, what does that indicate, and who is responsible for fixing the failure?
A failed assertion means that we checked and found the tested variables to be in an unexpected state, which means at least that part of the program’s state is corrupt. Because the program should never have been able to reach that state, two things are true:
There is a program bug, possibly in the assertion itself. The first place to look for the bug is in this same function, because if prior contracts were well tested then likely this function created the first unexpected state. [5]
The program cannot recover programmatically by reporting a run-time error to the calling code (via an exception, error code, or similar), because by definition the program is in a state it was not designed to handle, so the calling code isn’t ready for that state. It’s time to terminate and restart the program. (There are advanced techniques that involve dumping and restarting an isolated portion of possibly tainted state, but that’s a system-level recovery strategy for an impossible-to-handle fault, not a handling strategy for run-time error.) Instead, the bug should be reported to the human developer who can fix the bug.
GUIDELINE: Don’t use assertions to report run-time errors. Run-time errors should be reported using exceptions, error codes, or similar. For example, don’t use an assertion to check that a remote host is available, or that the user types valid input. Yes, std::logic_error was originally created to report bugs (logic errors) using an exception, but this is now widely understood to be a mistake; don’t follow that pattern.
Referring to this example:
// Example 3
void f() {
int min = /* some computation */;
int max = /* some other computation */;
// still yawn more yawn computation
assert( min <= max ); // A
// ...
}
In this code, if the assertion at line A is false, that means what the function actually did before the assertion doesn’t match what the assertion condition expected, so there is a bug somewhere in this function — either before or within the assertion.
This demonstrates why assertions are primarily about eliminating bugs, which is why we test…
4. Are assertions primarily about checking at compile time, at test time, or at run time? Explain.
Assertions are primarily about finding bugs at test time. But assertions can also be useful at other times because of some well-known adages: “the sooner the better,” “better late than never,” and “never [will] I ever.”
Bonus points for pointing out that there is also a fourth time in the development cycle I didn’t list in the question, when assertions can be profitably checked. Here they are:
Of course this can be even more nuanced. For example, you might make different decisions about enabling assertions if your “run time” is an end user’s machine, or a server farm, or a honeypot. Also, checking isn’t free and so you may enable run-time checking for severe classes of bugs but not others, such as that an operating system component may require checking in production for all out-of-bounds violations and other potential security bugs, but not non-security classes of bugs.
First, “the sooner the better”: It’s always legal and useful to find bugs as early as possible. If we can find a bug even before actually executing a compiled test, then that’s wonderful. This is a form of shift-left. We love shift-left. There are two of these times in the graphic:
(Earliest, best) Edit time: By using a static analysis tool that is aware of assert and can detect some violations statically, you can get some diagnostics as you’re writing your code, even before you try to compile it! Note that to recognize the assert macro, you want to run the static analyzer in debug mode; analyzers that run after macro substitution won’t see an assert condition when the code is set to make release builds since the macro will expand to nothing. Also, usually this kind of diagnostic uses heuristics and works on a best-effort basis that catches some mistakes while not diagnosing others that look similar. But it does shift some diagnostics pretty much all the way left to the point where you’re actually writing the code, which is great when it works… and you still always have the next three assertion checking times available as a safety net.
(Early) Compile time: If a bug that depends only on compile-time information can be detected at compile time even before actually executing a compiled test, then that’s wonderful. This is one reason static_assert exists: so that we can express tests that are guaranteed to be performed at compile time.
Next, the primary target:
Test time: This is the main time tests are executed. It can be during developer-machine unit testing, regression testing, build-lab integration testing, or any other flavor of testing. The point is to find bugs before they escape into production, and inform the programmer so they can fix their bug.
Finally, “better late than never” (safety net) or “never [will] I ever” (intolerable severe condition):
(Late) Run time: Even after we release our code, it can be useful to have a way to enable checking at run time to at least log-and-continue (e.g., using facilities such as [6] or [7]). One motivation is to know if a bug made it through testing and out into the field and get better late-debug diagnostics; this is sometimes called shift-right but I think of it as much as being about belt-and-suspenders. Another motivation is to ensure that severe classes of bugs ensure execution will halt outright if we cannot tolerate continuing after such a fault is detected.
Importantly, in all cases the motivation is still debugging: Findings bugs early is still debugging, just better (sooner and less expensive). And finding bugs late that escaped into production is still debugging, just worse (later and more expensive). Each of these times is a successive safety net for bugs that make it past the earlier times.
Because at run time we may want to log a failed assertion, our assertion violation handler should be able to USE-A logging system, but the relationship really is USES-A. An assertion violation handling system IS-NOT-A general-purpose logging system, and so a contracts language feature shouldn’t be designed around such a goal. [5]
Finally, speaking of run time: Note that it can be useful to write an assertion, and also write code that does some handling if the assertion is false. Here’s an example from [8]:
// Example 4: Defense in depth
int DoSomething(int x) {
assert( x != 0 && "x should be nonzero" ); // finds bug, if checked
if( x == 0 ) {
return INVALID_COOKIE; // robustness fallback, if not checked
}
// do useful work
}
You might see this pattern written interleaved as follows to avoid duplicating the condition, and this is one of the major patterns that leads to writing assert(!"message"):
if( x == 0 ) {
assert( !"x should be nonzero" ); // finds bug, if checked
return INVALID_COOKIE; // robustness fallback, if not checked
}
At first this may look like it’s conflating the distinct “bug” and “error” categories we saw in Question 3’s table. But that’s not the case at all, it’s actually deliberately using both categories to implement “defense in depth”: We assert something in testing to minimize actual occurrences, but then in production still provide fallback handling for robustness in case a bug does slip through, for example if our test datasets didn’t exercise the bug but in production we hit some live data that does.
Notes
With thanks to Wikipedia for the first two references.
[2] Alan Turing. “Checking a Large Routine” (Report of a Conference on High Speed Automatic Calculating Machines, pp. 67-9, June 1949).
[3] H. Sutter and A. Alexandrescu. C++ Coding Standards (Addison-Wesley, 2004). Item 68, “Assert liberally to document internal assumptions and invariants.”
[4] G. Dos Reis, J. D. Garcia, J. Lakos, A. Meredith, N. Myers, and B. Stroustrup. “P0542: Support for contract based programming in C++” (WG21 paper, June 2018). To keep it as legal compilable (if unenforced) C++20 for this article I modified the syntax from : to ( ). That’s not a statement of preference, it’s just so the examples can compile today to make them easier to check.
[5] Upcoming GotWs will cover postconditions, preconditions, invariants, and violation handling.
Thank you to the following for their comments on drafts of this article: Joshua Berne, Gábor Horváth, Andrzej Krzemieński, Andrew Sutton. Thanks also to Reddit user “evaned” and Anton Dyachenko for additional feedback.
Assertions have been a foundational tool for writing understandable computer code since we could write computer code… far older than C’s assert() macro, they go back to at least John von Neumann and Herman Goldstine (1947) and Alan Turing (1949). [1,2] How well do we understand them… exactly?
[Update: On second thought, I’ll break the “assertions” and “postconditions” into two GotWs. This GotW has the assertion questions, slightly reordered for flow, and GotW #99 will cover postconditions.]
JG Questions
1. What is an assertion, and what is it used for?
2. C++20 supports two main assertion facilities:
assert
static_assert
For each one, briefly summarize how it works, when it is evaluated, and whether it is possible for the programmer to specify a message to be displayed if the assertion fails.
Guru Questions
3. If an assertion fails, what does that indicate, and who is responsible for fixing the failure? Refer to the following example assertion code in your answer.
void f() {
int min = /* some computation */;
int max = /* some other computation */;
// still yawn more yawn computation
assert (min <= max); // A
// ...
}
4. Are assertions primarily about checking at compile time, at test time, or at run time? Explain.
Notes
Thanks to Wikipedia for pointing out these references.
Sutter’s Mill 