What Are Assertions

What is an assertion?

An assertion is a boolean expression at a specific point in a program which will be true unless there is a bug in the program. An assertion could simply be a comment used by the programmer to think about how the code works. Or an assertion could document a constraint on the system. (See also: ExceptionsAsConstraints) However, it is often possible to actually compile the assertion to code and let it be executed in context to see if the statement it makes really does hold. When programmers talk about assertions they usually mean this kind of executed assertion.

Standard CeeCeePlusPlus provides the "assert" macro, which looks just like a function call. Here's a simple example:

 assert(0 <= index && index < length);

As long as the index is within the range specified, this statement is a no-op. If the index is out of range, however, the assertion will terminate the application and (ideally) display the failed boolean expression "0 <= index && index < length" as well as the filename and line number of the assertion in the source code.

The diagnostic information printed by a failed assertion (not to mention the potentially dramatic act of terminating the offending program) is obviously not very end-user friendly. This goes to an important point: assertions are not an error-handling mechanism. The purpose of an assertion is not to handle an error, it is to (ultimately) notify a programmer of an error so that he can fix it.

Since assertions that don't fail are no-ops, once a program has been thoroughly tested and bug-fixed, it is possible to recompile the source code without the assertions to produce a program that is both smaller and faster; for programs that make heavy use of assertions, this can result in an appreciable difference in performance. In C, this can be accomplished simply by making sure the NDEBUG macro is defined before all inclusions of the <assert.h> header file.

Handling Failed Assertions

When an assertion fails, the first order of business is to communicate that failure to the user. This can be accomplished by printing an error message to the console or to a log file, displaying a message in a dialog box, or any combination of these.

Once the diagnostic message has been displayed, there are a number of alternatives:

Terminate the program
Allow execution to continue unhindered
Throw an exception to back out of the erroneous code path
Allow the user to choose from two or more of the above options

Benefits of Assertions

The role of assertions is to identify bugs in a program. Ideally, thorough testing of a program will find the bugs in a program even without the aid of assertions. In practice, the major benefit of assertions is to make testing more effective. This is an important point. An assertion that is never executed tells you nothing. An assertion is only useful if the code path containing it is executed.

Assuming the code is being properly tested, assertions do several useful things:

Detect subtle errors that might otherwise go undetected
Detect errors sooner after they occur than they might otherwise be detected
Make a statement about the effects of the code that is guaranteed to be true

The benefits of #1 are obvious. The benefit of #2 is not in finding a bug that's going to be found anyway, but rather in making it possible to find the bug faster. This can have significant effect on the bottom line, since programmers are not cheap. The benefit of #3 is harder to quantify, but there is not doubt that seeing a function assert that one of its parameters is not null tells a programmer more than a simple comment that the parameter should not be null, and much, much more than totally undocumented code.

Assertion Pitfalls and Limitations

Bugs in Assertions

Like everything else, there is a downside to assertions. One of the biggest is that, like other kinds of code, assertions may themselves contain errors. A bug in an assertion will likely cause one of these problems:

Reporting an error where none exists
Failing to report a bug that does exist
Not being side-effect free

The first problem is perfectly tractable. The assertion is going to report to you in no uncertain terms that there is a problem. Certainly this kind of bug can send an unwary programmer a long way down a blind alley. Most programmers are pretty smart people, and after making this mistake a couple of times, they'll learn to approach assertion failures with the proper skepticism.

The second problem is certainly no more severe than not having an assertion in the first place. If the program uses assertions heavily, chances are good that the error will be caught sooner or later by another assertion. Even if it's not, testing may still turn up the error the old-fashioned way. Finally, the bug may get through testing, but then some of them always do. Since an assertion that actually detects errors is more valuable than one that doesn't, it is a good idea to simulate failure cases to test that the relevant assertions really do fail. This is good advice for all kinds of debugging code, not just assertions.

The third problem can be especially dangerous. Assertions are not supposed to affect the logic of a program. It should run the same with them (albeit more slowly) than without them. As a consequence, assertions are usually stripped out of a program before it is delivered to customers. However, side-effects do sometimes slip in, and when the assertions are stripped out the program behavior changes in a way it shouldn't; in the worst cases these will introduce severe bugs into the build released to the customer. The only reliable way to avoid these kinds of bugs is to test the release (assertions stripped out) build before release to the customer. It is probably always a good idea to test the exact bits that will be released to a customer, anyway.

Assertions Can Impact Performance

Assertions (assuming no bugs in the assertions themselves) should not affect the logic of a program, but they can and often do impact the program in other ways. In particular, they:

Take time to execute
Take up extra memory

Simple assertions such as the common check for null are relatively inexpensive, but more stringent checks, especially in assertion heavy code can measurably slow code down, sometimes severely so. Assertions also take up space, and not necessarily just for the code itself. For example, C/C++ assert macros often embed the string representation of the assertion's boolean expression as well as the filename and line number of the assertion in the source code. This can add up if assertions make up a substantial percentage of the code.

Normally assertions are turned off for builds that are released to actual customers for performance reasons. It is never stated, but an additional motivation may be that most organizations would rather have a program crash in an unexplained fashion than to display a message that an average end-user will find a little disturbing (especially if they then have to choose between terminating the application, throwing an exception, or continuing as if nothing happened). Since many bugs are not fatal, it's usually a good idea to let the user continue in case no further problems ensue.

Assertions May Block Testing

When a programmer stumbles across an assertion failure he usually has the choice to either fix the bug (often obvious) or, in extreme cases, simply disable the assertion. However, if a tester runs into an assertion failure in an official daily (or worse yet, weekly) build, the consequences could be severe. If the assertion merely catches what would otherwise be a crashing bug, there is no down side. However, an assertion failure could elevate what would otherwise be a minor bug into a bug which blocks a substantial amount of testing for a whole day. Giving the user a choice of whether to continue running is not necessarily a great help, as anyone who has ever tried to ignore an assertion inside a loop can attest.

When assertions allow execution to continue (either by default or by user choice) one often runs into exactly the opposite problem where testing should have been blocked and wasn't. In this scenario, a tester reports many different symptoms of the original bug as separate bugs. When an exception is thrown to attempt recovery, the situation can be even worse. Sometimes the throwing of the exception causes more harm than good, resulting in many bugs that would never occur if execution had simply been allowed to continue.

None of this should come as any surprise to the experienced developer (or tester). Assertions are just another development tool which sometimes works well and sometimes doesn't.

Some Things Are Difficult to Check For

Some conditions you wish to check for are conceptually simple, but very difficult to check in practice. Consider this simple example:

    void Sort(int a[], int numItems)
    {
        /* sort algorithm here */

        for (int i = 1; i < numItems; i++)
        {
            assert(a[i - 1] <= a[i]);
        }
    }

Now consider the following case:

        Sort(a, 6);

where a = {2, 0, 9, 4, 2, 1} before the call and {0, 0, 0, 0, 0, 0} after the call. The assertion will detect many possible errors, but it won't detect detect errors like this one. We can detect this error and many (but not all) similar ones if we are willing to do a little more bookkeeping:

    void Sort(int a[], int numItems)
    {
        int beforeChecksum = 0;
        for (int i = 0; i < numItems; i++)
        {
            beforeChecksum += a[i];    
        }

        /* sort algorithm here */

        for (int i = 1; i < numItems; i++)
        {
            assert(a[i - 1] <= a[i]);
        }

        int afterChecksum = 0;
        for (int i = 0; i < numItems; i++)
        {
            afterChecksum += a[i];    
        }
        assert(afterChecksum == beforeChecksum);
    }

This is a simple example of how assertions can be made stronger with the addition of some enabling code. Of course the code becomes more cluttered, especially if you add the conditional compilation comments usually used for this kind of debugging infrastructure. Also note that if we were determined, we could enhance the final check to the point that it would detect all possible errors in the output of the sort routine. This may well be more trouble than it's worth. Normal assertion usage is to find relatively cheap checks that would identify most, but not necessarily all bugs.

Other problems with assertions:

Sometimes people use assertions as a lazy form of exception handling. As a result, if the release build removes assertions, important error checking is also removed. However, if assertions are only used to detect programming errors, they can be safely removed from a thoroughly UnitTested production build.

If assertions are only active for DEBUG builds that contain debugging code and disable optimizations, they may not detect programming errors that only show up in fully optimized builds. Therefore, use three builds:

DEBUG -- a build that disables optimizations, creates debugging code, and includes assertions
RELEASE -- a build with full optimizations, no debugging code, and no assertions
INTEG -- a build with full optimizations, may or may not have debugging code, but with assertions included

The INTEG build is identical to the RELEASE build but includes assertions for programming errors or compiler bugs that may only show up in optimized code. Debugging code is useful, but not required for the INTEG build.

It seems to me this is ignoring the possibility of AssertionsAsContracts?. A powerful alternative to including assertions in the runtime is to utilize them at type-safety time, as part of a HoareTriple. The assertions could, ideally, be proven to hold before running code is produced; a failed assertion indicates a programming error elsewhere. You may also need to be able to document Assumptions. E.g. for proof of invariance over some atomic action, you assume that some property holds before you start then assert that it still holds after you've finished.

If assertions can be added to the function interfaces, involving the inputs, then the language is capable of a rather complex typing system even if no other forms of typing are included.

I believe Eiffel language provides at least some support of Assertions under this notion.

Once upon a time assertions were not executable: See AssertionsAsComments.

Discussion:

The code being tested is typically not the code being delivered. Examples include using assertions in removing the assertion code from production builds or only asserting non-optimized debug code. Therefore, you should not use assertions to replace good exception handling. Assertions not removed before shipping can cause bugs (such as compiler bugs). Leaving them in can cause assertions to appear in front of the end user. [Added this to the list of common assertion bugs. -- CB]

These days, I use a better method: test the code that will be delivered.

All assertions are replaced by UnitTests. Unit tests then include passing null parameters, checking data structures, and everything else. [UnitTesting should not replace assertions but instead make sure that all are being exercised.] -- EricUlevik

I've heard real horror stories about this, but I've never seen it happen in practice. Anyway, in the projects I know about, it's standard practice to thoroughly test release builds (with the assertions stripped out) before ever letting them anywhere near a customer. -- CurtisBartley

It's happened to me. Developers test with assertions, things work. Remove assertions, pass onto test department, severe bugs appear quickly. Test department complain about low quality deliveries. -- EricUlevik

I use an assertion macro that throws an exception instead of terminating the program. The only behavior that gets stripped out is popping up a dialog box when the assertion happens.

Testing is actually much easier this way. Assertions can detect things that can only be seen from within the code (like the state of private members). Testing becomes a joint effort between an tool that checks external behavior and code coverage and asserts which check internal behavior. -- PhilGoodwin

This sounds more like exception handling and is different from the traditional use of assertions. There is an important reason to distinguish between the two. For example, I try to break up (a) tests for programming errors that should be caught and removed before shipping and (b) tests for program errors that must be included in the shipping code. For (a), I use assertions, and plenty of them, 'without regard to the impact on runtime performance'. I'll even add alternative algorithms into my assertions! So, for some complicated optimal sort or search algorithm I may assert a second brute force method and make sure they come up with the same results. Clearly, one doesn't want this in their shipping code. For (b), I use exception handling. I still use a macro, but it is not called assert and it cannot be compiled out - at least not using the same identifiers assertions can be compiled out with.

BradCox discusses assertions at some length in SuperDistribution. There, he suggests that they can be used in the interface of a component (separately from its implementation) to validate that a putative implementation does whatever is necessary to fulfill the interface.

Assertions can be run at compile time, as opposed to run time. That's why they are expressed as macros in C. They help answer the question "Does this design solve the problem", as opposed to "Does the implementation fulfill the design". Of course, those two questions are recursively composable in most interesting situations.

I think this is why another participant here discussed replacing them with UnitTests. They play a very similar role: "If it WalksLikeaDuck, quacks like a duck, and swims like a duck, it is a duck".

-- TomStambaugh

I often write assertions like this

 assert(foo!=bar && "some reason why I think foo can't equal bar");

It documents the intent of the insertion and provides more info if it is triggered. It is then easier to decide if the assertion should be removed or if you have a real problem. -- JamesKeogh

There's an interesting report about Microsoft's use of assertions, by CarHoare: http://research.microsoft.com/%7Ethoare/Assertions_prog_prosp.htm.

Is there a page that mentions "Java assertions" (I think they started in J2SE 1.4)? Or should I just discuss them here, in WhatAreAssertions?

    assert (boolean_expression);
    assert (boolean_expression) : error_string;

... so how do I enable assertions in the EclipseIde?

Or is it better to use the idiom

    static final boolean assertionsEnabled = <true | false>;
    ...
    if( assertionsEnabled && (boolean_expression) ){ System.out.println(Assert failed: error_string); };

? A quick google found

http://www.javaworld.com/javaworld/jw-11-2001/jw-1109-assert.html http://www.ensta.fr/~diam/java/online/notes-java/20language/30statements/40assertions/20assert-usage.html http://www-106.ibm.com/developerworks/java/library/j-diag0723.html

Is this something completely different: http://mozart-dev.sourceforge.net/pi-assert.html ?

-- DavidCary