Do Not Use Assertions

Don't use assertions. BairsLaw applies - what will you do if the assertion fails? (Fix the bug, perhaps?) If you don't like the parameters you're given, throw an IllegalArgumentException. If you don't get a correct result, your UnitTest will tell you. There is no room left for assertions.

I confess - I am the Infidel who wrote this. However I am not trying to be rude, I want to find out why other people like assertions so much. -- JohnFarrell

Step zero: Refactor comments into assertions

Step one: Refactor assertions out of the code into unit tests

Step three: Escalate the remaining assertions into program exceptions

In Java, an assertion should throw an IllegalArgumentException, or some other exception derived from RuntimeException - those exceptions are used to signal programming errors, rather than errors caused by the environment or issues out of the programmers' control.

Other languages should dump core, or do whatever the language's idiom is for reporting a programming error.

-- NatPryce

In Java, a failed "assert" statement throws java.lang.AssertionError?, which extends java.lang.Error. It doesn't extend java.lang.RuntimeException, but it is an unchecked exception, giving it the properties Nat suggested. -- JeffGrigg

Italics are CurtisBartley, from UseAssertions.

When an assertion fails, the program halts, and a programmer figures out what's wrong, and fixes it. Then the assertion doesn't fail any more. It's important to understand that the code is not supposed to handle the error - a person is.

I agree. But I never saw any code which handled InvalidArgumentException? either.

{ Assertions aren't just used for cases of InvalidArgument? , that's Preconditions. What about ordinary mid-execution, sanity checks...and Postconditions . Are they not infix forms of UnitTest? -- AlanFrancis }

It is also important to remember that standard operating procedure for assertions is to turn them off before doing a build that will be sent to real customers; In this case I'd argue that BairsLaw does apply, but the conclusion that assertions should not be used by the programmer does not follow.

In Java, it's much harder to turn them off before you ship, especially since inter-class optimization is rarely used. But I am surprised that you turn them off - that only means that your program will fail silently and the customer won't even be able to report what messages it spat out. If my program crashes at a customer site, I would rather be embarrassed and receive an error report than be embarrassed and still have no information about the problem.

I should also mention not everybody uses unit tests, and they are not required to ship good software. In a development process that relies heavily on human testing, assertions might well increase the rigor of the testing.

I agree that if you're not going to use UnitTests, you will need assertions. Assertions are a (runtime) tax to ensure that your code is still working. UnitTests are a service level guarantee.

UnitTests are not a "guarantee": they just improve the probability that bugs will be found earlier. Assertions also improve the probabilities.

Not an absolute guarantee, just a guarantee of service at a particular level. As you extend the UnitTest, the level becomes higher.

Supposing a human is doing the work, and they realize they've made a mistake, but they don't know how to recover. Wouldn't you rather they tell you about it so that you can tell them how to avoid the mistake or recover next time? Wouldn't you rather have your employees make a few checks that what they're doing is OK?

My take on assertions (at least of the C variety) is that they were an early attempt to create a test framework. In C, assertions were turned on during DEBUG (testing) mode, and turned off in production mode. Two different executables were built, but from the some source code. ProgrammerTests (nee UnitTests) differ in that, in addition to two executables, we have two sets of source code. This provides much more capability in driving the execution of the code to cause our assertions to be fired. Assertions were a good first step, but the various XUnit frameworks provide an improvement.

You can always write your own assertion which displays the usual failure message and then asks the user if they want to continue at their own risk.

...I am surprised that you turn them off - that only means that your program will fail silently and the customer won't even be able to report what messages it spat out.

I use an assertion macro that throws exceptions. In debug mode, it puts up a dialog that gives me an opportunity to pop into the debugger first, but an exception is thrown either way. -- PhilGoodwin

I see assertions, various pre/post conditions, UnitTests, AcceptanceTests etc. etc. as a set of overlapping tools in a given programmer's toolbox. A given programmer expends effort E to ensure the correctness of code C using a mix of tools appropriate to the job and influenced by personal history.

The level of effort and selection of tools is best done by the programmer on the job. I almost never use assertions and for the most part my peers seem to think I'm a very 'good' programmer. I, however, fault myself for not trying to consistently apply assertions to at least one project so that I can judge by hands on experience if the enthusiasm for assertions is relevant to what I do. So I will be writing a few classes RealSoonNow with a consistent set of assertions to see how things pan out. -- MikeKrajnak

I like assertions because once one is in the habit of using them the effort is very small, and so any bugs they do catch are pure profit.

UnitTests are important and valuable, but you have to explicitly write and run the tests, while assertions can often be written automatically just by typing in your assumptions about program state as you write the code: they'll fail if the assumptions are wrong. Certainly they work better in some situations than others: complex data-structure manipulations where you know what the situation ought to be but may have made a logic mistake are a good place to use them. WritingSolidCode and CodeComplete talk about this. -- MartinPool

YouArentGonnaNeedIt applies. Don't throw IllegalArgumentException if nobody needs this. Just get minimal - assert - until someone does. -- PhlIp

In SuperDistribution, BradCox makes argument in support of assertions that I find appealing: they provide an unambiguous way to specify in an interface (as opposed to implementation) the expected behavior of a method. This provides a mechanism for implementation interchangeability. Meanwhile, an assertion failure means that the logic of a design (as opposed to its implementation) is faulty. This is valuable information, particularly when components are coming from multiple sources. I'm under the impression (but I don't know first-hand) that this is also the way assertions are used in EiffelLanguage. The fact that a particular runtime environment is too stupid to automagically remove any runtime overhead imposed by an assertion is a shortcoming of the environment, not the concept (in my opinion). -- TomStambaugh

Your impression of the EiffelLanguage is correct, but there assertions are also inheritance-aware. E.g. if a superclass A specifies an assertion for method m and subclass B overwrites m, then a call to B.m would cause the assertion in A.m to be checked. Details in [ObjectOrientedSoftwareConstruction]. I think that feature makes assertions much more useful in OO languages. -- HaskoHeinecke

I often use assertions to check "can never happen" conditions - if the software is written correctly, the assertion can't possibly fail. That way, if some silly programmer (like me, sometimes ;-) fails to do things right, they'll be notified right away, and the debugger (typically) will take them to that spot in the code and show them where things went wrong. However, once the program passes programmer integration testing, the assertions practically never fail, so they can be removed from the production release. (On the other hand, I've been known to release code, with assertions in place, to production.)

Perhaps if I had the time and authority to throughly refactor the code and put a good set of UnitTests in place, there might not be any need for assertions. But the legacy code I have to work with isn't like that, and it will take several man-years of effort to fix the problem. (...if I don't change projects first ;-) -- JeffGrigg

It's fun to think of UnitTests as perfect and ever reliable and will replace many other good practices, but they aren't and can't. People make them. The quality of tests will vary a great deal between developers, between managers, between projects, and between delivery pressures.

Is it being forgotten that UnitTests consist of two parts: invocation of code (with inputs) plus checks regarding the results? UnitTests don't compete with assertions; the two complement each other. Specifically, even if a UnitTest doesn't check any results, it still exercises code and any contained assertions. Similarly, assertions are useless unless they are executed; running UnitTests executes assertions under more conditions. -- DanielBarclay?

Remember in C++ being frustrated by assertions in libraries other than your own and not knowing if they were serious or not? If you had access to the author, they generally said, "Oh, don't worry about that, that happens every time there is a full moon." So you just started ignoring them all until nothing seemed to work any more.

In Java, an Exception means something is wrong and needs fixing. If the developers stick with this throughout the development, they will be sure to make robust handling on RuntimeExceptions?. It's great to download a java library, include it in your class path, and bam! it works. If it started blurting assertions, I'd start cursing and wondering if C++ wasn't so bad after all.

I use IllegalArgumentException, IllegalStateExceptions?, or extensions to RuntimeException in areas where outside forces cannot be predicted, especially related to User inputs (although normal exceptions for "expected" negative paths cases) and let Exceptions like NullPointer and ArrayOutOfBoundsExceptions? take care of the rest. True, the error messages on the last two are useless, but the full Stack is most of what you need. -- DouglasHubler?

IllegalArgumentException and IllegalStateException exceptions are not meant to be used where outside forces cannot be predicted - you should use checked exceptions for that. IllegalArgumentException and IllegalStateException are used to signal that a method's preconditions or an object's invariant has been violated. These are both programming errors rather than exceptional conditions occurring in the environment.

Java uses unchecked exceptions to indicate programming errors (it is unfortunate that they are derived from a class named RuntimeException, but that is by the by), and checked exceptions to indicate that an unpredictable, environmental error has occurred. This is reasonable because it forces the programmer to deal with exceptions that may happen at any time, but still reports exceptions that should never happen.

But all of this has only a tangential relationship to assertions. It would be interesting, however, for someone to measure how much actual overhead an assertion adds to the runtime environment. I wonder what, if any, advantage java compilers & runtime environments actually take from all that marvelous type-checking the language forces on the developer. -- TomStambaugh

We have had code that is very time sensitive and has undergone much optimization. Even in this code the assertions were left in because their impact was minimal and had caught some interesting errors. -- Oogoody

From a strictly C/C++ perspective, assertions (i.e. the ASSERT macro), can be viewed as a simple way to create test stubs for higher level classes. If a function receives a bad parameter, then someone higher up in the chain has done something he shouldn't. -- WayneMack

In C/C++ there is no runtime overhead for the ASSERT macro -- if the program is compiled with NDEBUG defined. (by inserting 

  #define NDEBUG
at the beginning of the source file, or in some other way -- such as adding the "-DNDEBUG" switch to the compiler options).

You can also #if-test for NDEBUG to conditionally include longer tests that can't fit inside ASSERT(). ASSERT() has nothing to do with the DEBUG switch. See assert(3). -- Ralph Corderoy.

I disagree with this completely. An assertion should not have anything to do with checking usage (e.g. types or values of arguments to a function). An assertion should be a condition which, if false, implies a bug in the code. e.g. if there is some sort of invariant through a loop, you assert the invariant. If the assertion fires, you've found a bug; thus, by definition, the program cannot continue.

Checking for validity of arguments is entirely different; you should signal some sort of runtime error, so that this particular call path can be aborted/cleaned up, and the next one attempted. So Assertions are just executable documentation of what must be true of the current state of the application at any time. -- AlainPicard

I don't understand your disagreement. Sometimes people write code in layers. The "outer" layers test all the data passed to the program, and gracefully handle any problems (perhaps by exiting with an appropriate error code). The "inner" layers assume the data has already been validated, right? You're absolutely right that an ASSERT() at the beginning of a function, checking the "types or values of arguments to a function", is useless for finding bugs in *that* function. But what is so wrong with using that ASSERT() to help detect bugs in the "outer" layers? -- DavidCary

--- No-one has mentioned the documentation aspect of assertions. I like to think of code as communicating with other developers, not just the machine. -- HenryAndrew

Exactly. What is better: 

// We assume that ptr is non-NULL
or 
assert(ptr);
-- StephanHouben

or 

        assert(ptr != NULL);

Can't we view an assertion as an alternative way of packaging a UnitTest? There are arguments why it might be preferable to have the UnitTest within the main source code as an assertion and there are arguments why it might be better to have the UnitTest in a separately compiled file. The key issue is to have the test, not how you choose to package it. -- WayneMack

I would view it as a part of the test, but not the whole test. Two aspects of any test are the stimulus and the expected result. Assertions are a good way of defining the expected results, but do nothing to define the stimulus (a precondition failure is a response, not an input). Another view is that a (precondition) assertion says "I haven't tested this, so don't even think of using it".

This view of assertions as the interface does suggest that assertions should not be mixed with implementation. In a language like C++, this can be achieved by creating an "interface" as a set of (non-virtual, inline?) methods that use the TemplateMethod pattern, with the implementation being pure-virtuals. E.g. 

  template <class T>
  class Stack
  {
    INVARIANT( true )


    public:
      inline void pop ( )
      {
        PRE( ! isEmpty() )
        unsigned old_numberOfItems = numberOfItems();
        _pop();
        POST( numberOfItems() == old_numberOfItems - 1 )
      }


      inline T top ( ) const
      {
        PRE( ! isEmpty() );
        unsigned old_numberOfItems = numberOfItems();
        T result = _top();
        POST( numberOfItems() == old_numberOfItems );
        return result
      }


      // etc...


    private:
      virtual void _pop () = 0;
      virtual void _top () = 0;
      // etc...
  };
The contracts could, of course, be improved. Better macro support (or even a meta-compiler) could simplify the boilerplate. The important thing is that this class serves the same role as an interface in Java, but provides stronger contracts that a Java interface is capable of. Furthermore, this paradigm can't be used (to its full potential) in Java because implementing several such interfaces would require MultipleInheritance.

I see the arguments around "assert" as being similar to those around "goto". Assert is a low level primitive. Just as structured gotos are named "while", "for", etc.; so structured assertions are named "precondition", "invariant", etc. So, by all means, DoNotUseAssertions; but do use their structured equivalents! -- DaveWhipp

I worked for about 6 years as a development lead at Microsoft, and I always mandated that ASSERT be used liberally, but correctly. I provided a very specific set of criteria for when to use ASSERT, and when not to use it:

Again, these are the guidelines that I established, on the projects that I led. The team overall found these guidelines to be very clear, and very useful. The liberal (but consistent/correct!) use of ASSERTs allowed us to find many bugs, far faster than we otherwise would have found them. But the criteria we used also allowed us NOT to generate spurious assertion failures - if module X calls into module Y, module Y is responsible for validating parameters, and generating an ASSERTion failure is not a valid response.

You may debate the "purity" of this approach, but I don't care for purity. Since we had very clear criteria for when to use ASSERTs and when not to use them, and since we also had a very clear agreement on what happened when an assertion failed (immediate severity 1 priority 1 bug, indicating fatal internal defect, must fix), we found ASSERTs to be a very effective tool.

However, I do agree with previous authors in this article, that if a language has a native construct for expressing invariant conditions, then by all means, use it. The ASSERT macro, in C/C++, is just to make up for the lack of any language construct. (Arguably, though, ASSERTs can be more powerful than pre/post-conditions, because they may be used at any point inside a method, not just at the beginning and end.)

We also found it very beneficial to write AssertIntegrity? methods for nearly all of our classes, as well as for common datastructures that did not use classes. The AssertIntegrity? method would use the ASSERT macro to validate the state of all data members (for example, validating the range of enums), and would call the AssertIntegrity? method of any data members that provided one. And for classes that use critical sections for thread synchronization, the Lock method would always acquire the critical section, then immediately call AssertIntegrity?. Symmetrically, the Unlock method would call AssertIntegrity?, then release the critical section. This allowed us to quickly find which method was responsible for deviating from the state consistency rules that we defined, even when under heavy load from many threads.

The best of it all was that it cost us relatively little, in developer/maintenance time, to do this, once we agreed on the rules of when to use ASSERT and when not. And standardizing on an AssertIntegrity? method meant that we distributed the cost of checking data structures, since many of our existing datastructures already had AssertIntegrity? methods, so it was just a matter of wiring them all together.

-- ArlieDavis

Where I work, we are explicitly told "do not use assert()": at least, not the version that comes with the standard library (C/C++). The reason: when we run regressions on the farm (Windows or Linux/Wine), we don't want any pesky dialog boxes ("assertion hit: debug/abort/ignore") to popup: no human watches them, so jobs tend to hang for hours, waiting for someone to decide that it's dead. In theory, it's possible to use a runtime that doesn't do this; in practice, it happens far too frequently to be worth the hassle. For our own code, we have our own macros. But some libraries have asserts that, if they fail, will hang the job.

UnitTests define the concrete examples (Pseudo-Code, of course): 

  void testEven() {
    even = isEven (2);
    if (!even) fail();
  }
Assertions define the logic correctness condition for all possible concrete examples: 

  boolean isEven (int value) {
    if (value == 0) throw new IllegalArgumentException("Precondition violated");
    int halfValue = value / 2;
    assert (halfValue < value);
    return (value / 2 * 2 == value);
  }
Assertions can not drive concrete tests, but assertions can detect (more? concrete) errors earlier than UnitTests and they can locate errors more exactly!

-- ThorstenVanEllen?

What kind of event warrants an instantaneous abort? They could exist in theory, but they would be signs of severe problems that have no place in a production system. Here is a reasonable assertion situation:

"This runaway process is unable to return and continued operation could hold a relief valve closed that will cause the system to explode and management has decided not to sacrifice another system to theoretical ideas of elegant program design because they do not know who to believe any more. This 'assert' statement is the only recourse, because the code is surrounded by junk written by assertion happy nutters. We can't fix that because we no longer have the source. It was lost because of an assertion triggered by the revision control system, midway through a check-in and those same nutters built it to annihilate the entire code base under those conditions by crashing on an assertion because they felt that fall-back to a known backup was better. They knew they had solid backups due to their crafty use of assertions to ensure the stability of the backup system. Failure was impossible. When we went to backup, we discovered that backup had been silently failing, on an assertion, with a message to a non-existent console in the service that they built and tested as an ordinary program because they wrongly assumed that the way they were currently using the code was the way we would use it forever. Charlie, who was supposed to be monitoring this, was killed by an angry mob last month when they discovered it was *his* assert statement that caused the nuclear launch suppression fail-safe system to abort, and the island of Hawaii was vaporized in a catastrophic disaster rivaling this code. Before his demise, Charlie assured us that this message would never appear and we believed him because for practical purposes the assert statement in the backup code never did. We are aborting due to fears that we may never spaghetti back from here to a known state. The only State we really knew that well was Hawaii anyway, and that, of course is now gone. We do not think that you will ever see this message, but if you are reading it, you know we were wrong about that too. Never fear, though, an Email has been automatically sent to … errrm … charlie@flybynightsoftwareco.arg. Hey -- there's a co-incidence, his name is Charlie too. What are the odds?"

It is doubtful you would see such a thing in practice. Due to a little known line in the library code to the effect of:

assert(msglen < arbitrarylimitsetbythesamereasoningthatgotusintothismess)

Here is a more realistic (and unreasonable) assertion:

"The limited experience and imagination of the programmer caused him to think that this situation was impossible. His idea of dealing with the impossible is to do something impossibly stupid and crash the system so that it will happen over, and over, forever unless we have not fired him, and he himself happens to be staring at the screen with the current source, a compiler and a debugger handy and the code continues to write to a visible console. He somehow thinks that this will all happen (and yet not happen, he is so dialectical!) in production. What the heck, it is impossible, is it not? His limited intelligence caused him to reason that an impossible situation (that by definition cannot arise, like, is that not what impossible means?), might arise anyway and as IMPOSSIBLE as it all seems, he thought he would be there to see it. He did not have his 'thinking pants' on the day he wrote this code. You should never see this message. It should never be reported if it happens and the programmer will continue to suffer the delusion that absence of proof is equivalent to proof of absence."

BobTrower -- sort of tongue in cheek, but in case it is not obvious, I discourage the use of assertions.

Just today, at 5:15PM Pacific time, I was reading a PPT slideshow on software correctness. It was written by TonyHoare. Upon completing the slideset, I clicked in the File menu, with the intent on clicking Exit afterwards. After File highlighted, the mouse froze, the application failed to present the menu, the keyboard went dead, and my USB devices inoperative. In other words, something that OpenOffice did, perhaps indirectly through Xorg, rippled up into the kernel, and crashed it hard enough to deadlock it.

What an incredible, potent, cogent, and topical coincidence, eh?

THIS is a situation that just screams for assertions. It's a situation that cannot be detected through unit tests. It only happened this one time (the first Linux kernel crash I've experienced in 1.5 years), and without a reliable means of reproducing the bug, no basis exists what-so-ever for writing a unit test.

Samuel A. Falvo II -- drop-dead serious, who is a staunch advocate of combined test-driven and formal proof development.

[Though, given the symptoms -- especially the frozen mouse pointer -- it's quite possibly a hardware problem like a flaky power supply, power transient (brown-out), or an overheating CPU (check the fan(s))...]

Actually, given the symptoms, the absolute opposite comes to mind. Were this a hardware issue, failures would occur far more frequently than once every 1.5 years, as my experience running three data centers can attest.

The fans are free spinning (they're brand new even), and the heat sink compound is still fluid. The power supply is less than eight months old, so the filter capacitors are like new (it'll take five years for them to start to dry out anyway). And while there have been brown-outs here before, none have actually caused my computer to crash in any duration less than 1/10th of a second before (indeed, precisely because the P/S' capacitors do an excellent job at filtering the switch-mode ripple from the supply rails). Speaking of which, no lights or motors which were running at the time had any noticeable "glitch" characteristic of a brown out, so that rules out ANY transient longer than 1/10th of a second.

In other words, it's physically impossible for a glitch strong enough to disrupt the computer's normal operation in a period less than 1/10th of a second to occur without damaging the computer in some secondary manner (indeed, such a glitch is what finally killed my AND Athlon slot-A computer, which is why I have experience with these kinds of glitches too!). The computer remains undamaged -- it is in full operational order as I type this. Sound card works, video card throws triangles on the screen like before. Harddrives don't click. The mouse is responsive and the keyboard obeys my every command. Finally, memtest86+ reports my 2GB of RAM is in perfect condition.

This is unequivocally a transient software issue. The absolute closest you can come to saying that this is a hardware issue is that a single bit of DRAM, which happened to be holding Xorg or kernel code or data, was flipped by a cosmic ray. An assertion would have eliminated the faulty data aspect of the equation completely.

I have seen the above 'screams for assertions' a few times now, without commenting. However, I feel moved to say that it unequivocally *does not* call for assertions in code. Whatever the assertion could have done, a logging call could have done as well. Heck, if you are hell-bent on seeing this appear on some imaginary console somewhere, you could have a special case of 'CRAZY_FATAL_EMERGENCY_MESSAGE_LOGGING' and spit your message to the console first, just as if you were about to ungraciously crash to the operating system. You could then continue on in code with a controlled shutdown if the error is that bad. Asserts allow no recovery. They assume they know what is going on with the caller and they have no right to make such an assumption. The call could conceivably be a unit test of that very condition to ensure that a 'meta-caller' properly alerts a human operator and then restarts the shut down process.

There are, in essence, two courses to follow:

1) Use an assert, which forces the process to exit, fails to release resources like file handles, fails to flush buffers (which would include logging information) and possibly fails to alert anyone that the process even failed at all, let alone why it failed. Why you assume your program's facilities are completely ruined and yet you can still depend upon logging a console message is beyond me.

2) Do the same test, but log the test failure, release those resources you can, and return the appropriate status information to the caller so that the caller can either repair the condition or pass *its* appropriate error back up the call stack.

In case one (which I strongly recommend against), it is quite possible that nobody ever discovers the line of code that failed. Arguably, the majority of code has no console to which it can properly send information with any probability that it will ever be seen. Whatever logging was in place may not entirely make it to its destination, because the assert abruptly terminates the runtime.

In case two (which I strongly recommend), either the problem was so catastrophic that you are left with the identical situation as above (in which case it is doubtful the assert would have caught it anyway), *or* you have a complete log, all resources released as appropriate, return to the caller with useful information and either a terminate with a graceful termination or actually recover and continue.

The only thing that distinguishes case two from case one is that case one *guarantees* the worst-case scenario.

There might be, as I mention above, a highly peculiar situation where abrupt termination is desirable. However, it likely has no place in production code. It would be patching a grievous fault elsewhere and the best way to deal with a grievous fault at point A that affects point B is to correct the fault at A, not catch it and blow up at point B.

Can anyone involved in this debate give me a single reasonable example of an assert that is likely the best way to deal with the unexpected in code that plausibly belongs in an actual production system? I have seen TONS of asserts over the years and I have never seen a single one that I thought was necessary.

Please try to limit your examples to decent code that adheres to the hoary structured programming principle of one point of entry and one point of exit (attributable to Dijkstra**). Two wrongs do not make a right. If the code is spaghetti code, it has other problems besides the assert statement.

--- BobTrower

See ShipWithAssertionsOn for the opposite point of view.

CategoryAssertions

EditText of this page (last edited May 12, 2009) or FindPage with title or text search