Dangers of Cargo-culting

Posted by Jamis Wednesday, August 01, 2007 16:10:00 GMT

“Cargo culting”, when used in a computer-programming context, refers to the practice of using techniques (or even entire blocks of code) seen elsewhere without wholly understanding how they work. (The term “cargo cult”, if you are unfamiliar with it, has its own fascinating etymology, which is covered nicely at wikipedia.) Cargo culting is a dangerous phenomenon, watering down the state of the art and encouraging cookie-cutter code shoved blindly into black boxes.

Consider the following snippet of code, taken from a project that was submitted to us some time ago. (Alas, I cannot find the original submitter—I apologize for that!)

1
2
3
def account_code?
  !! @account_code.nil?
end

To me, this looks cargo-culted, since it is seems that the programmer did not understand what the ”!!” idiom was all about. They probably saw it used somewhere and “cargo culted” it, using it without knowledge, assuming that it was, for some reason, “necessary”.

Now, the way ”!!” works is this: take the value behind the ”!!”, negate it, and negate it again. It’s just double-negation: !(!(@account_code.nil?)). The ultimate effect is to take some value, and convert it into an honest-to-goodness “true” or “false”. (In my ever-so-humble opinion, the ”!!” idiom is an abomination: it’s far too clever for its own good. First of all, you rarely ever need a real boolean value, and for those times you do, it is better to be explicit in the conversion, by using a ternary operator or full-blown if statement, for instance.)

In other words, the double-negation of nil? results in absolutely no difference from the use of nil? by itself, since nil? will return a true/false value. This, in turn, means the effect in the original is actually not what was intended for the account_code? predicate. It should have returned “true” if the account code existed (was “non-nil”), not “false”. Thus, the method should have actually been written thus:

1
2
3
def account_code?
  ! @account_code.nil?
end

In this case, cargo-culting resulted in the code being buggy. This is not an uncommon outcome of using techniques or code without understanding their purpose. If you ever find yourself copying something into your own code, with a justfying “I-don’t-know-what-it-does, but-it-appears-to-work”, stop immediately. Do some research. Figure it out. Learn what it means.

Further, note that unless you actually need a true boolean value from that, you can shorten the implementation of the account_code? predicate even further:

1
2
3
def account_code?
  @account_code
end

This works because Ruby treats nil and false as false, and everything else as true.

If there is one thing that Koz and I want you, our readers, to come away from this site with, it is an understanding of why you should do things one way and not another. Ultimately, it makes the difference between being a mediocre programmer, and becoming a great programmer.

Comments

Leave a response

  1. LukeAugust 01, 2007 @ 05:52 PM

    The only exception I see is when you are cutting and pasting code that will act as a ‘library’ within your application. In that case, it’s the same thing (to me, at least) as installing a gem and not bothering to check out the source to find out how the gem does what it does.

    Other than that, I’ve tried to get this point across to people many times, and they usually just don’t get it (and so will remain mediocre programmers).

  2. JohnAugust 01, 2007 @ 05:53 PM

    How strange. I know I’ve seen that exact same code before (although the method/variable names were potentially different). I wonder how widespread the usage of it is? Or did we just happen to look at the same code?

  3. JGeigerAugust 01, 2007 @ 06:12 PM

    That code (or very similar) is in acts_as_authenticated.

  4. fozAugust 01, 2007 @ 06:14 PM

    When I first started coding with Ruby (coming from Perl), I was so confused that zero and “0” are both true.

    I’ve seen ’!!’ used sometimes in plugins. ActiveRecordStore::Session uses it too. I didn’t realize what it was useful for till you posted this :)

  5. JamisAugust 01, 2007 @ 07:05 PM

    Yeah, the !! idiom shows up in the Rails source code a few times, but it was never placed there by me. :) My stance is, and shall always remain, that such cleverness obscures the meaning of the code and only makes things more difficult for people who need to maintain it.

  6. LanceAugust 01, 2007 @ 07:39 PM

    lol! Very nice. Could you even further shorten it by using attr_reader :account_code? or if it is an active record model, it magically finds the boolean value.

  7. SandofskyAugust 01, 2007 @ 08:02 PM

    There may be a reason to avoid the ‘everything is true’ philosophy. An html person who understands basic Ruby might display @user.account? in a field, expecting “true” or “false”, and instead display the user’s account code. It’s an edge case, but I cast sensitive data to true/false.

  8. Keith LancasterAugust 01, 2007 @ 08:06 PM

    Several years ago I dealt with a (supposedly) senior programmer who had taken this to the next level. Not only did she not understand the code that she was using, she had left all the variable names as they were in the original in order to avoid breaking the code.

  9. PhilAugust 01, 2007 @ 08:07 PM

    IMHO the only reason we see the !! idiom is that Ruby lacks an Object#to_bool method. If you hate !! just define to_bool. But really I think ”!! @account_code” is more expressive than just ”@account_code” because it indicates you’re interested in a predicate. If the method name ends in ? then it’s perhaps redundant, but in other cases just because ”@account_code” happens to work the same way doesn’t mean it *communicates *the same thing to a reader.

    As long as everyone reading the code base understands that !! is shorthand for “I want a yes/no, not the original value” then there shouldn’t be any confusion or accusations of cleverness.

  10. JohnAugust 01, 2007 @ 08:13 PM

    “Yeah, the !! idiom shows up in the Rails source code a few times, but it was never placed there by me. :)”

    Except what I saw was also combined with .nil?, making the !! useless (and reversing the expected result), like in the example. Unless that too is found in the Rails source?

  11. KozAugust 01, 2007 @ 10:54 PM

    @Phil: The predicate method certainly makes it clearer to the user that you’re expecting a ‘true/false thing’ back. However requiring it to return a boolean is a smell, just return the account_code and everything will work.

    Attempting to hide your users from something as fundamental as what’s true/false in ruby is just a recipe for disaster.

  12. DmitryAugust 02, 2007 @ 12:51 PM

    Phil has a point. Just imagine someone storing result of obj.account_code? call in a variable, and by that creating additional reference to @account_code object, when all he wanted was just to store true or false. There are lots of cases where that would be bad. Just imagine what result of the {:has_account_code => obj.account_code?}.to_yaml will be.

    Ruby makes it clear, that methods ending with ? should return true or false, and I see no reason to return anything else (even nil will brake the assumptions, I think), except for laziness maybe. I’m not sure that !! is the way to go, though.

  13. SamAugust 02, 2007 @ 01:17 PM

    I like !!. It’s not hard to read and it wont prematurely swallow an exception. @thing ? true : false … Why should we opt for the long code. @thing.not_null_or_false? @thing.to_bool, to_bool?

  14. MichaelAugust 02, 2007 @ 02:07 PM

    I disagree that requiring it to return a boolean is a smell. Returning non boolean objects is likely to break equivalence. Since if a.method? == b.method? will most likely fail.

  15. JamisAugust 02, 2007 @ 07:51 PM

    @Michael: tell me honestly, how often have you needed to compare booleans like that? I know it does happen, but does it happen often enough to warrant the extra effort needed to “booleanize” everything?

    Note that Koz and I are not saying that you never need an explicit boolean. I’ve needed it myself, on rare occassions (usually when dealing with Rails before filters). But that’s the point. It is rare. If you are doing it, note the smell. Think about whether you really need it. If you don’t, don’t do it.

    And the point with !! is not that it can’t be learned, or that it is confusing even after you know it. The point is that it is noisy, and ugly. :) Yes, those are aesthetic claims, but I posit that your code will look cleaner, and be more accessible to more people, if you avoid clever tricks like that.

    A skilled programmer doesn’t care so much about knowing how to use clever tricks. Instead, a skilled programmer cares more about knowing when to use clever tricks, and (perhaps even more importantly) when not to use clever tricks, and why.

  16. MichaelAugust 02, 2007 @ 08:16 PM

    I just tend to expect object.method? to return a boolean. In the same way I would expect a variable named options_hash to conform to Hash and not BigDecimal. I don’t really need to have options_hash conform to a Hash, but it is pretty confusing when it doesn’t. Why leave a potential snakepit when it is trivial to remove it?

    I do totally agree that something like !! @account_code.nil? is ridiculous.

  17. JamisAugust 02, 2007 @ 08:50 PM

    The Hash-to-Bigdecimal comparison doesn’t really fly, Michael, since Ruby treats any non-nil and non-false object as true. If your only requirement is that the value work as expected in an if-statement or similar, I still disagree that you need an explicit boolean. If you have stricter requirements, then need for a real boolean may change.

  18. MichaelAugust 02, 2007 @ 10:59 PM

    Clearly we disagree on what the best behavior is, but I would like to say that I really enjoy the site. It’s great that you guys take time out of your schedules to post a ton of really good content, even replying to the comments. I really appreciate it. Thanks.

  19. Pat MaddoxAugust 04, 2007 @ 08:43 PM
    <quote>If you ever find yourself copying something into your own code, with a justfying “I-don’t-know-what-it-does, but-it-appears-to-work”, stop immediately. Do some research. Figure it out. Learn what it means.</quote>

    I would say you should also write one or more tests that describe the behavior. This will leave you with some documentation as to how it works, avoid bugs in the future, and allow you to preserve the desired behavior should you wish to change the implementation after reading an article such as this :)

  20. Tim LucasAugust 05, 2007 @ 04:48 AM

    I also must admit that poor writing and a lack of explanation in my bangbang you nil is dead article could have encouraged some cargo culting.

    You forgot to include the alternative, also correct but arguably less-concise equivalent which uses double negation:

    
    def account_code?
      !!@account_code
    end
    

    I think its less surprising to return explicit booleans, but I do agree with you that it feels clunky, similar to checking an object is_a? class, and probably too clever for its own good.

    I do notice though that Matz’s Ruby seems to always return booleans on methods ending with a question mark. Do the other Ruby implementations do the same?

  21. williamAugust 06, 2007 @ 03:16 PM

    1) In my opinion there is no smell involved in doing what it takes to return a boolean value when one is expected. Thats the entire point of encapsulating the state of an object and exposing only what is appropriate. I don’t care how often you end up NEEDING that boolean or not. Just returning an instance variable when you are obviously writing a function that is meant to return a boolean defeats the purpose of OOP.

    2) If you don’t like [!!], most of your arguments against it will also apply to [test ? true : false] as a replacement for [if else] which is all over the place. That said I think both ARE a bit obscure and take more time for the brain to parse. But I don’t see that either are such a huge deal.

    3) Yup I agree. Its important to know what the code does and why when you paste stuff into your program.

  22. JamisAugust 07, 2007 @ 05:14 AM

    Tim, thanks for your comment. However, I didn’t actually “forget” to include the version you mentioned…I intentionally omitted it, because I am strenuously opposed to the bang-bang. :)

    For all commenters in this thread: in the end, Koz and I encourage you to do what makes you happy. What we’re saying most of the time is just this: sometimes, what you think will make you happy, won’t. Trying to be too clever is one of those things. Use your head, but don’t over-think things. Always focus on making something work sooner, rather than later. At all costs, avoid over-engineered solutions. Be pragmatic about your craft.

  23. YanSeptember 11, 2007 @ 07:00 PM

    How about this version:

    def account_code?; true if @account_code; end

    it’s short, clear, reads like english, and returns the boolean as expected

    one reason not to return the whole value, but use a true boolean, is in some cases you may break encapsulation and encourage bad practices (for example if this wasn’t an account code but an account object, and your class only wanted to expose whether the account exists, not give full access to the caller)