This math blog has begun a series of articles on cryptography. While that blog had some good articles on mathematics, it was pretty clear the cryptography series would have some issues. Whenever I read articles about engineers picking up cryptography, warning sirens go off in my head.
When the first article on block ciphers stated incorrectly that the DES initial/final permutations were to prevent brute force attacks, I posted a correction. I also noted that if he’s going to write about how to do cryptography, it would be good to provide a disclaimer.
I like your blog overall, but this series on crypto is a bit dangerous for amateurs. The devil is in the details, and it’s very easy to make a critical mistake. So while it’s good to give people an intro, I suggest a big warning letting them know that there are some dangerous assumptions lurking below the surface, and that they should always use well-reviewed implementations and protocols (i.e. SSL) rather than rolling their own for pure ego reasons.
The next article on modes of operation for block ciphers had a completely incorrect description of counter mode. John Kelsey pointed out other issues with this article. I again requested that Mark use this as a good example how it is hard to get crypto right.
After that, he posted a correction to the description of counter mode. Unfortunately, his correction was wrong also as another reader and I pointed out.
I am not picking on Mark. He’s a smart person. However, it’s important that we pay attention to how hard it is to get cryptography right. I write about crypto here as well, often getting friends to review articles to be sure there aren’t issues with my description. I hope my articles have never given the idea that crypto is easy. When someone focuses on the primitives, like block ciphers, even a correct description can give the reader the false impression that using block ciphers is straightforward. The book Applied Cryptography, with its nice but simple descriptions, created a generation of engineers with this misconception.
The articles and comments on them are vaguely reminiscent of the “rainbow tables” fiasco. Even a well-written, detailed description of password hashing with a big disclaimer attached still resulted in dozens of engineers proposing flawed solutions of their own. When will we learn?
rdist 
Any recommendations? What recommendations would you have for engineers or developers to get it right, anything besides don’t try and roll your own crypto library?
Ryan S: If you want to write your own cryptographic functions/algorithms/protocols, I would recommend checking out your local university. There you can obtain a 4-year degree in mathematics, if you don’t already have one, followed by 2 to 5 years studying cryptography at a graduate level. At that point you should have all the tools you need to write your own cryptographic library.
Honestly, though, why *not* use OpenSSL or Crypto++ or MS CryptoAPI or something licensed from a credible company that employs cryptographers?
“We” will never learn — because a) the ‘we’ keeps changing; collectively, ignorance is a constant. And because b) the motivations of those who ‘roll their own’ isn’t just ego, it’s also (perceived) economic expediency. How many times have you seen cryptography used products where the leadership of the parent company were professional (or even amateur) cryptographers versus how many times you haven’t? My guess is that the reply to “we need to spend the time and money to get a reliable cryptographic implementation” is generally “Git R Done!” or “It’ll be close enough for our needs.”
In short, the problem isn’t those interested in cryptography who are also willing (or trying) to learn, it’s the much larger group who are interested in (using) cryptography who have no interest or motivation to learn or are too ignorant to even appreciate the risks.
That said, I agree with your post that providing learning materials that are incorrect will hurt everyone.
Ryan, I knew that was the next logical question. It will take an entire post to explain what I think is the best approach for adding crypto as an engineer. It’s not just picking the right library — it’s an entire development approach.
Mango: I think you misunderstood Ryan. He’s asking how to add security to an application while not rolling his own crypto. Recommending libraries that offer raw crypto primitives won’t help. Entire categories of higher-level functionality along with low-level visibility are needed. I’ll follow up with a post explaining what I recommend here.
Bill: absolutely, the crew keeps changing. Most Web 2.0 programmers weren’t even around for the transition from Unix DES crypt to MD5 or Blowfish (8 char password limit anyone?) Some companies do value external review, hence why I still have a job.
You raise all sorts of good points, but what’s funny to me is how the whole infosec profession runs around telling people not to write their own crypto, yet _everyone_ writes their own authN and authZ. My theory is that the crypto stuff is provably bad and so people like to focus on that because they write a really nice diagnoses, whereas we may better served if people spent just as much time on prescribing better access control. Fair?
“Applied Cryptography” does not teach anyone anything except establishing Bruce Schneier as a brand. It’s very badly written.
If you’re a beginner, and want a very good understanding of underlying mathematical principles, please pick up S.C. Coutinho’s “The Mathematics of Ciphers: Number Theory & RSA Cryptography”. It’s mind-blowingly simple to follow, but goes very deep.
Gunnar: yes, that’s my next post about how to use crypto at a higher level than standard crypto libraries. The mantra that we should not implement crypto extends from algorithms to protocols also.
period, Applied Cryptography is the old thing to rail at. Now we complain about blogs. I don’t think lack of mathematical understanding is the problem. The problem is that engineers are working at too low a level if they have to know anything about the math behind crypto.
“The best explanation why the permutations were there was to make software implementations more difficult than hardware”
How did you come to this conclusion?
Did the NSA actually print that? or is it a conspiracy theory? seriously..i can’t see how something like that would be for the purpose of a more difficult implementation in software.
Surely the reason behind permutation was due to available hardware at the time of the algorithms design?
And the only reason to use it in software was to comply with the standard..no?
witherspoon, the NSA did not publish details behind the design criteria for DES (versus Lucifer, which it was derived from). Schneier says in chapter 12 of Applied Cryptography:
“The initial permutation and the corresponding final permutation do not affect DES’s security…” and “this bit-wise permutation is difficult in software (although it is trivial in hardware)”
Interestingly enough, he thinks the permutation was intended to deal with byte-wise loading of plaintext data into the hardware implementation. I don’t see any justification for that in the structure of the IP. So it remains a mystery, although everyone agrees it doesn’t add security to the cipher itself.