Yesterday, I gave a talk at Google on common crypto flaws, with a focus on web-based applications. Like my previous talk, the goal was to convince developers that the most prevalent crypto libraries are too low-level, so they should reexamine their design if it requires a custom crypto protocol.
Designing a crypto protocol is extremely costly due to the careful review that is required and the huge potential damage if a flaw is found. Instead of incurring that cost, it’s often better to keep state on the server or use well-reviewed libraries that provide SSL or GPG interfaces. There are no flaws in the crypto you avoid implementing.
I hope you enjoy the video below. Slides are also posted here.
Thanks for the great talk. Looks like they got the title wrong on YouTube, though.
Comment by Steve Weis — August 6, 2009 @ 11:39 pm |
Great talk, with complete coverage of many important points. However, I’ve never been a fan of the “please don’t ever do crypto yourself.” This seems to be a popular talking point nowadays, and I think it’s a bit over the top. With some specific education, and straight-forward guidelines, programmers should be able to do crypto. As long as they understand that programming crypto is like arming a bomb, that is. Treat it carefully, and with respect and forethought.
Comment by s — August 7, 2009 @ 10:12 am |
I respectfully disagree. If you look at cryptographers, they don’t do crypto like programmers seem to. No respectable cryptographer designs a protocol without planning right from the beginning for massive peer review. On the other hand, programming is often a solo sport. Pair programming does not compare to standing in front of hundreds of people at CRYPTO and having your multi-year thesis torn apart publicly.
All I’m saying is that embarking on crypto design without the assurance of this kind of review is foolhardy. If you can avoid this danger by using crypto protocols that already exist, why wouldn’t you? A cryptographer would! Just because you can program well in assembly doesn’t mean you use it casually for writing large applications. Why is it easier to avoid assembly than implementing crypto?
Comment by Nate Lawson — August 7, 2009 @ 10:40 pm |
A lot of what you talked about raises a big question with me: what *is* a proper security paradigm for web applications?
Lets take user-password security for a web application. The common way to store passwords in the database is: sha1( password + salt ), or the precise way you said isn’t so good. So, if the defacto standard for 80% of open-source software isn’t secure enough, then what should it be replaced with?
In an effort to be more secure I’ve been attempting to rewrite the technique I use to store passwords.
My new technique is to generate an HMAC from the password, the key being a very large random number (per user), and then to regenerate the HMAC from the result and the large number four more times. This was inspired by someone else’s suggestions … on a blog. But I feel like this isn’t any more secure than the “standard”.
Comment by Robert K — August 8, 2009 @ 8:53 am |
Your approach seems better, but why are both you and the 80% not using what’s already out there? For example, the openwall project provides a PHP version of the OpenBSD password crypt or you can use the one built into your OS if you have a Unix C library.
Why not spend the time you would have spent designing, implementing, and testing your own crypt with surveying what’s out there and doing a security review of its implementation?
Comment by Nate Lawson — August 9, 2009 @ 9:08 am |
I think you’re assuming that we know what to do, cryptographically. I’m a hobbyist when it comes to ciphers and understand the rudimentary physical/paper ciphers like the airplane cipher; I’ve gotten as far as writing an implementation of XTea in Pascal. But, that doesn’t mean I actually know what I’m doing … that just means I could take the code I found and translate it to another language.
How do I evaluate cryptographic implementations to determine their relative security?
Understand what I see when I look at that code you linked me to: That code is three years old, writes a native-PHP implementation of the existing base64_encode, uses MD5 which I know is easily broken, DES fallback should be never due to its weakness (I do know about the weak keys, etc), and it uses the highly esoteric crypt() function built into PHP. The problem with crypt() being a lack in clear use, and the use of several CIPHERS to “hash” something?
All this makes me want to scream, wave my arms in panic, and run away.
From what I understand, ciphers aren’t destructive enough for password storage when the implementation of crypt is freely readable, as would any passwords the system would use. They would simply need to glance at my database table, see the signature “$2$” and then figure out what my salt is–then it all falls to pieces. Again, by my understanding this makes the serious hacker more dangerous than using hashes like SHA-512 or Whirlpool with a salt, or using their HMAC algorithms.
In short: how can I possibly tell when there is a good cryptographic implementation, so that I can look for one; and how can I trust what appears both unverified and poorly written? Besides, I’m just a freelancer and contractor, and cannot afford to conduct a security review; I could look at the logic of an implementation, but again, I don’t know what to search for.
Comment by TheWickedFlea — August 10, 2009 @ 6:44 am
If you want to see a horror show, browse the questions tagged “cryptography” or “encryption” on StackOverflow.
Comment by kme — August 9, 2009 @ 7:41 pm |
I actually used a set of those comments in my slides for why “design by blog” is a bad idea.
Comment by Nate Lawson — August 10, 2009 @ 8:17 am |
If you think Stack Overflow is bad, check out the user-contributed notes for PHP-mcrypt:
http://us.php.net/manual/en/function.mcrypt-encrypt.php
Comment by Daniel Franke — August 17, 2009 @ 3:05 pm
Have you read the book “The Cult of the Amateur”? Though it does not talk specifically about crypto, it does talk about some of the issues with Web 2.0, some of which probably led to things like the rainbow table fiasco.
Comment by Yuhong Bao — November 1, 2009 @ 3:47 pm
WickedFlea,
Perhaps you just had trouble finding this section of the website I linked to. Perhaps this will help clear up some of your questions.
So you as the person responsible for your site’s security would use the Blowfish mode, avoiding fallbacks to the other modes that are only there for backwards compatibility. Not too hard to use the default, is it? Apparently, this is built into PHP 5.3.0+ or the Suhosin patch (again, just reading the page).
And no, an attacker can see your salts and a well-designed password hash scheme should not fall apart. A salt is not secret. It only needs to make a given hash unique.
Comment by Nate Lawson — August 10, 2009 @ 8:26 am |
For the record, Robert K and I are synonymous; WordPress didn’t recognize me when I commented the first time.
Okay then, to distill what I wanted from the responses you gave…
Ideal security involves: using trusted, tested, established ciphers and hashes; presuming that your salt is known, and ought to be unique per-record; taking enough time to generate the hash so as to make brute force difficult (or infeasible). In web development, the greatest prevention of spam and automated attacks is uniqueness, which means that customization of the algorithms, rounds, and modifiers used would prevent such things as precalculated hash attacks from blindly compromising more than a handful of sites.
Comment by TheWickedFlea — August 10, 2009 @ 10:50 am |
The salt has to be unique per record, AND per instance of a record. So if a user changes their password, you need a new, unique salt again.
I disagree that uniqueness is needed for the password hash algorithm, however. You have to assume that if the attacker has your hashes, they have the code that generates them. After all, they’re on the same server and require the same privileges to access.
Comment by Nate Lawson — August 10, 2009 @ 3:26 pm
Nate, I never said uniqueness in the algorithm is required: I said that it would prevent blind reuse of the same precalculated dictionary. IE, it would take more time to compromise even when in possession of the algorithm. I did forget to mention salt uniqueness per-instance, but I assumed nothing.
And, firstly, the database and application that I deploy to my server run on separate machines. While yes, if my FTP account is compromised (I do change my passwords) the attacker will have my database credentials and the exact algorithm; the database is just as often compromised by itself. Should the database alone be compromised, a custom algorithm (with no indicative header like $2a$) would be very difficult to diagnose and brute-force–since they would have no clue to any procedural variations within the algorithm.
EG: http://gist.github.com/165342
I won’t claim it reviewed, secure, or even a good idea, but I put it together to try to comprehend the concepts of crypt() and phpass. Oh, and it works on virtually all installations of PHP 5.2.9, which most shared servers out there use.
Comment by The Wicked Flea — August 11, 2009 @ 6:23 am
Great talk!
Comment by vierito5 — August 11, 2009 @ 7:47 am |
Hi Nate,
Does the attack described at slide 18th http://www.slideshare.net/rootlabs/crypto-strikes-back-google-2009-1826045 really work? If yes, please enlighten me.
I’ve done a survey on length-extension attack of MD construction. It seems that this attack works only on naive MD constructions. SHA1 and MD5 is MD strengthening.
Regards,
-Thai
Comment by Thai — August 12, 2009 @ 5:16 am |
You mentioned later that you figured this out. For the benefit of other readers, I’ll add some more notes here.
No attack is being described on the hash function itself. Its just an inherent property of the Merkle-Damgard construction that given a hash for some data, anyone can extend the hash to cover additional data. To do this, they simply use the previous output hash value as the initial chaining values, then hash the data they want to append as normal.
There are additional, more difficult attacks on constructions like H(key || data || key). Luckily, it’s easiest and best to just use HMAC.
Comment by Nate Lawson — August 14, 2009 @ 8:42 am |
A great talk. I have seen it just now.
Personally, I don’t agree with your your slide #9. You assume that the keys to encrypt/decrypt the data via Javascript are on the server that provide the data itself. In a Host-proof Hosting system this isn’t true. Only the user has their keys and all the data are sent to the server after that the data has been encrypted in the browser.
If you adopt standard algorythm (AES, SHA256, RSA…), avoid XSS problems, etc. you can create a good system that fully protect privacy of the users.
Comment by Francesco Sullo — January 7, 2010 @ 6:10 pm |
How can it be a “Host-proof” system when the user is generating the keys and then encrypting the data using Javascript code downloaded from the Host?
Javascript has no unique root of trust. Here is a “secret keeper” application that encrypts the user’s data using a password they enter into the browser. All communications are over SSL to prevent MITM.
Design #1:
A. Client gets HTML form
B. Client posts data and password
C. Server runs code that encrypts data with password and stores it
Attack: compromise server and trojan the code
Design #2:
A. Client gets HTML form and Javascript code
B. Client runs JS code that encrypts data with password
C. Client posts encrypted data to the server
Attack: compromise server and trojan the JS code
Both models have the same root of trust: the server. Everything else is just handwaving.
Comment by Nate Lawson — January 7, 2010 @ 10:48 pm |
Hi Nate, I agree that “Both models have the same root of trust: the server”.
I didn’t point the attention on trust, but on the possibility of create a crypto system that properly uses Javascript. I am a founder of Passpack, an online password manager. We launched our tecnology in december 2006 and now we have about 70,000 users. During theese years, we haven’t have security issues. This says something.
I know that we must trust a web system, always. But a well made Host-proof Hosting system offers same advantages for the user, because safe her privacy.
To grab your data, I need to create a back door in the code. Is is a clear violation of the contract with the user and you can notice the modification.
Instead, if the encryption is delegated to the server, during the job I can make a copy of you data and you can not know it.
Comment by Francesco Sullo — January 8, 2010 @ 2:49 am
I appreciate that you are getting down to the actual details, but it doesn’t change the fact that JS crypto is being oversold to the public. Just because you haven’t had known security issues in 3 years doesn’t mean the system provides the type of security that users think they’re getting.
The one thing JS crypto gets you that server-side crypto does not is client-side auditability. That’s it. However, this benefit comes with tremendous caveats that I think far outweigh it.
* The user has to be a cryptographer and can detect subtle cryptographic flaws. Note: this ignores all the malicious crypto research by Yung et al and assumes this is actually doable in reasonable time.
* The user always does “View Source” each time she connects to be sure the code is identical. If it was changed for bugfixes or whatever, she has to re-review the entire thing.
* The user always loads everything over SSL to be sure she’s getting untampered code and there is no MITM.
* The JS code does not have cache problems where users keep running old code after you’ve fixed a flaw and tried to upgrade them.
* You periodically audit all your servers to be sure they aren’t serving up malicious or outdated JS code. Note: this audit cannot be done via a client because an attacker can return different code to you than your users, based on src IP.
Since no one does all this and the disadvantages are so significant, JS crypto is worse than server-side crypto.
Comment by Nate Lawson — January 11, 2010 @ 11:12 am
You listed some interesting points. User has some difficult to verify the code and an hacker can discover holes in the code. You’re right. But if the crypto is server side the user can verify nothing at all. So the assumption is that the server side programmers mantain a good system and we trust them.
If you assume that the provider of the JS crypto has the same care of the server-side crypto, something changes. The difference:
* a well programmed host-proof hosting app garantees the user privacy
* a good programmed server-side crypto app cannot garantee it
Mantain a HPH system is more critic that mantain a server-side crypto system, it requires more attention, I know it. But I think that the choice depends from your objective.
Comment by Francesco Sullo — January 11, 2010 @ 1:10 pm
I’d rather have the crypto code on a locked-down server I control than pushing it to some unknown browser environment. Since no users actually are capable or even motivated to audit the JS crypto they receive anew each session, there is no advantage. Given the disadvantages listed before, I’m still unconvinced.
Comment by Nate Lawson — January 11, 2010 @ 1:24 pm
WickedFlea,
I looked at your design. There’s a salt and a hashed password, separated by a $. There are no identifiers so if a user wanted to increase the number of rounds later, upgrade the algorithm, etc., there’s no way to tell which configuration goes with a particular hash. So basically, the result is equivalent to the old Unix crypt but with more rounds by default, ignoring the features added in md5-crypt (algorithm identifier) and bcrypt (number of rounds identifier). I don’t think you’ve gained any security in exchange for the usability loss.
Comment by Nate Lawson — August 11, 2009 @ 9:42 am |