Rebooting

~ Nate Lawson

I’ve recently had some time to catch up on reading and research. I’ve decided to start writing again here about cryptography, embedded systems, and security.

Recently, I enjoyed being on the “Security, Cryptography, Whatever” podcast. I got a chance to cover some past projects and how the fields of cryptography and security have changed over the past 25 years.

  • Part 1 (1997 – 2007): Netscape, FIPS-140, Cryptography Research, satellite TV piracy wars, Blu-ray content protection wars
  • Part 2: (2008 – 2016): toll systems & smart meter hacking, designing a secure payment terminal (one of my favorite design projects), binary similarity analysis for app stores startup (SourceDNA), and the possible future of high assurance designs

I hope you enjoy hearing these stories as much as I did retelling them. More to come soon!

In Which You Get a Chance to Save Democracy

~ Nate Lawson

Let’s start with the end: you can do something to change the broken political landscape in the United States, but you have to act quickly. Here’s a link to donate directly to outsider candidates I support who aren’t getting the funds they need. They are dedicated to working for their constituents’ healthcare, jobs, and community, but their districts are low income and have been ignored by national Democratic leadership. Any amount helps, no matter how small, but if you’re not a US citizen or permanent resident and can’t donate, at least you can help get the word out.

A few decades ago, I started became concerned about the national political landscape. Bush’s war on terror and the PATRIOT act were nightmares for freedom, and then Obama increased surveillance and drone strikes. At the same time, corporate lobbyists and PACs had grown to overrun the electoral process with their outsized influence.

The US is a rich country in funds and resources, but also in values. The Declaration of Independence puts it right up front: people have inherent rights that can’t be legislated away, and the government is there to secure those rights, drawing its power from the people. We don’t always live up to these values, but change has come upon us time and again with simple questions like, “am I not a man and a brother?”

We are now at a critical point. Ignore the distracting presidency — Republicans are openly backing corporations & the wealthy by driving up the deficit and dismantling the thin safety net we have. Establishment Democrats are disorganized and out of touch with their roots, siding with financial interests instead of the poor and middle class. It’s time for a change.

Given that there isn’t a strong third party, the best bet is to renovate the Democrats from the inside. Too often, they have run weak candidates in districts that were too small or rural for them to care about, or even let the race go unopposed.

This has been a mistake. There is a huge vacuum for truly populist, outsider candidates. For 2018, the Democratic establishment appears to be cruising on anti-Trump sentiment, hoping enough voters are energized by his antics to turn against their local Republicans. Their tired playbook of dumping tons of outside money into business-friendly candidates to annoy residents with nonstop TV ads late in the campaign is another mistake.

The good news is that first-time Democratic candidates are running that support common sense policies like access to healthcare, investing in infrastructure, and education. And, they’re focused on the basics: build a ground campaign that talks to voters and recruits volunteers to get out the vote. The community they build will outlast this one election. But since they’re running in districts that have been ignored by the Democratic organization, they are lacking basic funds.

More good news: we’re not talking a lot of money. Your $10 or $100 donation makes a huge difference when $2,000/month hires a staff member. I know many of you are engineers & entrepreneurs that can give even more.

If these outsider candidates raise enough money before the primaries next year, it gets very interesting. The national Democratic party will notice the schoolteacher that just wiped the floor with their perennial, bland contender, outraising them with individual donations. Once they’re in the spotlight, these candidates will attract much more funding going into the main election and win their races.

Time is running out for 2017 to get these candidates off to a good start, so please consider giving an amount that hurts a bit, because it will be too late if we wait for the Democratic leadership to figure it out on their own.

Was the past better than now?

~ Nate Lawson

Here we go again — another article arguing whether the past was better or not (this one says “better”). These articles are tiresome, rehashing the debate whether technology is enabling or isolating and dehumanizing. But I’m interested in a different line of technology criticism: which parts of technology are a regression and what to do about that.

From the first stone tools, technology has both reflected us and changed us. When we became farmers, we became less portable and vulnerable to robbers, and it was possible to measure capital for the first time via a land’s quality and location.

When evaluating today’s technology, I think it’s important to keep a flexible point of view and not be limited by a linear view of history. For example, what would digital cash look like today if we had adopted a 10-year land ownership rotation back then? A linear progression from good to bad (or bad to good) ignores a more nuanced view that focuses on the good and bad, leading to an understanding what we can do about it.

Even though I work with developing new technology every day, I’m reticent to adopt it until I have time or motivation to review it thoroughly. There are two main reasons:

  1. Advances in technology often come with critical regressions
  2. What you use changes yourself, your way of thinking, and what you believe to be possible

The microwave oven was a huge advance in heating speed, but you lost the key aspect of temperature control. It is still difficult to find one that allows you to heat food to a particular temperature. Instead, you have to guess at the combination of watts and time. Software is even more plastic. You can be using code written by a 20-year-old Javascript newbie for reviewing the intricacies of your personal genome. Calling this entire technology a step forward or back is much too simplistic, and it lets said programmer off the hook for not knowing their own history.

Computer history should be a mandatory part of the curriculum. I don’t mean dry facts like the date the transistor was invented or which CPU first implemented pipelining. I mean criticism of historical choices in software or system design, and an analysis of how they could have been done differently.

Here are some example topics to get you started:

  1. Compare the Mac OS X sandboxing architecture to the Rainbow Series. Which is more usable? Compare and contrast the feature sets of each. Create an alternate history where modern Unix systems had thrown out UIDs and built on a data-centric privilege model.
  2. In terms of installation and removal, how do users expect iOS and Android devices to treat mobile apps? How does this compare to Windows programs or Linux packages? What are the potential side effects (in terms of system or filesystem changes, network activity, etc.) of installing a program? Running it? Removing it?
  3. Some developers have advocated “curl | sh” as an acceptable installation method as a replacement for native packages. They argue that there is no loss of security compared to downloading and installing a native package from an uncertain origin. Compare the functionality and risks of “curl | sh” to both a common package system (e.g., Debian dpkg) and an innovative system (e.g., Solaris IPS), focusing on operations like installing a package for the first time, upgrading it, installing programs with conflicting dependencies, etc. What is truly being lost, if anything?

Good design and engineering involves knowing what has come before, so we can move forward with as little loss as possible. Engineers should learn more about what has come before to avoid repeating the mistakes of the past. The past wasn’t better than the present, but ignoring it makes us all worse off than we could have been.

Thought experiment on protocols and noise

~ Nate Lawson ~ 3 Comments

I hesitate to call this an interview question because I don’t think on-the-spot puzzle solving equates to a good engineering hire. On the other hand, I try to explore some simple thought experiments with candidates that have a security background.

One of these involves a protocol that has messages authenticated by an HMAC. There’s a message (with appropriate internal format) and a SHA-256 HMAC that covers it. As the implementer, you receive a message that doesn’t verify. In other words, your calculated MAC isn’t the same as the one in the message. What do you do?

“Throw an error” is a common response. But is there something more clever you could do? What if you could tell whether the message had been tampered with or if this was an innocent network error due to noise? How could you do that?

Some come up with the idea of calculating the Hamming distance or other comparison between the computed and message HMACs. If they are close, it’s unlikely that the message had been corrupted, due to the avalanche property of secure hash functions. If not, it may be a bit flip in the message, possibly due to an attack.

Ok, you can distinguish whether the MAC had a small number of errors or the message itself. Is this helpful, and is it secure? Consider:

  • If you return an error, which one do you return? At what point in the processing?
  • Does knowing which type of error occurred help an attacker? Which kind of attacker?
  • If you chose to allow up to 8 flipped bits in the MAC while still accepting the message, is that secure? If so, at what number of bits would it be insecure? Does the position of the bits matter?

There comes a moment when every engineer comes up with some “clever” idea like the above. If she’s had experience attacking crypto, the next thought is one of intense unease. The unschooled engineer has no such qualms, and thus provides full employment for Root Labs.

Timing-safe memcmp and API parity

~ Nate Lawson

OpenBSD released a new API with a timing-safe bcmp and memcmp. I strongly agree with their strategy of encouraging developers to adopt “safe” APIs, even at a slight performance loss. The strlcpy/strlcat family of functions they pioneered have been immensely helpful against overflows.

Data-independent timing routines are extremely hard to get right, and the farther you are from the hardware, the harder it is to avoid unintended leakage. Your best bet if working in an embedded environment, is to use assembly and thoroughly test on the target CPU under multiple scenarios (interrupts, power management throttling clocks, etc.) Moving up to C creates a lot of pitfalls, especially if you support multiple compilers and versions. Now you are subject to micro-architectural variance, such as cache, branch prediction, bus contention, etc. And compilers have a lot of leeway with optimizing away code with strictly-intended behavior.

While I think the timing-safe bcmp (straightforward comparison for equality) is useful, I’m more concerned with the new memcmp variant. It is more complicated and subject to compiler and CPU quirks (because of the additional ordering requirements), may confuse developers who really want bcmp, and could encourage unsafe designs.

If you ask a C developer to implement bytewise comparison, they’ll almost always choose memcmp(). (The “b” series of functions is more local to BSD and not Windows or POSIX platforms.) This means that developers using timingsafe_memcmp() will be incorporating unnecessary features simply by picking the familiar name. If compiler or CPU variation compromised this routine, this would introduce a vulnerability. John-Mark pointed out to me a few ways the current implementation could possibly fail due to compiler optimizations. While the bcmp routine is simpler (XOR accumulate loop), it too could possibly be invalidated by optimization such as vectorization.

The most important concern is if this will encourage unsafe designs. I can’t come up with a crypto design that requires ordering of secret data that isn’t also a terrible idea. Sorting your AES keys? Why? Don’t do that. Database index lookups that reveal secret contents? Making array comparison constant-time fixes nothing when the index involves large blocks of RAM/disk read timing leaks. In any scenario that involves the need for ordering of secret data, much larger architectural issues need to be addressed than a comparison function.

Simple timing-independent comparison is an important primitive, but it should be used only when other measures are not available. If you’re concerned about HMAC timing leaks, you could instead hash or double-HMAC the data and compare the results with a variable-timing comparison routine. This takes a tiny bit longer but ensures any leaks are useless to an attacker. Such algorithmic changes are much safer than trying to set compiler and CPU behavior in concrete.

The justification I’ve heard from Ted Unangst is “API parity“. His argument is that developers will not use the timing-safe routines if they don’t conform to the ordering behavior of memcmp. I don’t get this argument. Developers are more likely to be annoyed with the sudden performance loss of switching to timing-safe routines, especially for comparing large blocks of data. And, there’s more behavior that should intentionally be broken in a “secure memory compare” routine, such as two zero-length arrays returning success instead of an error.

Perhaps OpenBSD will reconsider offering this routine purely for the sake of API parity. There are too many drawbacks.

In Defense of JavaScript Crypto

~ Nate Lawson ~ 8 Comments

Thai Duong wrote a great post outlining why he likes JavaScript crypto, although it’s not as strong a defense as you might guess from the title. While he makes some fair points of some limited applications of JavaScript, his post is actually a great argument against those pushing web page JS crypto.

First, he starts off with a clever Unicode attack on JS AES by Bleichenbacher. It is a great way to illustrate how the language, with its bitwise and type hostility, actively works against crypto implementers. Though Thai points out lots of different ways to work around these problems, I disagree that it’s clear sailing for developers once your crypto library deals with these issues. You’ll get to pick up where your low-level library left off.

Oh, and those of you were looking for defense of web page crypto for your latest app? Sorry, that’s still dumb. Google’s End-to-End will only be shipped as a browser extension.

The most ironic part of Thai’s post involves avoiding PCI audit by shipping JS to the browser to encrypt credit card numbers. Back in 2009, I gave a Google Tech Talk on a variety of crypto topics. Afterwards, a Google engineer came up to me and gave exactly Thai’s example as the perfect use case for JavaScript crypto. “We avoid the web server being in scope for PCI audits by shipping JS to the user,” he said. “This way we can strip off the SSL as soon as possible at the edge and avoid the cost of full encryption to the backend.”

He went on to describe a race-condition prone method of auditing Google’s own web servers, hashing the JS file served by each to look for compromised copies. When I pointed out this was trivial to bypass, he said it didn’t really matter because PCI is a charade anyway.

While he’s right that PCI is more about full employment for auditors & vendors than security, news about NSA tapping the Google backbone also shows why clever ways to avoid end-to-end encryption often create unintended weaknesses. I hope Google no longer underestimates their exposure to nation-state adversaries after the Snowden revelations, but this use-case for JS crypto apparently hasn’t died yet.