September 23, 2013

20 Years of Internet

Filed under: Protocols,Security — Nate Lawson @ 5:00 am

This month marks my 20th anniversary of first connecting to the Internet. It seems like a good time to look back on the changes and where we can go from here.

I grew up in a rural area, suspecting but never fully realizing the isolation from the rest of the world, technology or otherwise. Computers and robots of the future lived in the ephemeral world of Sears catalogs and Byte magazines rescued from a dumpster. However, the amateur radio and remote-controlled plane hobbies of my father’s friends brought the world of computing and electronics to our house.

Still, communications were highly local. The VIC-20 could connect to a few BBS systems and my father’s industrial control of warehouse refrigeration systems (way before SCADA). However, anything beyond that incurred long distance charges and thus was irrelevant. Only the strange messages and terminology in cracked games, distributed from faraway places like Sweden, hinted at a much broader world out there.

Towards the end of high school, our local BBS finally got a FidoNet connection. Text files started trickling in about hacking COSMOS to change your “friend’s” phone service and building colored boxes to get free calls. One of those articles described how to use the Internet. I’d spend hours trying to remember all the protocol acronyms, TCP port numbers, etc. The Internet of my imagination was a strange amalgamation of X.25, ARPA protocols, TCP/IP, and the futuristic OSI protocols that were going to replace TCP/IP.

Once I arrived at college, I was one of the first in line to register for an Internet account. Our dorm room had an always-on serial connection to the campus terminal server and Ethernet was coming in a few weeks. It took some encouraging from my friends to make the jump to Ethernet (expensive, and 10BASE-T was barely standardized so it was hard to figure out if a given NIC would even work). Along with free cable TV, you’ve got to wonder, “what were they thinking?”

The dorm Ethernet experiment soon became a glorious free-for-all. There was a lot of Windows 3.1 and Linux, but also a few NeXTSTEP and Sun systems. Campus network admin had its hands full, bungling rushed policy changes intended to stop the flood of warez servers, IPX broadcast storms from Doom games, IRC battles, sniffing, hacking, and even a student running a commercial ISP on the side. Life on the dorm network was like a 24/7 Defcon CTF, but if you failed, you were reinstalling your OS from 25 floppies before you could do your homework.

There were three eras I got to see: Usenet (ending in 1994), early Web (1994-1997), and commercial Web (1998 to present). The Usenet era involved major changes in distributed protocols and operating systems, including the advent of Linux and other free Unixes. The early Web era transitioned to centralized servers with HTTP, with much experimentation in how to standardize access to information (remember image maps? Altavista vs. Lycos?) The commercial Web finally gave the non-technical world a reason to get online, to buy and sell stuff. It continues to be characterized by experimentation in business models, starting with companies like eBay.

One of my constant annoyances with technological progress is when we don’t benefit from history. Oftentimes, what comes along later is not better than what came before. This leads to gaps in progress, where you spend time recapitulating the past before you can truly move on to the predicted future.

Today, I morn the abandonment of the end-to-end principle. I don’t mean networking equipment has gotten too smart for its own good (though it has). I mean that we’re neglecting a wealth of intelligence at the endpoints and restricting them to a star topology, client/server communication model.

Multicast is one example of how things could be different. Much of the Internet data today is video streams or OS updates. Multicast allows a single transmission to be received by multiple listeners, building a dynamic tree of routes so that it traverses a minimal set of networks. Now, add in forward error-correction (allows you to tune in to a rotating transmission at any point in time and reconstruct the data) and distributed hash tables (allows you to look up information without a central directory) and you have something very powerful.

Bittorrent is a hack to leverage an oversight in the ISP pricing model. Since upload bandwidth from home broadband was underutilized but paid for, Bittorrent could reduce the load on centralized servers by augmenting them with users’ connections. This was a clever way to improve the existing star topology of HTTP downloads but would have been unnecessary if proper distributed systems using multicast were available.

We have had the technology for 20 years but a number of players have kept it from being widely deployed. Rapid growth in backbone bandwidth meant there wasn’t enough pricing pressure to reduce wastefulness. The domination of Windows and its closed TCP/IP stack meant it was difficult to innovate in a meaningful way. (I had invented a TCP NAT traversal protocol in 1999 that employed TCP simultaneous connect, but Windows had a bug that caused such connections to fail so I had to scrap it.) There have been bugs in core router stacks, and so multicast is mostly disabled there.

Firewalls are another symptom of the problem. If you had a standardized way to control endpoint communications, there would be no need for firewalls. You’d simply set policies for the group of computers you controlled and the OS on each would figure out how to apply them. However, closed platforms and a lack of standardization mean that not only do we still have network firewalls, but numerous variants of host-based firewalls as well.

Since the late 90’s, money has driven an intense focus on web-based businesses. In this latest round of tech froth, San Francisco is the epicenter instead of San Jose. Nobody cares what router they’re using, and there’s a race to be the most “meta”. Not only did EC2 mean you don’t touch the servers, but now Heroku means you don’t touch the software. But as you build higher, the architectures get narrower. There is no HTTP multicast and the same-origin policy means you can’t even implement Bittorrent in browser JavaScript.

It seems like decentralized protocols only appear in the presence of external pressure. Financial pressure doesn’t seem to be enough so far, but legal pressure led to Tor, magnet links, etc. Apple has done the most of anyone commercially in building distributed systems into their products (Bonjour service discovery, Airdrop direct file sharing), but these capabilities are not employed by many applications. Instead, we get simulated distributed systems like Dropbox, which are still based on the star topology.

I hope that the prevailing trend changes, and that we see more innovations in smart endpoints, chatting with each other in a diversity of decentralized, standardized, and secure protocols. Make this kind of software stack available on every popular platform, and we could see much more innovation in the next 20 years.


  1. Great article, Nate. I couldn’t agree more.

    Comment by crd — September 23, 2013 @ 4:51 pm

  2. Hah, as if software is the problem holding this back. The average user has no idea what their computer is really doing–and they shouldn’t have to. Maybe a good firewall UI for a cryptographically-verified control group and IPv6 will solve this. If you want this innovation to happen you need a culture that will support it and use cases to drive it. You need *folklore* (which you are providing pieces of here). My politics was early on informed by the efficiency of the thrush of kazaa and ed2k and slsk and dc++, but for some reason only just now are developers–not the general public, *developers*–getting hyped about p2p networks (hola @ git; of course, most of them use git through github >.>). I really can’t see people who are excited to come home and “read my tumblr” getting jiggy with the mesh network chacha. Hell, on that note, the branding that twitter and tumblr have is so strong that even if–even if–NCSA Mosaic had put in that edit button there would still be giant cloud blog servers with particular hives of culture attached.

    Maybe if Zeroconf actually worked reliably people would connect to each other instead of using well-known endpoints (like dropbox.com/u/dadada) to talk to each other. I guess the new fad with smartphones is local NFC search, but I think it’s telling that it has to be a whole new hardware protocol for people to be willing to give it an experimental twirl.

    tl;dr: branding and culture are the driving factors in tech rollout, not technical elegance.

    Comment by Nicknick — September 24, 2013 @ 11:22 pm

    • I’m a strong believer in an individual’s ability to make a big difference in software architecture. Examples include Van Jacobsen saving TCP from the dust heap, Sean Fanning’s Napster split of metadata lookup and bulk data transfer, and djb’s qmail & EC cryptosystems. I have a few personal examples that I won’t cite to avoid self-aggrandizement.

      Instead of attributing the lack of adoption to “culture”, just go out there and build something that becomes hugely popular.

      Comment by Nate Lawson — December 27, 2013 @ 3:19 pm

  3. This brings back some memories.

    Comment by ec — September 26, 2013 @ 7:53 am

  4. What a lovely article. Thanks!

    Comment by Zooko Wilcox-O'Hearn — September 26, 2013 @ 7:19 pm

RSS feed for comments on this post.

Blog at WordPress.com.