NTP: the rebirth of ailing, failing core network infrastructure

Network Time Protocol is how the computers you depend on know what time it is (this is critical to network operations, cryptography, and many other critical functions); NTP software was, until recently, stored in a proprietary format on a computer that no one had the password for (and which had not been updated in a decade), and maintained almost entirely by one person.

NTP's precarity was the whole internet's problem: NTP servers could be used to amplify denial of service attacks with devastating consequences, and failures in NTP endangered the many systems that depended on it.

The Internet Civil Engineering Institute and Indiana University's Center for Applied Cybersecurity Research assumed control over the project, undertaking a massive refactoring of the code, its maintenance infrastructure, the organization around it, and its culture and norms.

The result was a new version of NTP, robust, secure, and stable, with a long-term, sustainable future ahead of it.

Susan Sons led the NTP project, and she sat down with O'Reilly's Mac Slocum to discuss it. I really recommend watching it and supporting her work.

O'Reilly's Mac Slocum speaks with Susan Sons, Senior Systems Analyst at the Center for Applied Cybersecurity Research at Indiana University. They discuss:

Why Susan gravitated toward security (it all started when she broke into a computer at four years old). (00:04)

Her work to fix the Network Time Protocol (NTP), an essential part of the Internet's infrastructure. "It was just a moment of panic," she said, recalling her first evaluation of the NTP. "The Internet is going to fall down if I don't fix this." (01:53)

How the Internet's infrastructure can remain up to date and secure. (05:34)

How organizations can balance security with the need to move quickly. (09:43)

The single most important security issue we're facing is the battle between sound bites and first principles. (12:21)

The people and projects she's following. (16:49)

Notable Replies

  1. An important and interesting topic.

    Thanks for posting this!

  2. We all use it! Even though most of us don't know it.

  3. How are we supposed to take this person seriously when she's not talking about monetising NTP, securitising it and slicing it into tranches, and "disrupting" time itself to Make the World a Better Place(tm)?

    [seriously, what @davide405 said]

  4. I would say the biggest problem with NTP is that people keep reinventing this particular wheel (yes, I'm looking at you, Microsoft) and doing it badly. In that regard Susan Son's project, which is a fork of the NTP reference implementation, will probably be a major improvement.

    Until ICEI and CACR got involved with NTP, it was supported by one person, part time, who had lost the root passwords to the machine where the source code was maintained (so that machine hadn't received security updates in many years), and that machine ran a proprietary source-control system that almost no one had access to, so it was very hard to contribute to it.

    The "proprietary source-control system" mentioned would be BitKeeper, so not exactly RCS ifyouknowwhatImsayin. Moving code in and out of BitKeeper is pretty simple and well understood, it's what Linus Torvalds used for the Linux kernel until Tridge pissed off Larry McVoy.

    NTP was designed and implemented by David Mills, who also invented the fuzzballs as well as the family of exterior gateway protocols that has since grown into BGP, the most important global routing protocol of the Internet. It's not a big stretch to say he made the modern Internet possible, and he's still kind of the Big Kahuna of network time synchronization, despite his age and fading eyesight.

    I'm pretty certain Dr. Mills has never been formally responsible for security patching the University of Delaware's servers, although I suppose I could be wrong. I wouldn't expect him to have root access since his partial retirement in 2009, anyway, so somebody else would be responsible for patching today.

    I run the reference implementation, patched to the 2016-11-21 release. I haven't seen any pressing reason to move to NTPsec.

    Chrony's fine for laptops, I suppose, if you want more accuracy than OpenNTPd. As you've noted, most people don't need a datacenter grade time server, they just want a reasonably efficient NTP client running in the background.

  5. Maybe this is a good place to mention this?

    Don't run NTP clients on virtual machines. Seriously. Don't. It just mucks up your logs.

    Run a solid, efficient NTP client on each of the hypervisor nodes hosting the virtual machines, and let the VMs get their time from the fake hardware clock that exists as part of their virtualized environment.

    If you control large networks, take the time to design a real NTP hierarchy, with a few canonical servers (one per site usually suffices) that sync to high value servers on the Internet (such as NIST), and everything else syncing from them.

Continue the discussion bbs.boingboing.net

20 more replies

Participants