Time Isn’t Always On Our Side in IT

This year’s early daylight-saving time was a mini-Y2k crisis. Our manager scrambles and comes out OK, again.

Every once in a while, we find ourselves playing the IT version of Beat the Clock. This game involves the clocks in our computers. They always do as they’re told, but sometimes we haven’t given them enough information. Two cases in point: Y2k, and this year’s scramble to update devices that didn’t know that daylight-saving time would be instituted on March 11, three weeks earlier than the date they had been programmed to expect.

It seems like only yesterday that I was inventorying my company’s security infrastructure, contacting vendors and applying Y2k patches. On Dec. 31, 1999, while all my friends were out celebrating the new year, I was with the rest of the IT department, waiting for the clock to turn and the anticipated catastrophe. When the clock struck midnight, nothing happened. All systems were normal. We had a small office celebration (the CEO provided pizza).

This year’s DST crisis was Y2k on a smaller scale. At our company, we took a serious approach, since not only our IT systems, but also the tools that the company makes, depend on the correct time.

In the old days, systems administrators had to manually set clocks when DST arrived in the spring and when it ended in autumn. Nowadays, most modern operating systems have an internal mechanism for ensuring that the time is correct, including an automatic switch to DST. This system is called Network Time Protocol, or NTP. However, some applications don’t use the system’s internal clock for time synchronization. For example, certain versions of Sun Microsystems’ Java Runtime Environment (JRE) have their own implementation of time zones and DST rules. In some cases, we needed patches from vendors to ensure that the DST timetables were updated in specific applications. Without them, as of March 11, an application like the JRE and the Java Virtual Machine would have been out of sync with the operating system clock and other services. The results might have ranged from incorrect time stamps to application failures.

We hired a dedicated project manager for the DST job. His first step was to brainstorm with as many IT people as possible to come up with a comprehensive list of the systems that would have to be reviewed for DST compliance. Unfortunately, the company lacks a robust asset-inventory tool to track all IT assets.

Many of our Unix servers run applications that use the JRE. We had to verify that every server was DST-compliant. This included the operating system, JRE and any other third-party or custom application on the servers.

Accurate timekeeping is essential in the IT security department as well. Take RSA Security’s SecurID system as an example. We use SecurID tokens extensively — for administrative access to critical servers, remote access for employees, and internal access for partners and suppliers. The tokens generate a new code every minute, and time synchronization with the central authentication server is crucial. If the time is off by even a minute — let alone an hour — the token won’t properly authenticate a user. Fortunately, the vendor assured us that our infrastructure was DST-compliant. Only the system clock needed to be changed, and that was easy, since that system uses NTP.

Timing Is Everything

Tripwire is another time- sensitive application. We use Tripwire as a compensating control for Sarbanes-Oxley Act compliance and as an intrusion-detection and configuration management tool for our critical servers. It is imperative that when Tripwire detects a change to a critical system file, the details, including the time stamp, are valid. (This isn’t really a matter of functionality, but correct time stamps would be very important if we were gathering evidence for a law enforcement agency.) Fortunately, only the Tripwire Management console uses JRE technology, and we didn’t have to touch the agents running on the systems we monitor. Tripwire provided a patch, which we easily installed.

We also looked at our Juniper Networks Intrusion Detection and Prevention (IDP) infrastructure. Here, too, time-based services are important, and logs and time-based rules would be affected by the early time changeover. As with Tripwire, IDP would operate without a DST patch, but the accuracy of the reports would be questionable. Like most security managers, I don’t like leaving things to chance, and so I availed myself of the official DST project and made sure that all of our security infrastructure was attended to.

The DST project even gave us the opportunity to install some recommended patches to our Solaris operating system. And although physical security isn’t my direct responsibility, I contacted the manager of physical security to ensure that all systems under his purview were also reviewed for DST compliance. I’m glad I did, because it turned out that the computers that run the proximity door sensors needed to be patched. Without patches, the logs of contractor access to our buildings would be off by an hour. Again, that’s something that would be confusing in an investigation.

The camera systems were also time-sensitive and needed to be patched. We couldn’t leave any stone unturned, and the list of assets that needed to be secured kept growing beyond what was determined in that initial brainstorming exercise with the project manager. In the end, firewalls, VPN concentrators and event management systems were among the things we reviewed and applied patches to.

When March 11 arrived, things went off without a hitch. Just as with Y2k, all systems were normal. If only I could say the same about the findings from our recent security risk assessment. I’ll tell you all about that next time.

What Do You Think?

This week’s journal is written by a real security manager, “Mathias Thurman,” whose name and employer have been disguised for obvious reasons. Contact him at mathias_thurman@yahoo.com, or join the discussions in our security blogs: computerworld.com/blogs/security.

To find a complete archive of our Security Manager’s Journals, go online to computerworld.com/secjournal.


Copyright © 2007 IDG Communications, Inc.

Shop Tech Products at Amazon