Computerworld - I made a disconcerting discovery last month when Microsoft released a security patch outside of its regular monthly routine. How disconcerting was it? I'm talking about the discovery that a lot of the patching we've done to our more than 450 servers has never gone into effect.
Let me give you the background.
Each quarter, I present metrics to our CIO, including the percentage of servers that are compliant with the latest recommended critical security patches. Our Windows desktop and server team meets once a month to review the latest Microsoft patches and decide which ones we should apply based upon criteria we use to classify the threats. Patches we choose to apply are pushed to our servers and desktops, and then I check our Microsoft Systems Management Server to gather the information for the CIO.
I'd always assumed that when the SMS reported that a particular server had been patched, I could run with that information. Nope; there's a gotcha.
Issue: Over the years, hundreds of servers have been patched but never rebooted, so the patches haven't actually taken effect.
Action Plan: Take them all down on a Friday evening and keep our fingers crossed that no applications break.
If you don't reboot a Windows server after a patch is applied, the patch doesn't take effect, but SMS doesn't notice that failure to reboot. This insistence on rebooting is one of the things I dislike about Windows. In the Unix world, all that's usually required is that a particular process be restarted.
When that so-called out-of-band patch came up, I attended a meeting to discuss whether it would be necessary to install it on all of our servers and 8,000 desktops. From what I had read ahead of the meeting, I was convinced that it was absolutely necessary. The patch fixes a vulnerability in the "server service," which lets Windows communicate with network devices. A remote procedure call could cause a Windows system to execute arbitrary code, and reports were already circulating that exploit code and a worm had been developed. In other words, this was one vulnerability we couldn't afford to ignore.
It was during that meeting that I made my disconcerting discovery. The IT managers were freaking out that they would have to reboot over 450 servers. My first reaction to their concern was confusion. Making assumptions can get you into trouble, and my assumption that the servers were routinely rebooted after patches were installed certainly fit the bill. I found out, in fact, that many of our servers hadn't been rebooted in over two years. They had more than 24 months' worth of patches installed on them that were doing us no good.
Free Insider GuideCopyright © 1994 - 2012 Computerworld Inc. All rights reserved. Reproduction in whole or in part in any form or medium without express written permission of Computerworld Inc. is prohibited. Computerworld and Computerworld.com and the respective logos are trademarks of International Data Group Inc.
