The monitoring console for this data center is lit up like a Christmas tree when this IT pilot fish arrives at work one morning.
"We found out there was a power outage," fish reports. "The UPS did kick in, and so did the generator, but something happened. Fortunately, the facilities people were on it right away."
Next step is bringing the servers back up in the correct sequence. But while some of the servers are managed by fish and his team, some have been contracted out -- and while the contractor has root credentials necessary to bring those servers back up, fish's team doesn't.
And nothing's happening with those servers. What's going on? fish asks a long-time IT guy. Didn't the alert go out to the contractor as well? Why didn't they jump on it?
Yes, he's told, they got the alert. But they need to have a help desk ticket opened before they act.
Two hours later, fish sees the contractor amble in. Do you have the root password? "Yes," the contractor says. "I'll login." He does, but fish's systems still can't connect.
"Have you tried pinging?" contractor asks. I just tried that, fish replies, but the monitor console still shows the machines are down. Why don't you try seeing if there's an IP address assigned?
"Wait," contractor says, "I need a ticket open first."
Fish stifles his urge to respond, What the @#$%!, these boxes are critical and instead says, OK, I'll go back to the office and open one.
"I was in the office, an hour after opening the ticket, when I saw the contractor and our IT director walk by," says fish. "Hey, is the server up? I asked the contractor. 'No, I haven't seen the ticket yet,' he said. 'I haven't returned to my office to check email.'
"I showed him the ticket opened an hour before on my screen. An hour later, the machines were finally up.
"I asked the IT director, has it always been this way? He just put up his hands and winced."
Don't make Sharky wait for your true tale of IT life -- send it to me right now at sharky@computerworld.com. You can also comment on today's tale at Sharky's Google+ community, and read thousands of great old tales in the Sharkives.
Get Sharky's outtakes from the IT Theater of the Absurd delivered directly to your Inbox. Subscribe now to the Daily Shark Newsletter.