It's 10 p.m. when this on-call pilot fish responds to a panicked call from a company executive: IT is losing control of all the remote sites and four servers are crashing.
"Arriving on site, I found out this was definitely the case," says fish. "To make things worse, we didn't manage this system. The people who put it in were away at a training course, and the people who should have been supporting the application weren't responding to any phones calls."
Meanwhile, there's a thunderstorm bearing down on the data center, and 250,000 people in the neighboring area are already without power.
Don't touch the system, fish's manager tells him over the phone. We need to wait until we hear back from the software support people.
What are you doing to solve this problem? executive asks fish. The system is still falling apart!
Fish makes a decision. He not-so-politely suggests to his manager that if he's not going to offer anything helpful, he should leave the people on the firing line alone.
"He then sent somebody in to try to calm me down," fish says.
"Then we ran and documented a procedure for what to do if this happens -- where to move everybody and how to keep things alive.
"After a sleepless night for us, the software support people called back and confirmed what we did was correct and sorted out the issue."
And fish's reward for his decisive -- and successful -- action? "Despite saving the day," he says, "I was told off by my manager for disobeying him."
Tell Sharky about it. Send me your true tale of IT life at sharky@computerworld.com. You'll snag a snazzy Shark shirt if I use it. Add your comments below, and read some great old tales in the Sharkives.
Now you can post your own stories of IT ridiculousness at Shark Bait. Join today and vent your IT frustrations to people who've been there, done that.