Flashback to the 1980s, when this enlisted military pilot fish is stationed in a telecommunications center at a major military headquarters.
"We were running a 20-year-old telecommunications system and it was time for the first major upgrade in a decade," says fish. "After running it on an 'identical' system elsewhere, it was time to upgrade our live systems."
Fish's unit is supporting a major intelligence agency, so it's a very high visibility project. On go-live day, there are even personnel from D.C. on site.
Working according to the upgrade plan, fish's team brings the backup system down, installs the update and starts bringing it up as the active system.
It will take five or six minutes to bring the system up and start receiving message traffic, so as the upgraded system is coming up, fish quickly starts the process to bring the original system back up to its final start prompt.
"I was running back and forth with 9-track tapes and entering boot commands," fish says. "Someone asked what the hell I was doing. I said that I was preparing to switch back if there was a problem."
The observers scoff, but fish finishes up his startup work.
The upgraded system comes up, receives its first message -- and immediately and loudly crashes.
Fish answers the final prompt on the other system. Two minutes later, the site is back up receiving traffic on the old version just two minutes after the crash.
"We were required by regulation to formally report on outages of ten minutes or longer. We were offline less than nine minutes," says fish.
"Nobody said a word, though I did get a couple of pats on the back."
Help keep Sharky up and running. Send me your true tale of IT life at firstname.lastname@example.org. You'll score a sharp Shark shirt if I use it. Add your comments below, and read some great old tales in the Sharkives.
Get your daily dose of out-takes from the IT Theater of the Absurd delivered directly to your Inbox. Subscribe now to the Daily Shark Newsletter.