Everyone at this organization walks in one morning to discover that there's a total network outage -- nothing is connecting to anything, according to an IT pilot fish who's as networkless as the rest.
"A war-council meeting was quickly convened," fish says, "with half the managers in the building arguing about what they thought might be the problem, and about half as many technical people suggesting things that might need to be checked."
Early on, one manager suggests that, since all the individual pieces of hardware are turned on and appear to be working fine, the problem must be in the network itself.
But that suggestion gets a vehement denial from a network tech, who insists that there's no possibility that anything could take down the network because all the hardware is up and running fine.
The techs go hunting for the cause -- and six hours later, someone remembers the exchange and decides to check anyway.
Turns out that the network got some hardware upgrades over the weekend, and when everything was powered back up, the config files for the DNS server were missing some information -- a lot of information.
"It seems that all the various URLs for the network were entered manually about a year earlier, but no one ever saved them to the DNS config file," sighs fish. "So when the network came back up after the power down, 90 percent of the network didn't exist as far as the DNS server was concerned.
"Six months later, the same thing happened after another hardware upgrade -- except this time, it only took 30 minutes for someone to remember what happened before, and the admin promised to save the config info this time."
Sharky's pretty sure it won't take you 30 minutes to tell your story. So send your true tale of IT life to me at firstname.lastname@example.org. You'll snag a snazzy Shark shirt if I use it. Comment on today's tale at Sharky's Google+ community, and read thousands of great old tales in the Sharkives.
Get your daily dose of out-takes from the IT Theater of the Absurd delivered directly to your Inbox. Subscribe now to the Daily Shark Newsletter.