To err is human, to foul up big-time requires SOP

Flashback to 1975, when this pilot fish gets his first IT job as the night computer operator for a regional savings and loan.

"In those days, when dinosaurs roamed the earth and computers filled rooms, we could not provide both transaction processing and teller information at the same time," says fish.

"My job was to show up at four in the afternoon, wait for all the branch tellers to close out their transactions, and begin processing the day's transactions. This usually kept me busy until two or three in the morning."

Processing transactions involves loading punch cards for the program, tape for the transaction file and disks for account data. The disks come in packs the size of a cake box that are loaded into refrigerator-size drives -- and as fish soon learns, they're unreliable enough to show at least one error per night.

There's a standard workaround, though: When a disk pack shows an error, fish has been told to swap it to another drive and restart the job. No one knows whether dust gets blown off or the heads readjust themselves or what, but that usually solves the problem.

But one busy night when fish sees a read/write error on a disk pack, most of the drive units are busy, so there's no empty drive to move it to. OK, fish figures, I'll just swap it with another pack on the same job.

He does, and restarts -- and soon he's showing two read/write errors.

That's unusual, but not unheard-of. All fish has to do is swap for two other drives. Result: In short order he's got four read/write errors.

Fish starts to think it's just not his night. But he knows the standard procedure, so he switches two of the packs to two new drives. Those fail too.

"And so I was introduced to the concept of the head crash," fish says.

"But that wasn't all. This took place on a Thursday night. Backups were made regularly -- on Friday night. So we had to go back to the end of the previous week and rerun all the transactions, a four- or five-hour process for each day. And the computer couldn't be used for teller transactions while it was rerunning the previous days' processing.

"In the end, after a really long weekend, we were back up Monday morning with current information. Plus we had a new standard procedure: Backups were to be run every night, both to disk and to tape, in duplicate for on-site and offsite storage. And no more multiple drive swaps."

Feed the Shark! Send me your true tale of IT life at You'll snag a snazzy Shark shirt if I use it. Add your comments below, and read some great old tales in the Sharkives.

Get your daily dose of out-takes from the IT Theater of the Absurd delivered directly to your Inbox. Subscribe now to the Daily Shark Newsletter.

Computerworld's IT Salary Survey 2017 results
Shop Tech Products at Amazon