Who needs those itty bitty fields, anyhow?

This project involves statistical analysis of some massive government-collected data files -- so big that kicking unneeded fields out of the working database is a priority, reports a pilot fish in the know.

"Hotshot database analyst had the boss call a meeting so everyone could go through the file record documents, item by item," says fish. "Do we need this field? Do we need this one? Basically, any field no one spoke up for would get omitted from the big database load.

"The chief statistician on the project objected: 'Most of those fields we're eliminating are only one or two characters. If we don't know what they're used for, wouldn't it be safer to keep them?'

"His suggestion was, of course, brusquely overruled, and the useless little fields were left out.

"But these government data files were rather old -- they dated back to when storage was at a premium and many strange things were done to conserve space.

"Upon considerable further review, it turned out that the little one-character fields are switches. If this variable is a 1, the next field is, let's say, annual household income. If it's a 3, the next field is total Social Security benefits received. And so on.

"So omitting the little 'useless' fields made analysis impossible until they did a complete reload.

"But neither the boss nor the database analyst ever acknowledged that the chief statistician had been right."

Big stories, little stories, Sharky wants 'em all. So send me your true tales of IT life at sharky@computerworld.com. You'll snag a snazzy Shark shirt every time I use one. Add your comments below, and read some great old tales in the Sharkives.

Get your daily dose of out-takes from the IT Theater of the Absurd delivered directly to your Inbox. Subscribe now to the Daily Shark Newsletter.

To express your thoughts on Computerworld content, visit Computerworld's Facebook page, LinkedIn page and Twitter stream.
Related:
Windows 10 annoyances and solutions
Shop Tech Products at Amazon
Notice to our Readers
We're now using social media to take your comments and feedback. Learn more about this here.