Flashback to the late 1990s, when this big aerospace/defense contractor acquires a competitor -- although it takes a few years for them to convert to a common manufacturing system, according to a pilot fish on the scene.
And when they do, there's big trouble. "Immediately after the conversion, the Bill of Materials (BOM) parts expansion ran for days on the mainframe," fish says. "There was no obvious cause. The data center people would kill the process, assign it a high priority, ensure there was plenty of temporary space, and rerun. After several days, they'd kill it again.
"The only thing that had changed from the last successful run was the migration of the acquired division's assemblies. This was becoming critical, since the old system was shut down and they couldn't perform the Materials Requirements Planning and inventory purchases."
That's when fish -- who supported the old system before the acquisition -- is brought in. He knows that, in both the old and new systems, a basic BOM contains a list of subassemblies or individual parts, along with the quantities of each of those needed to build the assembled product.
And he knows that, in the old system, documentation also has the same part number as the basic assembly -- there's just a flag to indicate what kind of documentation it is, such as a D for a wiring diagram or I for assembly instructions.
It doesn't take long to figure out that the people bringing the BOM data into the new system hadn't bothered to consult with fish or the old system's developers, and weren't aware of that quirk.
The result: Say a radio's BOM consists of a chassis, knobs and "radio (assembly instructions)." Without the flag indicating that the third item is documentation, on the new system the BOM looks like a chassis, knobs and radio.
And the BOM expansion turns into infinite recursion as the system tries to resolve the part "radio" by treating it like a subassembly and performing a BOM expansion on that.
"As a result, I had to perform data analysis to find all assemblies that contained themselves as a part," says fish. "It wasn't simple, because the types of documentation varied for each assembly, from nothing in some cases to many types for more complex ones. Then I created transactions to change the 'and radio' to 'and radio-DOCUMENTATION-TYPE.'
"In order to get that done, entire mainframe drives were assigned for my use, along with top-priority access to the high-priority batch queue. I ended up on a first-name basis with the console operators."
My name is Sharky, and I need your story. So send me your true tale of IT life at email@example.com. You'll snag a snazzy Shark shirt if I use it. Add your comments below, and read some great old tales in the Sharkives.
Get your daily dose of out-takes from the IT Theater of the Absurd delivered directly to your Inbox. Subscribe now to the Daily Shark Newsletter.