Pilot fish's phone rings, and it's That Analyst again -- the one who won't open a help desk ticket like everyone else. His complaint this time? His server is sending alerts.
"I checked our monitoring software, and indeed the server hosting a particular piece of enterprise software was glowing red, including the global zone," says fish. "'Let me see what's going on', I told him."
Fish logs into the box, but it takes an awfully long time. And no wonder: Once he gets in, he sees that the global zone shows 128GB total, but with only 968MB free. And the top process is one that's currently using 28GB, leaving the local zone with less than 400MB to spare.
Isn't memory for this supposed to be capped? fish thinks. But that does explain why all the services are alerting. Fish is hesitant to just kill the process, but he does reply to the email thread that has just surfaced among the IT team members, laying out what he knows and adding that the server may be thrashing because it doesn't have enough memory to run properly.
But to his surprise, a few minutes later the oh-so-hungry process's memory usage drops down to 650MB, and all the alerts begin to go away after fish refreshes them.
Then he checks email again. There's a new message thread from That Analyst, saying there was an "unexpected user query surge," though he disavows any responsibility for the problem.
Gee, fish thinks, I must have the magic touch -- after my thought about the process taking too much memory, this willed the process to relinquish its memory usage.
Or maybe not. "I checked the last few commands that were executed for that application's admin account," fish says. "I suspected that an administrative command that ran a few minutes after I posted my email comment had more to do with it than my magic touch.
"Then I remembered That Analyst called a few months ago about the same issue. I guess he finally remembered my suggestion about tuning the app."
Remember Sharky -- I still need your true tale of IT life. Send it to me at sharky@computerworld.com. You'll snag a snazzy Shark shirt if I use it. Add your comments below, and read some great old tales in the Sharkives.
Get your daily dose of out-takes from the IT Theater of the Absurd delivered directly to your Inbox. Subscribe now to the Daily Shark Newsletter.